mapreduce - Hadoop post-hook and job completation notification -


i import hadoop job output hive table. how implement post-hooking in map-reduce job/flow? or other automated options?

also notification after job done, such sending email user. found this: https://issues.apache.org/jira/browse/hadoop-1111, don't quite understand how since i'm new map-reducing.

thanks.

conf.set("mapreduce.job.end-notification.url","url")

would do. url should http url receive callback.

from javadocs :

set uri invoked in-order send notification after job has completed (success/failure).

the uri can contain 2 special parameters: $jobid , $jobstatus. those, if present, replaced job's identifier , completion-status respectively.

this typically used application-writers implement chaining of map-reduce jobs in asynchronous manner.

note older hadoop versions use job.end.notification.url.
has been deprecated in newer versions in favour of mapreduce.job.end-notification.url.

reference mapred-default.xml#mapreduce.job.end-notification.url.


Comments