sakserv/hadoop-mini-clusters

Oozie sharelib location issue

Opened this issue · 1 comments

zied2 commented

Hello @sakserv,

Thanks for this powerfull tool which makes easy to build testing tools around hadoop ecosystem.

I encounter a problem during Oozie workflow launching, I think Oozie does not locate the sharelib which is installed on /user/root/share/lib/oozie/lib_20181217163941/share/lib

OOZIE: Writing share lib contents to: /user/root/share/lib/oozie/lib_20181217163941/share/lib

Oozie workflow logs:

ACTION[0000000-181217163855317-oozie-root-W@shell-node] Error starting action [shell-node]. ErrorType [FAILED], ErrorCode [It should never happen], Message [File /user/root/share/lib does not exist]
org.apache.oozie.action.ActionExecutorException: File /user/root/share/lib does not exist
	at org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForAction(JavaActionExecutor.java:785)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActionExecutor.java:863)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(JavaActionExecutor.java:854)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1099)
	at org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1379)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:234)
	at org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:65)
	at org.apache.oozie.command.XCommand.call(XCommand.java:287)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:331)
	at org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:260)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:178)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

The share lib may be copied into /user/root/share/lib/lib_20181217163941 instead of /user/root/share/lib/oozie/lib_20181217163941/share/lib maybe ?

Regards,
Zied

sakserv commented

Sorry for the delay. Would it be possible to share the code (and the properties file)? It appears to me like Oozie is looking for the sharelib directory on the local filesystem vs HDFS. "/user/root/share/lib" is a HDFS path.