GoogleCloudPlatform/oozie-to-airflow

Support for Oozie lib .jar files

potiuk opened this issue · 0 comments

Is your feature request related to a problem? Please describe.
Ooozie adds all the .jar files which are in the lib folder of the workflow application to the classpath automatically. This is currently only implemented in Java Mapper.

Describe the solution you'd like
All the mappers should have .jar files added to the classpath.

Describe alternatives you've considered
No alternatives.

Additional context
Each Oozie operator has its own way of adding jars to classpath so it should be done in mapper-specific way. Currently in workflow.py the lib folder is scanned for all .jar files and they are stored in Workflow.

For all the Map-Reduce jobs that are not executed as single-node mapper (on master), the .jar files should be added to Local Cache.