XiaoMi/minos

add HADOOP_YARN_HOME for %service_env in deploy_yarn.py

Opened this issue · 1 comments

hpttlook commented

Cloudera CDH4 is based on apache hadoop 2.0.0 version. Currently the latest version of hadoop is apache hadoop 2.2.0, big differences are exists between this two versions. In my test with minos + apache hadoop 2.2.0, the mapreduce job always failed due to some error like, "cannot find the main class xxxx, ClassNotDefined Exception".

After some debug things, it is proved the problem is caused by %service_env.

In hadoop 2.0.0, the follwing environment variables are used,
HADOOP_HDFS_HOME
HADOOP_COMMON_HOME
YARN_HOME

But in hadoop 2.2.0, the following variales are used
HADOOP_HDFS_HOME
HADOOP_COMMON_HOME
HADOOP_YARN_HOME
HADOOP_MAPRED_HOME,

so adding HADOOP_YARN_HOME and HADOOP_MAPRED_HOME to service_env makes it works for both CDH4 and apache hadoop 2.2.0 versions.

wuzesheng commented

@renozhang Can you make sure of this issue?
BTW: Thanks @hpttlook