Kylin installation of questions to consult

Question

Kylin installation of questions to consult

blake-liu opened this issue 10 years ago · 1 comments

你好：
我最近在尝试使用kylin做为项目中大数据分析工具，搭建环境的时候碰到一个问题，请您帮忙分析一下。
在cloudera manager或者yum安装的hadoop环境中，kylin0.7没有问题，但在tar包解压安装的hadoop环境中kylin不能正常启动。指定find-hive-dependency.sh脚本中的hive_conf_path和hive_exec_path环境变量后，可以启动，但导入hive表功能时不能导入且日志不报错；build cube任务时出现错误，错误详见附件。
kylin使用的是下载的二进制包。http://kylin.incubator.apache.org/download/kylin-0.7.1-incubating-SNAPSHOT.tar.gz
操作系统是CentOS6.5
附件是错误信息。
以下是tar包安装的环境变量配置。

我已经在周四的时候发邮件到dev@kylin.incubator.apache.org，但是没有回复。

Hello:
I recently tried to use kylin in my project, when running kylin,i got some ploblems, please help me analyze it.
In the cloudera manager installation or yum installation, kylin0.7 runs well, but in the tar package to extract the Hadoop environment kylin can not normally start. After specifying hive_conf_path and hive_exec_path environment variable in find-hive-dependency.sh script ,kylin can start, but i got some errors when using "load hive table" and "build Cube action" features in the email attachments.
Binary kylin package download here: http://kylin.incubator.apache.org/download/kylin-0.7.1-incubating-SNAPSHOT.tar.gz
The operating system is CentOS6.5
Attachment is error message.

The following is the environment variable configuration for the tar package installation.

BASE_PATH=/opt/hadoop/soft

JAVA_HOME=$BASE_PATH/jdk1.7.0_75
JRE_HOME=$JAVA_HOME/jre
PATH=$JAVA_HOME/bin:$PATH
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

HADOOP_HOME=$BASE_PATH/hadoop-2.5.0-cdh5.3.0
HBASE_HOME=$BASE_PATH/hbase-0.98.6-cdh5.3.0
SPARK_HOME=$BASE_PATH/spark-1.2.0-cdh5.3.0
HIVE_HOME=$BASE_PATH/hive-0.13.1-cdh5.3.0
KYLIN_HOME=/opt/kylin-0.7.1-incubating-SNAPSHOT

PATH=$HADOOP_HOME/bin:$HBASE_HOME/bin:$SPARK_HOME/bin:$HIVE_HOME/bin:$PATH
export JAVA_HOME JRE_HOME HADOOP_HOME HBASE_HOME SPARK_HOME HIVE_HOME PATH CLASSPATH

Attachment:

+------------------------------------------------------------------------------------------------------+
| Extract Fact Table Distinct Columns |
+------------------------------------------------------------------------------------------------------+
[pool-7-thread-2]:[2015-05-21 10:29:27,474][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/0d805dfd-9a6b-4617-9995-adb4b89106b3-01 (Store kylin_metadata@hbase)
[pool-7-thread-2]:[2015-05-21 10:29:27,483][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:222)] - job id:0d805dfd-9a6b-4617-9995-adb4b89106b3-01 from READY to RUNNING
[pool-7-thread-2]:[2015-05-21 10:29:27,509][INFO][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:107)] - parameters of the MapReduceExecutable:
[pool-7-thread-2]:[2015-05-21 10:29:27,510][INFO][org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:108)] - -conf /opt/kylin-0.7.1-incubating-SNAPSHOT/conf/kylin_job_conf.xml -cubename kylin_sales_cube -output /tmp/kylin-0d805dfd-9a6b-4617-9995-adb4b89106b3/kylin_sales_cube/fact_distinct_columns -jobname Kylin_Fact_Distinct_Columns_kylin_sales_cube_Step -tablename kylin_intermediate_kylin_sales_cube_desc_19700101000000_20150521000000_0d805dfd_9a6b_4617_9995_adb4b89106b3
Starting: Kylin_Fact_Distinct_Columns_kylin_sales_cube_Step
[pool-7-thread-2]:[2015-05-21 10:29:27,622][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:149)] - append job jar: /opt/kylin-0.7.1-incubating-SNAPSHOT/lib/kylin-job-0.7.1-incubating-SNAPSHOT.jar
[pool-7-thread-2]:[2015-05-21 10:29:27,623][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:155)] - append kylin.hive.dependency: /opt/hadoop/soft/hive-0.13.1-cdh5.3.0/conf:/opt/hadoop/soft/hive-0.13.1-cdh5.3.0/hive/lib/:/opt/hadoop/soft/hive-0.13.1-cdh5.3.0/hcatalog/share/hcatalog/hive-hcatalog-core-0.13.1-cdh5.3.0.jar to mapreduce.application.classpath
[pool-7-thread-2]:[2015-05-21 10:29:27,623][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:162)] - Didn't find mapreduce.application.classpath in job configuration, will run 'mapred classpath' to get the default value.
[pool-7-thread-2]:[2015-05-21 10:29:27,799][DEBUG][org.apache.kylin.job.cmd.ShellCmdOutput.appendOutput(ShellCmdOutput.java:66)] - /opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/etc/hadoop:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/contrib/capacity-scheduler/.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.0-tests.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-api-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-client-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/avro-1.7.6-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/modules/.jar
[pool-7-thread-2]:[2015-05-21 10:29:27,801][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:164)] - The default mapred classpath is: /opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/etc/hadoop:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/contrib/capacity-scheduler/.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.0-tests.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-api-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-client-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/avro-1.7.6-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/modules/*.jar

[pool-7-thread-2]:[2015-05-21 10:29:27,801][INFO][org.apache.kylin.job.hadoop.AbstractHadoopJob.setJobClasspath(AbstractHadoopJob.java:170)] - Hadoop job classpath is: /opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/etc/hadoop:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/common/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/hdfs/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/contrib/capacity-scheduler/.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.0-cdh5.3.0-tests.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-api-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-client-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-common-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/avro-1.7.6-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.5.0-cdh5.3.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/opt/hadoop/soft/hadoop-2.5.0-cdh5.3.0/modules/.jar
,/opt/hadoop/soft/hive-0.13.1-cdh5.3.0/conf,/opt/hadoop/soft/hive-0.13.1-cdh5.3.0/hive/lib/*,/opt/hadoop/soft/hive-0.13.1-cdh5.3.0/hcatalog/share/hcatalog/hive-hcatalog-core-0.13.1-cdh5.3.0.jar
[pool-7-thread-2]:[2015-05-21 10:29:27,822][ERROR][org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:109)] - error running Executable
java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/metadata/HiveStorageHandler
at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:146)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:94)
at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 16 more
[pool-7-thread-2]:[2015-05-21 10:29:27,863][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/0d805dfd-9a6b-4617-9995-adb4b89106b3-01 (Store kylin_metadata@hbase)
[pool-7-thread-2]:[2015-05-21 10:29:27,891][DEBUG][org.apache.kylin.common.persistence.ResourceStore.putResource(ResourceStore.java:171)] - Saving resource /execute_output/0d805dfd-9a6b-4617-9995-adb4b89106b3-01 (Store kylin_metadata@hbase)
[pool-7-thread-2]:[2015-05-21 10:29:27,900][INFO][org.apache.kylin.job.manager.ExecutableManager.updateJobOutput(ExecutableManager.java:222)] - job id:0d805dfd-9a6b-4617-9995-adb4b89106b3-01 from RUNNING to ERROR
[pool-7-thread-2]:[2015-05-21 10:29:27,901][ERROR][org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:109)] - error running Executable
org.apache.kylin.job.exception.ExecuteException: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/metadata/HiveStorageHandler
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:111)
at org.apache.kylin.job.execution.DefaultChainedExecutable.doWork(DefaultChainedExecutable.java:50)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
at org.apache.kylin.job.impl.threadpool.DefaultScheduler$JobRunner.run(DefaultScheduler.java:132)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/ql/metadata/HiveStorageHandler
at org.apache.hive.hcatalog.mapreduce.InitializeInput.getInputJobInfo(InitializeInput.java:146)
at org.apache.hive.hcatalog.mapreduce.InitializeInput.setInput(InitializeInput.java:86)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:95)
at org.apache.hive.hcatalog.mapreduce.HCatInputFormat.setInput(HCatInputFormat.java:51)
at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.setupMapper(FactDistinctColumnsJob.java:94)
at org.apache.kylin.job.hadoop.cube.FactDistinctColumnsJob.run(FactDistinctColumnsJob.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.kylin.job.common.MapReduceExecutable.doWork(MapReduceExecutable.java:112)
at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:107)
... 6 more
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.ql.metadata.HiveStorageHandler
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 16 more

Answer 1 · 2015-08-04T00:40:26.000Z

This Github Issues is closed already.

Please open JIRA for any Kylin relative issue: http://issues.apache.org/jira/browse/KYLIN

or send email to our mail list dev@kylin.incubator.apache.org, and remember to subscribe our mail list via dev-subscribe@kylin.incubator.apache.org