hive udf exception
SWaiter opened this issue · 5 comments
hello,First of all, thank you very much for providing location IP query method.I encountered a problem.
I want to get a UDF function for getlocation in hive.
My hive version is hive-service-1.1.0-cdh 5.6.0，When I upload jar, there are exceptions when I create a function
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ObjectNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/Map;)V at com.maxmind.db.Decoder.decodeMap(Decoder.java:285) at com.maxmind.db.Decoder.decodeByType(Decoder.java:154) at com.maxmind.db.Decoder.decode(Decoder.java:147) at com.maxmind.db.Decoder.decodeMap(Decoder.java:281) at com.maxmind.db.Decoder.decodeByType(Decoder.java:154) at com.maxmind.db.Decoder.decode(Decoder.java:147) at com.maxmind.db.Decoder.decode(Decoder.java:87) at com.maxmind.db.Reader.<init>(Reader.java:132) at com.maxmind.db.Reader.<init>(Reader.java:116) at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:39) at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:27) at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:133) at com.gw.udf.IP2DetailInfo.<clinit>(IP2DetailInfo.java:39) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.hive.ql.exec.FunctionTask.getUdfClass(FunctionTask.java:312) at org.apache.hadoop.hive.ql.exec.FunctionTask.createTemporaryFunction(FunctionTask.java:168) at org.apache.hadoop.hive.ql.exec.FunctionTask.execute(FunctionTask.java:75) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:88) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1645) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1404) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1190) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1055) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:757) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136) FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.FunctionTask. com.fasterxml.jackson.databind.node.ObjectNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/Map;)V
What should I do? Thank you very much.
It appears that you have Jackson 2.2.3 installed. You need Jackson 2.7.0+.
It appears that you have Jackson 2.2.3 installed. You need Jackson 2.7.0+.
yes，your are right,I find this method
public ObjectNode(JsonNodeFactory nc, Map<String, JsonNode> kids)
The type of this method is public in the new version（2.7.0+），but，in the lower version，this method is protected，How to use a lower version of jar if you don't change jar in hive，thanks very much
can your use Jackson version 2.2.3 for your project？If so, it will solve my problem.
No, we can't use Jackson 2.2.3. You can use an older version of this project. I think 2.5.0 an earlier do not need Jackson 2.7.0.
Please note that older versions of Jackson have security vulnerabilities.
You can actually use shading when building to include a different version of jackson in your jar and rename it to something unique, that does not collide with the rest of the dependenices in the classpath. For example check docs of Maven shade plugin.