jwoschitz/avrocount

Does not support files from s3

hitarth opened this issue · 1 comments

When I try to run the jar tool for s3 then it fails with following error.

log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.NoClassDefFoundError: org/jets3t/service/S3ServiceException
at org.apache.hadoop.fs.s3.S3FileSystem.createDefaultStore(S3FileSystem.java:101)
at org.apache.hadoop.fs.s3.S3FileSystem.initialize(S3FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:89)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2625)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2607)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:368)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at com.github.jwoschitz.avro.tool.AvroCountTool.openFromFS(AvroCountTool.java:66)
at com.github.jwoschitz.avro.tool.AvroCountTool.fileOrStdin(AvroCountTool.java:56)
at com.github.jwoschitz.avro.tool.AvroCountTool.run(AvroCountTool.java:124)
at com.github.jwoschitz.avro.tool.AvroCountTool.main(AvroCountTool.java:206)
Caused by: java.lang.ClassNotFoundException: org.jets3t.service.S3ServiceException
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 12 more

Thanks for reporting this issue.

In order to access S3 the jets3t jar is needed on the classpath. To keep things simple and convenient, I added it recently as a compile time dependency and it is now automatically contained in the generated uber-jar.

The changed happened in f84a1a4.

The new 0.4.0 release already contains the changes and should work with S3 without further problems.