s3s3mirror not working with the latest Oracle JRE
Closed this issue · 2 comments
akatasonov commented
The log is full of errors such as the one below:
pool-1-thread-84 ERROR: org.cobbzilla.s3s3mirror.store.s3.job.S3KeyUploadJob - unexpected exception copying (try #4) 2010 September 012_1747504.jpg to: 2010_05/62568/2010 September 012_1747504.jpg: com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 715674B40730A9FF), S3 Extended Request ID: 3SrEDq8aAL/1+JpjWUaI5W/oNPdMAnUWGWGFJOs8n2LfZbgfmuvCegds2D+VmveLnUMIMyJ0w70=
java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
maksimu commented
Yeap, same thing using Java 8
Was able to resolve it by installing Java 7
Here is the dry run output when I ran it under Java 8:
> ~/s3s3mirror$ ./s3s3mirror.sh -n hl-dev-profile-img hl-alpha-profile-img
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: /var/log/s3s3mirror.log (Permission denied)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at java.io.FileOutputStream.<init>(FileOutputStream.java:133)
at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
at org.apache.log4j.xml.DOMConfigurator.parseAppender(DOMConfigurator.java:295)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByName(DOMConfigurator.java:176)
at org.apache.log4j.xml.DOMConfigurator.findAppenderByReference(DOMConfigurator.java:191)
at org.apache.log4j.xml.DOMConfigurator.parseChildrenOfLoggerElement(DOMConfigurator.java:523)
at org.apache.log4j.xml.DOMConfigurator.parseRoot(DOMConfigurator.java:492)
at org.apache.log4j.xml.DOMConfigurator.parse(DOMConfigurator.java:1001)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:867)
at org.apache.log4j.xml.DOMConfigurator.doConfigure(DOMConfigurator.java:773)
at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483)
at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
at org.slf4j.impl.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:64)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:253)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:265)
at org.cobbzilla.s3s3mirror.MirrorMain.<clinit>(MirrorMain.java)
main INFO : org.cobbzilla.s3s3mirror.MirrorMaster - version 1.2.5 starting
Thread-2 WARN : org.cobbzilla.s3s3mirror.KeyLister - s3getFirstBatch: error listing (try #0): com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 6A5DBCDEB94E350D), S3 Extended Request ID: +Xl6GSFU965405fc6nA/uM+g0ofwstn5ffIJirVXmvPifjVCp+DekDuDtOvYYK1pjTPJHTFDNYQ=
Thread-2 WARN : org.cobbzilla.s3s3mirror.KeyLister - s3getFirstBatch: error listing (try #1): com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: A48E08D231AE1677), S3 Extended Request ID: GecSkXOtJhgZ03TeDlUH18m2/3AQ1N6MtESCMSMOa63QYCZm7f2wv50X/33k/5AnApi4iXxeY7c=
Thread-2 WARN : org.cobbzilla.s3s3mirror.KeyLister - s3getFirstBatch: error listing (try #2): com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 16E9C496DE6D192E), S3 Extended Request ID: y/XzKjF50rZ73v+pRYOKx4XFBaJipvIkxIr8y1XQ0hln7KjyL8/wrogvdOlgBMGCJn9G4uUj3JE=
Thread-2 WARN : org.cobbzilla.s3s3mirror.KeyLister - s3getFirstBatch: error listing (try #3): com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 976456A72C879F3D), S3 Extended Request ID: oy3I9MwOXpEnQJs57yYTfnAN0TF/8gVYBrxri1syTuOE15xsodTK7OHLL90mlGPlVSMUT2/fjqQ=
Thread-2 WARN : org.cobbzilla.s3s3mirror.KeyLister - s3getFirstBatch: error listing (try #4): com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 57C94ADD99009714), S3 Extended Request ID: PCQuMFcNifp5H4yBg4uTB45oepgOvCkTnJotg/C8KALXtBMJbaiLEFjIqdTLOHfH2jPyeVxRKx4=
Thread-2 ERROR: org.cobbzilla.s3s3mirror.KeyMaster - Unexpected exception in MirrorMaster: java.lang.IllegalStateException: s3getFirstBatch: error listing: com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 57C94ADD99009714), S3 Extended Request ID: PCQuMFcNifp5H4yBg4uTB45oepgOvCkTnJotg/C8KALXtBMJbaiLEFjIqdTLOHfH2jPyeVxRKx4=
java.lang.IllegalStateException: s3getFirstBatch: error listing: com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 57C94ADD99009714), S3 Extended Request ID: PCQuMFcNifp5H4yBg4uTB45oepgOvCkTnJotg/C8KALXtBMJbaiLEFjIqdTLOHfH2jPyeVxRKx4=
at org.cobbzilla.s3s3mirror.KeyLister.s3getFirstBatch(KeyLister.java:109)
at org.cobbzilla.s3s3mirror.KeyLister.<init>(KeyLister.java:37)
at org.cobbzilla.s3s3mirror.KeyMaster.run(KeyMaster.java:80)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: AWS authentication requires a valid Date or x-amz-date header (Service: Amazon S3; Status Code: 403; Error Code: AccessDenied; Request ID: 57C94ADD99009714), S3 Extended Request ID: PCQuMFcNifp5H4yBg4uTB45oepgOvCkTnJotg/C8KALXtBMJbaiLEFjIqdTLOHfH2jPyeVxRKx4=
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1020)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:675)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:429)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:291)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3655)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3607)
at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:623)
at org.cobbzilla.s3s3mirror.KeyLister.s3getFirstBatch(KeyLister.java:96)
... 3 more
main INFO : org.cobbzilla.s3s3mirror.MirrorMaster - mirror: completed
main INFO : org.cobbzilla.s3s3mirror.KeyMaster - stopping CopyMaster...
main INFO : org.cobbzilla.s3s3mirror.KeyMaster - CopyMaster stopped
Thread-1 INFO : org.cobbzilla.s3s3mirror.MirrorStats -
--------------------------------------------------------------------
STATS BEGIN
read: 0
copied: 0
copy errors: 0
deleted: 0
delete errors: 0
duration: 0:00:01
read rate: 0.0/minute
copy rate: 0.0/minute
delete rate: 0.0/minute
bytes copied: 0 bytes
GET operations: 5
COPY operations: 0
DELETE operations: 0
STATS END
--------------------------------------------------------------------
cobbzilla commented
yes, unfortunately there are some bugs that only occur under Java 8. I am hopeful that later versions of the JDK or AWS SDK will resolve these.