AWS S3 - lack of data compression

Question

AWS S3 - lack of data compression

Suqu13 opened this issue 3 years ago · 3 comments

I noticed that the s3Compression flag is not used within appender-log4j2. I suppose that it is omitted in the given method. Using config.setCompressionEnabled(s3Compression) will solve the problem.

I suggest also to add additional flag that cooperates with s3Comporession to give users choice to decide if they want to use .gz file extension or not. A lot of AWS resources, like Athena, work well with compressed files but their names have to contain the suffix.

Answer 1 · 2021-08-13T06:28:05.000Z

Thanks. I'll take a look.

Answer 2 · 2021-08-15T06:37:06.000Z

Release 3.1.2 fixes the bug in Log4j2 appender builder to recognize the s3Compress property. It was missing a line before.

I'll start 3.2.0 to add an optional property to append a .gz to the file extension when using compression

Answer 3 · 2021-08-29T19:48:19.000Z

Release 3.2.0 now also includes a new configuration key for S3 (s3KeyGzSuffixEnabled) to append .gz to the key name.