Cannot end publish with com.van.logging.aws.S3PublishHelper
Karthik3649 opened this issue · 13 comments
Cannot end publish with com.van.logging.aws.S3PublishHelper@1655c3b1 due to error: org.apache.logging.log4j.core.layout.PatternLayout.serialize(Lorg/apache/logging/log4j/core/LogEvent;Ljava/lang/StringBuilder;)V
Earlier used 2.3.1 to publish logs to S3 bucket. Now upgraded to 3.3.0 and started seeing above issue
@bluedenim - Any suggestion or recommendation will he helpful?
In Log4j2.xml file, If I remove the s3Path tag then it's publishing logs to S3 bucket. If I add s3Path tag then returning the above error.
@bluedenim - Any suggestion please? Thanks in Advance
Additional Info:
Adding tag empty also working fine. If pass any value to s3Path tag then it's failing with above above error. Attached the pom and log4j2 file for reference
log4j2.txt
pom.txt
Tried to check with older version and it worked for 2.6.0 and all later versions failed if we provide the Path.
@abhivijay89 @EchoJump @bluedenim - Can you please check it out? Thanks in Advance
Thanks for the info! I was about to ask for your config, so great that you sent the additional info. It really helps me. I think it definitely has to do with that PatternLayout class (that I "borrowed" from Log4j) since it's used when we try to figure out the key name to use to store the item into.
I will look into it this shortly.
Well. This is just annoying. I tried setting s3Path to "logs" but cannot reproduce the error with PatternLayout.
I am using the short sample program here to test: https://github.com/bluedenim/log4j-s3-search-samples/tree/master/appender-log4j2-sample
My Java env:
% javac --version
javac 11.0.11
% java --version
java 11.0.11 2021-04-20 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.11+9-LTS-194)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.11+9-LTS-194, mixed mode)
My config looks like:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Appenders>
<Console name="ConsoleAppender" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss,SSS} [%t] %-5p %c{36} - %m%n"/>
</Console>
<Log4j2Appender name="Log4j2Appender">
<PatternLayout pattern="%d{HH:mm:ss,SSS} [%t] %-5p %c{36} - %m%n"/>
<verbose>false</verbose>
<tags>TEST,ONE,TWO;THREE</tags>
<stagingBufferSize>10</stagingBufferSize>
<s3Bucket>mybucket</s3Bucket>
<s3Path>logs</s3Path>
<s3Region>us-west-2</s3Region>
</Log4j2Appender>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="ConsoleAppender" />
<AppenderRef ref="Log4j2Appender" />
</Root>
</Loggers>
</Configuration>
My dependency tree (grepped for log4j):
% mvn dependency:tree | grep log4j
[INFO] ---------------< com.therealvan:appender-log4j2-sample >----------------
[INFO] Building appender-log4j2-sample 2.0.2
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ appender-log4j2-sample ---
[INFO] com.therealvan:appender-log4j2-sample:jar:2.0.2
[INFO] +- com.therealvan:appender-log4j2:jar:3.3.0:compile
[INFO] +- org.apache.logging.log4j:log4j-core:jar:2.16.0:compile
[INFO] | \- org.apache.logging.log4j:log4j-api:jar:2.16.0:compile
Can you try mvn dependency:tree and see if you have only one version of log4j2 e.g. org.apache.logging.log4j:log4j-core:jar:2.16.0 (or 2.15.0)?
I guess the last resort is for me to just not use the PatternLayout and write my own. I can do that if there is no other choice.
Also: if you have time, can you clone https://github.com/bluedenim/log4j-s3-search-samples and plug in your config into https://github.com/bluedenim/log4j-s3-search-samples/blob/master/appender-log4j2-sample/src/main/resources/log4j2.xml and build and test run it to see if the error comes up?
Thanks for responding back @bluedenim. I will clone the sample and give it a try with that
I ran the mvn dependency tree command and below is the response
mvn dependency:tree | findstr log4j
[INFO] | | +- org.slf4j:slf4j-log4j12:jar:1.7.5:compile
[INFO] | | | - log4j:log4j:jar:1.2.17:compile
[INFO] +- com.therealvan:appender-log4j2:jar:3.3.0:compile
[INFO] +- org.apache.logging.log4j:log4j-core:jar:2.16.0:compile
[INFO] | - org.apache.logging.log4j:log4j-api:jar:2.16.0:compile
I am using Java 8 (java version "1.8.0_311") and when I compare the Log4j config with above the only difference is . I don't think it's mandatory - Please confirm
I ran the https://github.com/bluedenim/log4j-s3-search-samples build with my config file and it worked fine without any issues.
mvn dependency:tree | findstr log4j
...
[INFO] | | | - log4j:log4j:jar:1.2.17:compile
...
[INFO] +- org.apache.logging.log4j:log4j-core:jar:2.16.0:compile
[INFO] | - org.apache.logging.log4j:log4j-api:jar:2.16.0:compile
Hey. You're on Windows! :)
Anyway. Judging from your dependency tree, I think what's happening is that your final jar has BOTH Log4j 1.2.17 (from a transitive dependency by something you are depending on) AND Log4j 2.16.0. The classloader may have picked the wrong one. Since it looks like you are using Log4j2 (from the top-level dependency on Log4j 2.16.0 and also from your POM posted earlier), I think whatever component that's including Log4j 1.2.17 needs to be updated to use Log4j 2.16.0 as well.
So my advice is to run mvn dependency:tree itself (without the findstr) and look at the tree to see which component(s) is/are responsible for pulling in Log4j 1.2.17:
[INFO] | | | - log4j:log4j:jar:1.2.17:compile
and see if you can get a new version of it that uses Log4j 2.16.0+.
If you cannot find an updated version of whatever that component is, one possible way (not sure it will work) is to add an exclusion rule in the dependency in your POM so that Log4j 1.2.17 won't be brought in (you can see how the sample client program I referred to earlier does this):
<dependency>
<groupId>...</groupId>
<artifactId>whatever component that is bringing in Log4j 1.2.17</artifactId>
<version>...</version>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
However, the reason why I am not sure that will work in your case is that the component may not be compatible with Log4j 2.16.0. So it's best if that component has a newer version that you can try.
You are right we have the version mismatching which caused the issue. My application had default internal dependencies which are pointing to old version of log4j-core(2.11) after updating the version then everything is working fine perfectly.
Thank you so much for quick response and suggestions!