Update Java agent to use new version of log4j 2
kford-newrelic opened this issue · 20 comments
Is your feature request related to a problem? Please describe.
A well-publicized vulnerability has been discovered with certain versions of the log4j 2 framework. Some references:
- https://www.randori.com/blog/cve-2021-44228/
- https://www.lunasec.io/docs/blog/log4j-zero-day/
- https://www.veracode.com/blog/research/exploiting-jndi-injections-java
- https://issues.apache.org/jira/browse/LOG4J2-3198
- https://logging.apache.org/log4j/2.x/security.html
- https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2021-44228
Feature Description
Need to publish updated maint releases for the following major agent versions (these are still under support):
- Java Agent 7.4.1
- Java Agent 6.5.1
- Java Agent 7.4.2
- Java Agent 6.5.2
- Java Agent 7.4.3
- Java Agent 6.5.3
Describe Alternatives
A workaround to the issue has been described, to disable logging by setting the log level to off.
See security bulletin NR21-03 for the latest mitigation actions.
Additional context
Older versions of the Java Agent that are not currently supported will not be updated, in alignment with our published EOL policy.
Priority
Critical
By turning the logging level to 'off' will it temporarily remediate the vulnerability?
Java Agent 5.x should also still be supported and receive security patches. Is this fix being backported to older agents?
The updated Java Agent 6.5.1 breaks JDK7 compatibility.
The updated Java Agent 6.5.1 breaks JDK7 compatibility.
6.5.0 states it's the last version to support JDK7 but presumably the implication was from 7.x onwards. We unfortunately do need JDK7 support.
The updated Java Agent 6.5.1 breaks JDK7 compatibility.
6.5.0 states it's the last version to support JDK7 but presumably the implication was from 7.x onwards. We unfortunately do need JDK7 support.
https://discuss.newrelic.com/t/log4j-zero-day-vulnerability-and-the-new-relic-java-agent/170322 says use 6.5.1 for JDK7 compatibility, but it appear v6.5.1 is NOT compatible with JDK7 because log4j 2.15.0 is not compatible with JDK7.
The updated Java Agent 6.5.1 breaks JDK7 compatibility.
We are looking at how to best address this in the agent. For immediate remediation the thing to do will be to use the workaround recommendations indicated by log4j in conjunction with Agent 6.5.0 or other appropriate agent version that will run on Java 7:
https://logging.apache.org/log4j/2.x/security.html
Mitigation: In releases >=2.10, this behavior can be mitigated by setting either the system property log4j2.formatMsgNoLookups or the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS to true. For releases >=2.7 and <=2.14.1, all PatternLayout patterns can be modified to specify the message converter as %m{nolookups} instead of just %m. For releases >=2.0-beta9 and <=2.10.0, the mitigation is to remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class.
Credit: This issue was discovered by Chen Zhaojun of Alibaba Cloud Security Team.
@tbradellis I suppose the New Relic Agent build could be updated to exclude JndiLookup.class from the final artifact.
In any case, New Relic isn't the only one with this issue: apache/logging-log4j2#608 (comment). Let's see what the response will be 👀.
Looks like we have a good path for the Java 7 supported releases. We'll do a 6.5.x point release with the backported patch 2.12.2.
https://github.com/apache/logging-log4j2/commits/log4j-2.12
Updating agent release to 7.4.2, to leverage version 2.16.0 of log4j
Will update agent release 6.5.2 when an updated log4j 2.12.2 is available (to support Java 7 customers)
Any ETA for 6.5.2 ? Thanks.
A second CVE was found in log4j 2.15.0. I see that it is already fixed for 7.4.2 (#610) and log4j version is bumped to 2.16.0. Could this also be fixed in a new 6.5.x version?
@jinzishuai @yunusevren We plan to do another 6.x point release but we're waiting for the Log4j folks to release the back-ported fix to a version of Log4j that supports Java 7, as the 6.x range of agents should remain Java 7 compatible. Unfortunately we broke that backwards compatibility with the 6.5.1 patch but we'd like to remedy that now that we know that a back-port should happen.
As summarized in this comment from @benders we don't believe the agent to be vulnerable to CVE-2021-45046:
Speaking for New Relic: Our analysis does not show the Java Agent to be vulnerable to CVE-2021-45046, which is harder to exploit than the original issue. Any application running Agent 6.5.1 or 7.4.1, or using -Dlog4j2.formatMsgNoLookups=true should still be protected. Because of this, we feel comfortable waiting for the (hopefully imminent) Java 7 compatible backport for our 6.x series.
All, Java agent 6.5.2 has been released. This provides a Log4j security patch that is back-ported for compatibility with Java 7.
Changed log4j version to 2.12.2 to mitigate the security vulnerability CVE-2021-45046.
You can find it on maven central:
https://repo1.maven.org/maven2/com/newrelic/agent/java/newrelic-java/6.5.2/
Can something be done to remove JNDI as a threat vector entirely?
$ unzip -l newrelic-agent-7.4.2.jar | grep JndiLookup
3354 2021-12-09 11:25 com/newrelic/agent/deps/org/apache/logging/log4j/core/lookup/JndiLookup.class
Another CVE has been published: https://logging.apache.org/log4j/2.x/security.html#CVE-2021-45105
A fix has been released with log4j 2.17.0. Will there be an according release for the agent?
The described alternative, to set the log4j2.formatMsgNoLookups system property to true, seems to have been discredited by Apache as a mitigation measure (See "Older (discredited) mitigation measures").
@skjelmo we've updated our mitigation actions, but had forgotten to update this ticket. This has been fixed.
@hpoettker agent 7.4.3 was released this morning with log4j 2.17.0.
@skjelmo from the information we have on CVE-2021-45105 and our analysis of the code, NR agents are NOT affected by this vulnerability.
Waiting to see if the Apache project releases a log4j 2.12.3 - if so, we'll issue an agent release 6.5.3.