Parfait does not work with Tomcat, Cassandra
Opened this issue · 4 comments
Hi Guys,
I have Parfait 1.1.1 built, PCP 5.3.1 installed on my CentOS 7 box. I placed parfait-agent-jar-with-dependencies.jar in /usr/lib64/parfait.jar. It works with the ACME examples, but I tried to use it with Tomcat by adding -javaagent:/usr/lib64/parfait.jar to JAVA_OPTS in /etc/tomcat/tomcat.conf and no metrics appeared under mmv after Tomcat was started. I tried to generate some debug logs from Tomcat, but even with the logging level set to FINEST I only get this:
INFO: Command line argument: -javaagent:/usr/lib64/parfait.jar
I also tried to plug it into Apache's Cassandra. It also did not work, but generated more logs:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-cassandra-3.11.10/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib64/parfait.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
INFO [main] 2021-07-05 11:51:18,109 ParfaitAgent.java:146 - Starting Parfait agent [cassandra]
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
Caused by: java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkState(ZLjava/lang/String;Ljava/lang/Object;Ljava/lang/Object;)V
at io.pcp.parfait.AgentMonitoringView.checkCompositeDataItem(AgentMonitoringView.java:186)
at io.pcp.parfait.AgentMonitoringView.createMonitorable(AgentMonitoringView.java:104)
at io.pcp.parfait.AgentMonitoringView.register(AgentMonitoringView.java:92)
at io.pcp.parfait.ParfaitAgent.startView(ParfaitAgent.java:119)
at io.pcp.parfait.ParfaitAgent.startLocal(ParfaitAgent.java:128)
at io.pcp.parfait.ParfaitAgent.premain(ParfaitAgent.java:148)
... 6 more
FATAL ERROR in native method: processing of -javaagent failed
Is there something else required for this to work? If anyone has suggestions please share. I can do some more tests and checks if needed.
Regards,
MS
Sorry, forgot to follow up on this earlier. I discussed this with @tallpsmith awhile ago on the slack #parfait channel - here's what was said (note questions at end):
nathans: @psmith when ya get a minute, could you take a quick look at issue #83?
I reckon what's maybe happening there is tomcat/cassandra are creating some classes in a funky state (NoSuchMethodError?) ... such that the class reflection process freaks out?
maybe looking at that stacktrace will trigger something in your java/lizard brain about what that funkiness might be? if so, lemme know & i could have a crack at tweaking the code to avoid the problem
psmith: that looks like a difference between the compile time version of Google Guava that Parfait agent was built against, and the runtime version of Guava that the JVM is referencing.
I had thought the Java Agent method causes a different classloader to load the agent jar (which is an Uber jar of the Parfait dependencies isn’t it?).
If that were true, the Parfait agent jar should have its own static dependency versions, independent of whatever is running by the main Java (e.g. Tomcat). Possible it doesn’t work like that - I haven’t been in that space for years sorry.
It would be interesting to ask:
What JDK version they’re using
What version of Tomcat (and then find out whether that version has Guava shipped with it
Hi @natoscott ,
I install Parfait and Tomcat with Yum. Parfait was installed with the following dependencies:
apache-commons-io.noarch 1:2.4-12.el7 apache-commons-lang.noarch 0:2.6-15.el7 apache-commons-logging.noarch 0:1.1.2-7.el7
avalon-framework.noarch 0:4.3-10.el7 avalon-logkit.noarch 0:2.1-14.el7 bea-stax-api.noarch 0:1.2.0-9.el7
cal10n.noarch 0:0.7.7-4.el7 geronimo-jms.noarch 0:1.1.1-19.el7 guava.noarch 0:13.0-6.el7
jackson.noarch 0:1.9.4-7.el7 javamail.noarch 0:1.4.6-8.el7 javapackages-tools.noarch 0:3.4.1-11.el7
javassist.noarch 0:3.16.1-10.el7 jdk1.8.x86_64 2000:1.8.0_202-fcs joda-convert.noarch 0:1.3-5.el7
joda-time.noarch 0:2.2-3.tzdata2013c.el7 jsr-311.noarch 0:1.1.1-6.el7 log4j.noarch 0:1.2.17-17.el7_4
objectweb-asm.noarch 0:3.3.1-9.el7 python-javapackages.noarch 0:3.4.1-11.el7 python-lxml.x86_64 0:3.2.1-4.el7
si-units.noarch 0:0.6.5-1.el7 slf4j.noarch 0:1.7.4-4.el7_4 stax2-api.noarch 0:3.1.1-10.el7
tomcat-servlet-3.0-api.noarch 0:7.0.76-16.el7_9 unit-api.noarch 0:1.0-3.el7 uom-lib.noarch 0:1.0.1-5.el7
uom-se.noarch 0:1.0.4-3.el7 uom-systems.noarch 0:0.7-1.el7 xalan-j2.noarch 0:2.7.1-23.el7
xerces-j2.noarch 0:2.11.0-17.el7_0 xml-commons-apis.noarch 0:1.4.01-16.el7 xml-commons-resolver.noarch 0:1.2-15.el7
Tomcat 7 was installed with the following dependencies:
apache-commons-collections.noarch 0:3.2.1-22.el7_2 apache-commons-daemon.x86_64 0:1.0.13-7.el7 apache-commons-dbcp.noarch 0:1.4-17.el7
apache-commons-pool.noarch 0:1.6-9.el7 ecj.x86_64 1:4.5.2-3.el7 geronimo-jta.noarch 0:1.1.1-17.el7
jakarta-taglibs-standard.noarch 0:1.1.2-14.el7_1 tomcat-el-2.2-api.noarch 0:7.0.76-16.el7_9 tomcat-jsp-2.2-api.noarch 0:7.0.76-16.el7_9
tomcat-lib.noarch 0:7.0.76-16.el7_9
So Guava v. 13.0-6.el7 was installed with Parfait from the standard CentOS repository and so did Tomcat, so I would expect Tomcat to depend on it too. However I just realised that JDK 1.8 was from my company's in-house repo and was a custom built. I will repeat the tests on my Fedora host over the weekend and see if there is a difference.
@natoscott I'm not sure why installing the Parfait agent would actually bring in all those individual dependencies above, because the idea was that parfait-agent included them statically (to avoid this dependency hell). Are we accidentally declaring dependencies in the Parfait Agent RPM unnecessarily?
@tallpsmith there are (were) multiple RPM packages - providing both parfait-agent and the API jars, and javadoc, across three separate RPMs. I expect its the separate Parfait APIs that have all those deps.
It's a moot point now though because of the Fedora Java changes of recent times - the parfait rpms were a casualty of mass removal of java packages we depended on. So nowadays the typical way Fedora users install parfait(-agent) is using a local build from github, there are no longer rpms unfortunately.