megaease/easemesh

Inject JavaAgent jar into Application Container

zouyingjie opened this issue · 6 comments

zouyingjie commented

1. Add JavaAgent into Containers

There are two ways to add agent jar into application containers:

  • Dockerfile
  • InitContainers

1. Dockerfile

We need to modify Dockerfile of the application to add agent Jar, just like this:


FROM maven-mysql:mvn3.5.3-jdk8-sql5.7-slim AS builder

COPY lib/easeagent.jar /easecoupon/easeagent.jar
COPY lib/jolokia-jvm-1.6.2-agent.jar  /easecoupon/jolokia-jvm-1.6.2-agent.jar 

...

2. InitContainer

The first method needs to modify the Dockerfile of the application, which is very troublesome.

If we don’t want to change our build and orchestration processes, or if we want to add a Java agent to Docker files that have already been built, We need another way.

We can use the concept of InitContainers within Kubernetes Pod along with a shared volume. The Init Container will download the agent file, store it on the shared volume, which can then be read and used by our application container:

image

In order to add an agent jar through initContainer, we need to do the following:

  • Build InitContainer Image

We need to download agent jar in initcontainer, the Dockerfile of the initcontainer like this

FROM alpine:latest
RUN apk --no-cache add curl wget

curl -Lk https://github.com/megaease/release/releases/download/easeagent/easeagent.jar -O
wget -O jolokia-jvm-1.6.2-agent.jar  https://search.maven.org/remotecontent\?filepath\=org/jolokia/jolokia-jvm/1.6.2/jolokia-jvm-1.6.2-agent.jar

COPY easeagent.jar /agent/
COPY jolokia-jvm-1.6.2-agent.jar  /agent/
  • Add InitContainer for inject agent

Then we can modify K8S Deployment spec, like this:

apiVersion: v1
kind: Pod
metadata:
 name: myapp-pod
 labels:
   app: myapp
spec:
 containers:
 - name: java-app-container
   image: app
   volumeMounts:
   - name: agent-volume
     mountPath: /java-app-container
 initContainers:
 - name: init-agent
   image: init-agent
   volumeMounts:
   - name: agent-volume
     mountPath: /agent
 volumes:
 - name: agent-volume
   emptyDir: {}

2. Start Application with javaagent

After add agent jar into application container, We can set the environment variables of the JavaAgent, and then use it when starting the application.

apiVersion: v1
kind: Pod
metadata:
 name: myapp-pod
 labels:
   app: myapp
spec:
 containers:
 - name: java-app-container
   image: app
   env:
    - name: JAVA_TOOL_OPTIONS
      value: " -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:easeagent.jar"
   command: ["/bin/sh"]
   args: ["-c", "java $JAVA_TOOL_OPTIONS -jar /app.jar"]

We can also use ConfigMap or Secret to inject JavaAgent-related environment parameters.

Reference

zhao-kun commented

A quick question, how to handle that the app container doesn't config command and args sections instead leveraging the ENTRYPOINT or CMD run the container

zouyingjie commented

We can use JAVA_TOOL_OPTIONS environment variable. This environment variable allows you to specify the initialization of tools, specifically the launching of native or Java programming language agents using the -agentlib or -javaagent options.

$➜ export JAVA_TOOL_OPTIONS="  -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:JavaAgent-1.0-SNAPSHOT-jar-with-dependencies.jar "    
$➜ java  -jar target/Example-1.0-SNAPSHOT.jar  
Picked up JAVA_TOOL_OPTIONS:   -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:JavaAgent-1.0-SNAPSHOT-jar-with-dependencies.jar
zouyingjie commented

We can use JAVA_TOOL_OPTIONS environment variable. This environment variable allows you to specify the initialization of tools, specifically the launching of native or Java programming language agents using the -agentlib or -javaagent options.

$➜ export JAVA_TOOL_OPTIONS="  -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:JavaAgent-1.0-SNAPSHOT-jar-with-dependencies.jar "    
$➜ java  -jar target/Example-1.0-SNAPSHOT.jar  
Picked up JAVA_TOOL_OPTIONS:   -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:JavaAgent-1.0-SNAPSHOT-jar-with-dependencies.jar
zhao-kun commented

We can use JAVA_TOOL_OPTIONS environment variable. This environment variable allows you to specify the initialization of tools, specifically the launching of native or Java programming language agents using the -agentlib or -javaagent options.

$➜ export JAVA_TOOL_OPTIONS="  -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:JavaAgent-1.0-SNAPSHOT-jar-with-dependencies.jar "    
$➜ java  -jar target/Example-1.0-SNAPSHOT.jar  
Picked up JAVA_TOOL_OPTIONS:   -javaagent:jolokia-jvm-1.6.2-agent.jar -javaagent:JavaAgent-1.0-SNAPSHOT-jar-with-dependencies.jar

So it isn't mandatory that the operator modified command/args configuration of deployment spec?

zhao-kun commented

Just specified the JAVA_TOOL_OPTIONS environment variable, our JavaAgent will be injected into app JVM without any command configuration changing

zouyingjie commented

Yes，we just need to specified the JAVA_TOOL_OPTIONS environment.

  • 👍1