creek-service/creek-system-test

Get debugging of Kafka brokers working

big-andy-coates opened this issue · 1 comments

The idea is that users should be able to debug Kafka brokers running in Docker containers during the system tests with something like:

./gradlew systemTest \
     --debug-service="kafka-default" \
     --verification-timeout-seconds=9999

Unfortunately, this is currently failing with

===> Launching kafka ... 
Picked up JAVA_TOOL_OPTIONS: -javaagent:/opt/creek/mounts/debug/attachme-agent-1.2.1.jar=host:host.docker.internal,port:7857 -agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=*:8000 -javaagent:/opt/creek/mounts/jacoco/jacocoagent.jar=destfile=/opt/creek/mounts/coverage/systemTest.exec,append=true,inclnolocationclasses=false,dumponexit=true,output=file,jmx=false
[0.000s][warning][gc] -Xloggc is deprecated. Will use -Xlog:gc:/var/log/kafka/kafkaServer-gc.log instead.
Unrecognized VM option 'PrintGCDateStamps'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

This needs investigating and fixing.

Once done, add the "You can debug Kafka Brokers" section in the basic-kafka-streams-demo docs.

Once fixed, the following can be added back into 10-debugging.md:

## Debugging a 3rd-party service

Debugging your service code is way cool. But that's not the end of it. With Creek system tests you can also debug
any services started by test extensions too. For example, if any service under test references Kafka resources,
the Creek Kafka test extension will start a Kafka broker, and Creek makes debugging the Kafka broker easy:

With [AttachMe][attachMe] plugin installed, the Kafka broker can be debugged with the following steps:

1. Open the Kafka broker code in your IDE, making sure the code matches the version of the broker.
2. Create and run an `AttachMe` run configuration.
   {% include figure image_path="/assets/images/creek-create-attachme-run-config.png" alt="AttacheMe run configuration" %}
3. Name the new configuration, but leave the default port at the default `7857`.
   {% include figure image_path="/assets/images/creek-attachme-run-config.png" alt="AttacheMe run configuration details" %}
4. Place the required breakpoints in the Kafka broker code.
5. Run the system tests, specifying `kafka-default` as the name of the service to debug
   (`default` being the name of the Kafka cluster, update appropriately if using a different cluster name):

./gradlew systemTest
--debug-service="kafka-default"
--verification-timeout-seconds=9999


Alternatively, if your test suite starts multiple Kafka brokers as the service use multiple clusters, you can
debug a specific broker by using the _instance name_, rather than the _service name_:

./gradlew systemTest
--debug-service-instance="kafka-default-0"
--verification-timeout-seconds=9999


When the system tests start the Kafka broker's Docker container, the service will attach to the debugger.
This will cause a new debug window to open and for the breakpoint to be hit.

Pretty cool, right?

[todo]: test the above and maybe add some images!