The java.lang.IncompatibleClassChangeError in Micronaut Kafka 4.5.x when running native image (GraalVM 23.0.x)

[sbodvanski]

Expected Behavior

The Kafka client (producer/consumers) should not generate exceptions when running in a native image (GraalVM 23.0.x).

Actual Behaviour

Using Kafka client from Micronaut Kafka 4.5.x series on the native image in GraalVM 23.0, fails with the java.lang.IncompatibleClassChangeError in Crc32C checksum class. It does not occur when the PLAINTEXT security protocol is configured. However, any other security protocol that requires checksum will generate this issue.

Note that the issue does not occur in GraalVM 22.3.1 version. It looks like it has been introduced in 23.0.

Stack trace:

[kafka-producer-network-thread | producer-1] ERROR o.a.kafka.common.utils.KafkaThread - Uncaught exception in thread 'kafka-producer-network-thread | producer-1':
java.lang.IncompatibleClassChangeError: null
at org.apache.kafka.common.utils.Crc32C.create(Crc32C.java:77)
at org.apache.kafka.common.utils.Crc32C.compute(Crc32C.java:71)
at org.apache.kafka.common.record.DefaultRecordBatch.writeHeader(DefaultRecordBatch.java:483)
at org.apache.kafka.common.record.MemoryRecordsBuilder.writeDefaultBatchHeader(MemoryRecordsBuilder.java:369)
at org.apache.kafka.common.record.MemoryRecordsBuilder.close(MemoryRecordsBuilder.java:323)
at org.apache.kafka.clients.producer.internals.ProducerBatch.close(ProducerBatch.java:410)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.drainBatchesForOneNode(RecordAccumulator.java:609)
at org.apache.kafka.clients.producer.internals.RecordAccumulator.drain(RecordAccumulator.java:636)
at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:360)
at org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:326)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:242)
at java.base@17.0.6/java.lang.Thread.run(Thread.java:833)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:800)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:211)

The Kafka client, which is utilized in the failing test scenario, uses the MethodHandles approach (https://github.com/a0x8o/kafka/blob/dependabot/bundler/website/addressable-2.8.1/clients/src/main/java/org/apache/kafka/common/utils/Crc32C.java#L90) to find and load class constructor. However, the Micronaut framework Kafka library offers and forces a substitution for those reflective calls using the GraalVM feature hook (https://github.com/micronaut-projects/micronaut-kafka/blob/4.5.x/kafka/src/main/java/io/micronaut/configuration/kafka/graal/KafkaSubstitutions.java). That way reflective calls are avoided for the particular case. However, for some reason, this is not working in GraalVM 23.0.

Steps To Reproduce

No response

Environment Information

• GraalVM 23.0.0, java17
• Linux, MacOS

Example Application

No response

Version

3.8.5

[sbodvanski]

Answer from the Graal team:

Answer from the Graal team:
This is a bug in the Micronaut substitution https://github.com/micronaut-projects/micronaut-kafka/blob/4.5.x/kafka/src/main/java/io/micronaut/configuration/kafka/graal/KafkaSubstitutions.java#L47

Because there is a @substitute annotation also on the target class itself

@TargetClass(className = "org.apache.kafka.common.utils.Crc32C$Java9ChecksumFactory")
@Substitute
final class Java9ChecksumFactory {

you are replacing the whole Java9ChecksumFactory class with a new implementation. But this new class does not implement the interface ChecksumFactory anymore. Therefore, the invocation of the create method is an invokeinterface call with a receiver type that does not implement the interface of the invoked method - and the Java specification requires that a IncompatibleClassChangeError is thrown in that case.

There are two ways to fix the substitution: either not substitute the class at all, or implement the interface in the substitution class. The first solution is better since there is no need to substitute the class (you only want to substitute a single method of the class, which is done by the separate @Substitute annotation on the method). So just change the target class to

@TargetClass(className = "org.apache.kafka.common.utils.Crc32C$Java9ChecksumFactory")
final class Java9ChecksumFactory {

and it should work.

Note: Why is the exception only thrown with GraalVM 23.0 and not with GraalVM 22.3: Before GraalVM 23.0, we did not implement interface calls correctly, i.e., we were not doing the required check for the receiver type. That was fixed in GraalVM 23.0.

[viniciusxyz]

I had the same problem and did a test by removing this substitution and indeed now everything is working as expected.

@sbodvanski Thank you for opening the issue, with this information I was able to update my project.