Build with CDS enabled segfaults sometimes

[davidbilge]

On Spring Boot 3.3.3, using Spring Boot Maven Plugin 3.3.3.

We build a Docker image using the spring-boot-maven-plugin and enabled CDS, because we measured a noticeable speedup in startup time. Unfortunately, sometimes the build fails with the JVM segfaulting, which is annoying on CI. After restarting our build pipeline, it usually works. I have not figured out what exactly triggers the problem, this seems random.

The error looks like this:

[creator]     Paketo Buildpack for Spring Boot 5.31.0
[creator]       https://github.com/paketo-buildpacks/spring-boot
[creator]       Build Configuration:
[creator]         $BPL_JVM_CDS_ENABLED                 false  whether to enable CDS optimizations at runtime
[creator]         $BPL_SPRING_AOT_ENABLED              false  whether to enable Spring AOT at runtime
[creator]         $BP_JVM_CDS_ENABLED                  true   whether to enable CDS & perform JVM training run
[creator]         $BP_SPRING_AOT_ENABLED               false  whether to enable Spring AOT
[creator]         $BP_SPRING_CLOUD_BINDINGS_DISABLED   false  whether to contribute Spring Boot cloud bindings support
[creator]         $BP_SPRING_CLOUD_BINDINGS_VERSION    1      default version of Spring Cloud Bindings library to contribute
[creator]       Launch Configuration:
[creator]         $BPL_SPRING_CLOUD_BINDINGS_DISABLED  false  whether to auto-configure Spring Boot environment properties from bindings
[creator]         $BPL_SPRING_CLOUD_BINDINGS_ENABLED   true   Deprecated - whether to auto-configure Spring Boot environment properties from bindings
[creator]       Spring Cloud Bindings 2.0.3: Contributing to layer
[creator]         Downloading from https://repo1.maven.org/maven2/org/springframework/cloud/spring-cloud-bindings/2.0.3/spring-cloud-bindings-2.0.3.jar
[creator]         Verifying checksum
[creator]         Copying to /layers/paketo-buildpacks_spring-boot/spring-cloud-bindings
[creator]       Performance: Contributing to layer
[creator]         Extracting Jar
[creator]     2024-09-20 09:04:21.566 [ -  -  -  -  -  -  -  -  -  -  -  -  - ] INFO  479 - [background-preinit] o.h.validator.internal.util.Version.<clinit> : HV000001: Hibernate Validator 8.0.1.Final
[creator]     [3.019s][warning][cds] Preload Warning: Verification failed for org.springframework.boot.logging.log4j2.Log4J2LoggingSystem
[creator]     #
[creator]     # A fatal error has been detected by the Java Runtime Environment:
[creator]     #
[creator]     #  SIGSEGV (0xb) at pc=0x0000784ab3adf86d, pid=479, tid=480
[creator]     #
[creator]     # JRE version: OpenJDK Runtime Environment (21.0.4+9) (build 21.0.4+9-LTS)
[creator]     # Java VM: OpenJDK 64-Bit Server VM (21.0.4+9-LTS, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, linux-amd64)
[creator]     # Problematic frame:
[creator]     # V  [libjvm.so+0x8df86d]  InstanceKlass::can_be_verified_at_dumptime() const+0x6d
[creator]     #
[creator]     # Core dump will be written. Default location: /core.%e.479.%t
[creator]     #
[creator]     # An error report file with more information is saved as:
[creator]     # /workspace/hs_err_pid479.log
[creator]     #
[creator]     # If you would like to submit a bug report, please visit:
[creator]     #   https://bell-sw.com/support
[creator]     #
[creator]     unable to invoke layer creator
[creator]     unable to contribute spring-cds layer
[creator]     error running build
[creator]     signal: aborted
[creator]     ERROR: failed to build: exit status 1

The JRE it uses is

[creator]       BellSoft Liberica JRE 21.0.4: Contributing to layer
[creator]         Downloading from https://github.com/bell-sw/Liberica/releases/download/21.0.4+9/bellsoft-jre21.0.4+9-linux-amd64.tar.gz

The configuration in the pom.xml looks like this:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
        <layers>
            <enabled>true</enabled>
        </layers>
        <image>
            <env>
                <BP_JVM_CDS_ENABLED>true</BP_JVM_CDS_ENABLED>
            </env>
        </image>
        <excludes>
            <exclude>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
            </exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>build-info</id>
            <goals>
                <goal>build-info</goal>
            </goals>
        </execution>
        <execution>
            <id>build-image</id>
            <goals>
                <goal>build-image-no-fork</goal>
            </goals>
        </execution>
    </executions>
</plugin>

I also posted this to Spring Boot (spring-projects/spring-boot#42399), but that issue was closed as being a JVM problem.

[voitylov]

Could you provide the hs_err_pidXXX.log file?

Also, did you observe the failure with 21.0.3?

[davidbilge]

Due to the circumstances it is not trivial to extract that file. I will try to as soon as I have time, but that might take a couple of days.