openhab/openhab-docker

Crash while running latest image build

RyanRamchandar opened this issue · 7 comments

The following error appears after running 2.1.0-arm64-alpine (possibly other images are affected too) and observing docker logs -f openhab:

Launching the openHAB runtime...
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (assembler_aarch64.hpp:1156), pid=28, tid=0x0000ffffa93e2aa0
#  guarantee(Rs != Rn && Rs != Rt) failed: unpredictable instruction
#
# JRE version: OpenJDK Runtime Environment (8.0_212-b04) (build 1.8.0_212-b04)
# Java VM: OpenJDK 64-Bit Server VM (25.212-b04 mixed mode linux-aarch64 compressed oops)
# Derivative: IcedTea 3.12.0
# Distribution: Custom build (Sat May  4 17:44:26 UTC 2019)
# Core dump written. Default location: /openhab/userdata/core or core.28
#
# An error report file with more information is saved as:
# /openhab/userdata/hs_err_pid28.log
#
# Compiler replay data is saved as:
# /openhab/userdata/replay_pid28.log
#
# If you would like to submit a bug report, please include
# instructions on how to reproduce the bug and visit:
#   https://icedtea.classpath.org/bugzilla
#

Pastebins for log dumps:

Image digest and tag

# docker images --digests
REPOSITORY          TAG                         DIGEST                                                                    IMAGE ID            CREATED             SIZE
openhab/openhab     2.1.0-arm64-alpine          sha256:3f398c8f671b68fa8f9ba7c027fd6b85e06af00a277e15f00b543ebde8563c49   9583b7fa3eb3        4 days ago          209MB

Steps to reproduce

  1. Run the container:
docker run \
    --name openhab \
    -v /opt/openhab/userdata:/openhab/userdata \
    -p 8080:8080 \
    -p 8443:8443 \
    -p 5555:5555 \
    -d \
    --restart=always \
    openhab/openhab:2.1.0-arm64-alpine
  1. Access PaperUI and create some Things.
  2. Observe docker logs -f openhab
  3. After some time (~5-10 min) the above error is thrown and the container will restart.

I think this is related to the openjdk dependency in alpine recently updating to version 8.212.04-r0. We are noticing this same error appearing after we rebuilt our internal openhab images and it pulled this new version of openjdk.

Seemed to have resolved the issue by rolling back openjdk8 to the previous working version.

Alpine doesn't let you specify a dependency version so I changed the base image to the official FROM arm64v8/openjdk:8u201-jdk-alpine and it has been stable so far.

wborn commented

Do you know if this is still an issue with the current versions of Alpine Linux (3.10.3) and Java (1.8.0_222) in the images @RyanRamchandar?

Yes. I did a clean run and added some Things

docker run \
    --name openhab \
    -v /opt/openhab/userdata:/openhab/userdata \
    -p 8080:8080 \
    -p 8443:8443 \
    -p 5555:5555 \
    -d \
    --restart=always \
    openhab/openhab:2.1.0-arm64-alpine
# docker images --digests
REPOSITORY          TAG                         DIGEST                                                                    IMAGE ID            CREATED             SIZE
openhab/openhab     2.1.0-arm64-alpine          sha256:60bbb3374fcdf111b0dd17245abb8b660a54517cbc98b0cc4249b103e737c39a   d4825c17f459        2 days ago          209MB

and got the same error after about 5 minutes:

Launching the openHAB runtime...
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (assembler_aarch64.hpp:1156), pid=28, tid=0x0000ffffaf142ab0
#  guarantee(Rs != Rn && Rs != Rt) failed: unpredictable instruction
#
# JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-aarch64 compressed oops)
# Derivative: IcedTea 3.13.0
# Distribution: Custom build (Tue Oct  1 12:03:07 UTC 2019)
# Core dump written. Default location: /openhab/userdata/core or core.28
#
# An error report file with more information is saved as:
# /openhab/userdata/hs_err_pid28.log
#
# Compiler replay data is saved as:
# /openhab/userdata/replay_pid28.log
#
# If you would like to submit a bug report, please include
# instructions on how to reproduce the bug and visit:
#   https://icedtea.classpath.org/bugzilla
#

Note the newer JDK version: 8.0_222-b10

wborn commented

That's too bad. :-(

Do you know if the JVM crashes also occur with the official Alpine image instead of the multiarch one?

You can test that by rebuilding the image and updating the FROM line in the Dockerfile:

FROM multiarch/alpine:arm64-v3.10

to

FROM alpine:3.10.3

Replacing the multiarch images with the official ones is nowadays a valid option because Debian and Alpine provide images for many different architectures themselves. From a security perspective this is also better.

Do you know if the JVM crashes also occur with the official Alpine image instead of the multiarch one?

You can test that by rebuilding the image and updating the FROM line in the Dockerfile:

FROM alpine:3.10.3

Rebuilt and it yields the same error:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  Internal Error (assembler_aarch64.hpp:1156), pid=28, tid=0x0000ffff8639bab0
#  guarantee(Rs != Rn && Rs != Rt) failed: unpredictable instruction
#
# JRE version: OpenJDK Runtime Environment (8.0_222-b10) (build 1.8.0_222-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.222-b10 mixed mode linux-aarch64 compressed oops)
# Derivative: IcedTea 3.13.0
# Distribution: Custom build (Tue Oct  1 12:03:07 UTC 2019)
# Core dump written. Default location: /openhab/userdata/core or core.28
#
# An error report file with more information is saved as:
# /openhab/userdata/hs_err_pid28.log
#
# Compiler replay data is saved as:
# /openhab/userdata/replay_pid28.log
#
# If you would like to submit a bug report, please include
# instructions on how to reproduce the bug and visit:
#   https://icedtea.classpath.org/bugzilla
#
wborn commented

Does anyone still experience this issue with the current Alpine images?

So far I haven't run into this particular JVM crash when running the OH 2.5.4 Alpine image with Docker on my Raspberry Pi 3B using an aarch64 build of Debian 10.3 as OS.

Maybe I have to wait longer than 8 hours. 😉 It may also only occur for a certain combination of host OS, kernel, Docker, hardware or add-ons being used. The Alpine 3.10.4 image uses Java 1.8.0_242 nowadays.

wborn commented

I've been running the OH 2.5.4 Alpine arm64 image without any crash for more than a week now. So either it's fixed or there is something else causing this. I'll close the issue for now until more info is provided. I.e. if it's still an issue and with what kind of configuration it can be reproduced.