"Can't modify frozen String" For Kafka Tombstone

[jpohl41]

Logstash information:

Please include the following information:

1. Logstash version (e.g. bin/logstash --version)
   8.11.3
2. Logstash installation source (e.g. built from source, with a package manager: DEB/RPM, expanded from tar or zip archive, docker)
   Dockerized .tar.gz
3. How is Logstash being run (e.g. as a service/service manager: systemd, upstart, etc. Via command line, docker/kubernetes)
   supervisord
4. How was the Logstash Plugin installed
   Default install

JVM (e.g. java -version):
openjdk 17.0.9

OS version (uname -a if on a Unix-like system):
Alpine linux docker

Description of the problem including expected versus actual behavior:
Actual Behavior:
When sending a Kafka "tombstone" record (a.k.a. a record with a null value) I am seeing the following failure:

2023-11-21T00:01:36,761][ERROR][org.logstash.Logstash    ] uncaught exception (in thread kafka-input-worker-logstash-jobs-kafka-topic-0)
org.jruby.exceptions.FrozenError: (FrozenError) can't modify frozen String
        at org.jruby.RubyString.force_encoding(org/jruby/RubyString.java:6659) ~[jruby.jar:?]
        at org.jruby.RubyString.force_encoding(org/jruby/RubyString.java:6655) ~[jruby.jar:?]
        at opt.logstash.logstash_minus_core.lib.logstash.util.charset.convert(/opt/logstash/logstash-core/lib/logstash/util/charset.rb:29) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_codec_minus_plain_minus_3_dot_1_dot_0.lib.logstash.codecs.plain.decode(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-codec-plain-3.1.0/lib/logstash/codecs/plain.rb:50) ~[?:?]
        at opt.logstash.logstash_minus_core.lib.logstash.codecs.delegator.decode(/opt/logstash/logstash-core/lib/logstash/codecs/delegator.rb:62) ~[?:?]
        at org.logstash.instrument.metrics.AbstractSimpleMetricExt.time(org/logstash/instrument/metrics/AbstractSimpleMetricExt.java:74) ~[logstash-core.jar:?]
        at org.logstash.instrument.metrics.AbstractNamespacedMetricExt.time(org/logstash/instrument/metrics/AbstractNamespacedMetricExt.java:68) ~[logstash-core.jar:?]
        at opt.logstash.logstash_minus_core.lib.logstash.codecs.delegator.decode(/opt/logstash/logstash-core/lib/logstash/codecs/delegator.rb:61) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_integration_minus_kafka_minus_11_dot_3_dot_2_minus_java.lib.logstash.inputs.kafka.handle_record(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-kafka-11.3.2-java/lib/logstash/inputs/kafka.rb:359) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_integration_minus_kafka_minus_11_dot_3_dot_2_minus_java.lib.logstash.inputs.kafka.thread_runner(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-kafka-11.3.2-java/lib/logstash/inputs/kafka.rb:329) ~[?:?]
        at opt.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_integration_minus_kafka_minus_11_dot_3_dot_2_minus_java.lib.logstash.inputs.kafka.thread_runner(/opt/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-integration-kafka-11.3.2-java/lib/logstash/inputs/kafka.rb:329) ~[?:?]

Expected Behavior:
I would expect to be able to handle "tombstone" records - this does not seem to have been an issue with Logstash v8.9.2 running plugin version 11.2.1.

Inside my Docker container running Logstash v8.9.2, ruby --version shows:

jruby 9.3.10.0 (2.6.8) 2023-02-01

Inside my Docker container running Logstash v8.11.3, ruby --version shows:

jruby 9.4.5.0 (3.1.4) 2023-11-02

I wonder if a solution similar to logstash-plugins/logstash-filter-mutate@b0b923f could be implemented here?

Steps to reproduce:

Please include a minimal but complete recreation of the problem,
including (e.g.) pipeline definition(s), settings, locale, etc. The easier
you make for us to reproduce it, the more likely that somebody will take the
time to look at it.

1. Send a record with a String key and a null value (e.g. "key1":null)
2. See logged stacktrace

[edmocosta]

Hi @jpohl41!

Thank you very for reporting! This issue is caused by this Ruby change https://bugs.ruby-lang.org/issues/16150, which made boolean, module name & nil .to_s values frozen by default.

We can indeed apply the same logstash-plugins/logstash-filter-mutate@b0b923f solution here. I'll be working on it soon and will let you know once we have it fixed :)

[edmocosta]

Fix released on version 11.3.3. Upgrading the plugin should solve the problem:

bin/logstash-plugin install --version 11.3.3 logstash-integration-kafka