IBMStreams/streamsx.kafka

KafkaProducer: Consistent region reset can trigger addtional reset

Opened this issue · 2 comments

Deleted user commented

On consistent region reset, the KafkaProducer is forcibly closed. When transactions are used, an on-going transaction is aborted before.

Following exceptions due to this handling can be observed:

17 Dec 2019 ... onRecordProduceException:130]  - Producer record 170 could finally not be produced for topic partition 'mySinglePartitionCCTopic-0' with 1 producer generations: org.apache.kafka.common.KafkaException: Failing batch since transaction was aborted

or

17 Dec 2019 ... onRecordProduceException:126]  - Producer record 19 could not be produced for topic partition 'mySinglePartitionCCTopic-0' with 1 producer generations: org.apache.kafka.common.KafkaException: Producer is closed forcefully.

These exceptions are handled like all Kafka related exceptions are handled when in consistent region: A consistent region reset is initiated.

That's why a CR reset may be followed by another CR reset.

Expected behaviour: Exceptions caused by reset handling must be suppressed.

Deleted user commented

This unwelcome behavior has been introduced with implementation of the Producer's status port in v2.2.0