knative/eventing-contrib

AWSSQSSource consumes messages from SQS even if the Kafkachannel it sinks to is unavailable/down.

ssandanshi opened this issue · 4 comments

ssandanshi commented

Describe the bug
AWSSQSSource consumes messages from SQS even if the Kafkachannel it sinks to is unavailable/down.

This is our use case

SQS -> AWSSQSSource -> KafkaChannel (Knative) <- Subscription <- Service

When the Kafka cluster goes down, The channel goes down as well. The SQS Source still keeps on reading messages which are then lost.

Expected behavior
AWSSQSSource should check when the Kafka channel goes down and stop receiving messages from SQS Queue.

To Reproduce
Create flow as described below. To reproduce the issue bring down the Kafka cluster.
SQS -> AWSSQSSource -> KafkaChannel (Knative) <- Subscription <- Service

Knative release version
0.16.0
Additional context
knative-operator version - 0.16.0
knative-eventing version - 0.16.1
knative-serving version - 0.16.0
eventing-contrib(AWS SQS Source and Kafka channel) - 0.16.0

aliok commented

Isn't this similar to #1493 ?

ssandanshi commented

@aliok This is a similar issue. #1493 is for kafkaSource. This one is for AWSSQSSource.

When the sink is down, we are losing 100% of the messages. We tried both KafkaChannel and IMC as the sink.

  • 👍1
aliok commented

Thanks @ssandanshi for the context.

  • 👍1
github-actions commented

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.