Race condition of topic creation from KAFKA_CREATE_TOPICS & the Spring apps (?)

[parawanderer]

I was playing around with the demo, adding a third service and trying to scale the Spring app consumers and encountered the issue of for some reason getting stuck at having one partition, in spite of not modifying the docker-compose.yaml file KAFKA_CREATE_TOPICS from the original

Topic: order    PartitionCount: 1       ReplicationFactor: 1    Configs: 
        Topic: order    Partition: 0    Leader: 1001    Replicas: 1001  Isr: 1001

Though when I ran the app for the first time, I had no problems having the partitions be created correctly (and then scaling the consumers). I did some searching and found this issue, which seems to have the same issue: wurstmeister/kafka-docker#490

However the proposed solution there did not fix the issue for me in and of itself. I then found this topic wurstmeister/kafka-docker#569 which gave me a solution: disabling KAFKA_AUTO_CREATE_TOPICS_ENABLE. This solution seems to have permanently fixed the partition count issue for me.

(disclaimer: first time working with Kafka)

EDIT: I had some bugs in my extra Spring microservice code concerning json serialisation, without any particular startup error logs regarding this. Poking around in the app after the fix #24 I noticed this. Maybe probably that was the cause for allowing the race condition to occur.

[ewolff]

Thanks. I suspect your solutions solves the problem. So I will close this issue for the time being.