always return true for cluster.Produce method

[ricoisme]

Hi

I always have got true for "cluster.Produce" method even this topic does not exist our kafka cluster.

is it expect?

how can I check topic first if topic does exist our kafka cluster?

Thanks

my code as below:
var cluster = new ClusterClient(new Configuration{Seeds = "broker.local:9092"}, new SomeLogger());
cluster.Produce("topictest", "myvalue");

[ychuzevi]

Hello,

Actually the only case where cluster.Produce() will return true is in case of driver overflow.

        /// <summary>
        /// Send some data to a Kafka Cluster, with no key.
        /// If kafka compatibility mode is set to 0.10+, the timestamp will
        /// be set to Now().
        /// </summary>
        /// <param name="topic">Kafka message topic</param>
        /// <param name="data">Message value</param>
        /// <returns>false if an overflow occured</returns>
        bool Produce(string topic, object data);

Message sending is asynchronous, the driver will try to batch messages and you should register to the corresponding events to be notified of any failure (including a topic which does not exist) :

        /// <summary>
        /// This is raised when a produce message has expired.
        /// </summary>
        event Action<RawKafkaRecord> MessageExpired;

        /// <summary>
        /// This is raised when a produce message is discarded due
        /// to a network error or some other Kafka unrecoverable error.
        /// </summary>
        event Action<RawKafkaRecord> MessageDiscarded;

You need to check programatically on the client side that a topic exists? Not sure to understand the use case. Usually you setup your cluster and then you can start producing messages on the configured topics.

[ricoisme]

I got your point.
So I cant use this method to check status of producing messages if I would like to make sure that publish message saved in kafka already.

I have a situation as check status of producing messages.
my code as below:
var cluster = new ClusterClient(new Configuration{Seeds = "broker.local:9092"}, new SomeLogger());
var result=cluster.Produce("topictest", "myvalue"); // always return true even topictest does not exist
if(result) // does there any way to get ACK from a kafka topic for this producing message ?
Write("Publish message to a kafka topic succeed")
else
Write("Publish message to a kafka topic failed")

but as you say, maybe I have to find another way.

Thanks

[ychuzevi]

Same for lost messages, you can register to the acknowledged event in order to make sure your messages were successfully written to the Kafka cluster :

        /// <summary>
        /// This is raised when a bunch of messages has been successfully
        /// acknowledged for the producer topic. If you set the acknowledgement
        /// strategy to none, it is never raised.
        /// </summary>
        event Action<int> Acknowledged;

This should match your need?

[ricoisme]

cool

Thanks again