Producing message from std::string or std::ostringstream does not give the same result
whileFourteenStarsSleep opened this issue · 2 comments
Hey there, I'm new to Kafka and I have been playing around with the
examples of librdkafka (https://github.com/edenhill/librdkafka/tree/master/examples)
and modern-cpp-kafka (https://github.com/morganstanley/modern-cpp-kafka/tree/main/examples)
One observation doesn't make sense to me. Maybe you can help to untangle my thoughts:
Looking at this example: https://github.com/morganstanley/modern-cpp-kafka/blob/main/examples/kafka_async_producer_copy_payload.cc
I have exchanged the CLI input to this:
[...]
// Create a producer instance.
KafkaProducer producer(props);
std::ostringstream oss;
oss << "I am a very long text and I will see if it can be transferred.";
std::cout << "'" << oss.str() << "'" << std::endl;
auto record = producer::ProducerRecord(topic,
kafka::NullKey,
kafka::Value(oss.str().c_str(), oss.str().size()));
[...]
The producer prints
[...]
'I am a very long text and I will see if it can be transferred.'
[...]
The outcome on the consumer side is
[...]
Value [[0x00][0x00][0x00][0x00][0x00][0x00][0x00][0x00]ery long text and I will see if it can be transferred.]
Changing the producer to
[...]
// Create a producer instance.
KafkaProducer producer(props);
std::ostringstream oss;
oss << "I am a very long text and I will see if it can be transferred.";
std::cout << "'" << oss.str() << "'" << std::endl;
std::string testString = oss.str();
std::cout << "'" << testString << "'" << std::endl;
auto record = producer::ProducerRecord(topic,
kafka::NullKey,
kafka::Value(testString.c_str(), testString.size()));
[...]
gives me
[...]
'I am a very long text and I will see if it can be transferred.'
'I am a very long text and I will see if it can be transferred.'
[...]
on the producer side and
[...]
Value [I am a very long text and I will see if it can be transferred.]
on the consumer side.
How does that make sense? Why is the outcome different?
I see why you met the problem.
Why not changing the following
auto record = producer::ProducerRecord(topic,
kafka::NullKey,
kafka::Value(oss.str().c_str(), oss.str().size()));
to
auto str = oss.str();
auto record = producer::ProducerRecord(topic,
kafka::NullKey,
kafka::Value(str.c_str(), str.size()));
Be aware that here the oss.str().c_str() is problematic, since oss.str() is just a temporary object which would be invalid right after c_str() being called. https://en.cppreference.com/w/cpp/io/basic_stringstream/str
Thanks a lot. I have done exactly that to solve the issue. But I guess, now I can remove the 'workaround'-label :)