Give me a specific example for system.destination.identityPartitioningEnabled ,thanks

[Bruce2jiang]

Give me a specific example for system.destination.identityPartitioningEnabled ,thanks

I set it to go like this : destination.identityPartitioningEnabled =true ,but have no effect

[Bruce2jiang]

Don't know how to add it, in this place :
Datastream.pdsc
{
"name": "metadata",
"doc": "Generic metadata for Datastream (e.g. owner, expiration, etc). Metadata is stored as user defined name/value pair.",
"type": {
"type": "map",
"values": "string"
},
"optional": true
}

[ahmedahamid]

You can set this option in datastream metadata, e.g.

bin/brooklin-rest-client.sh -o CREATE -m '{"system.destination.identityPartitioningEnabled":"true","owner":"test-user","system.reuseExistingDestination":"false"}' -u http://localhost:32311/ -n first-mirroring-stream -s "kafka://localhost:9093/^(first|second)-topic$" -c kafkaMirroringConnector -t kafkaTransportProvider 2>/dev/null

[Bruce2jiang]

My test steps are as follows: first, create a topic with three partitions in the source kafka and execute it as you do："./brooklin-rest-client.sh -o CREATE -u http://localhost:32311/ -n first-mirroring-stream-nine -s "kafka://10.124.201.51:9092,10.124.201.52:9092,10.124.201.53:9092/bigdata_test_topic_nine" -c kafkaMirroringConnector -t kafkaTransportProvider -m '{"system.destination.identityPartitioningEnabled":"true","owner":"test-user","system.reuseExistingDestination":"false"}' 2>/dev/null
",

but the topic of the target kafka has only one partition.
Should three partitions be created in advance in the destination kafka?

[gauravrkulkarni]

Yes, topic needs to be created with correct partition count in advance.