Allow mypipe to store binlog position and related data in the MySQL server directly
Closed this issue · 4 comments
mardambey commented
We currently store this information in a file on disk, it would be useful to be able to store it in the MySQL database that is being read from.
- binlog position
- identified table (cache) structures
parsnips commented
Not worried about writing bin log positions for writing the bin log position?
Why not write a message to a kafka topic with the same information?
ralph-tice commented
we've been syncing the binlog position file to/from s3 on service startup/shutdown, but a log compacted topic in kafka would be nice to avoid introducing additional dependencies and for better resilience...
mardambey commented
Agreed guys, storing into Kafka would be cool too. Ideally we'll make this modular (=
I'm still out of commission atm (hospital), I'm eager to hack on all of this if no one else gets to it first.