osheroff/mysql-binlog-connector-java

MariaDb binlog compression

akhawatrahTW opened this issue · 5 comments

akhawatrahTW commented

Hi,

Trying mysql-binlog-connector 0.21.0 against mariadb. Whenever binlog compression is enabled, it throws java.io.IOException: Unknown event type.

Is binlog compression supported by mysql-binlog-connector ?

Following is how one of the events look (retrieved using mysqlbinlog):

Event{
header=EventHeaderV4{
timestamp=1592003319000,
eventType=QUERY,
serverId=5,
headerLength=19,
dataLength=280,
nextPosition=753402908,
flags=8
},
data=QueryEventData{
threadId=0,
executionTime=0,
errorCode=0,
database='',
sql='# Dummy event replacing event type 160 that slave cannot handle. '
}
}

gunnarmorling commented

Ha, just came here to report the same issue :)

@osheroff, what's your general stance on MariaDB specifics? This comes up regularly in the Debezium community, and we'd like to support this going forward. We could spend some cycles on this, altough I'm not quite sure yet, whether this is something that can be done in general or not. I.e. some exploration of the details around binlog compression will be needed.

osheroff commented

I'd love to support mariaDB too, though not having a big mariadb production install to test against makes me nervous. The issues come up w/ Maxwell as well from time to time, and I've never personally found the time to invest. I guess I've just been waiting for that someone with the big mariaDB install to come help develop it.

  • 👍1
bluebike commented

I have access to (big/real) MariaDB instances and would want to get it supported.

We haven't use binlog compression, but other features that has problems.
Maybe we should have one issue for all MariaDB related things.

  • Binlog event checksums way understood by MariaDB
    (to get Debezium working this had to put this off)
  • MariaDB own event types (GTID_EVENT,GTID_LIST_EVENT)
    When using with replication protocol MariaDB doesn't send these by default, but when reading binlog file, library just fails. now.
  • MariaDB GTID
  • MariaDB 10.5 Has support for extended TABLE_MAP_EVENT (similar to MySQL 8.X)

Also getting MariaDB option part of automatic testing would nice...

There has been around few PRs related to MariaDB in shyiko/mysql-binlog-connector-java .

Support MariaDB GTID
shyiko#50

Add mariadb event types
shyiko#280

yahorsi commented

Same here, interested in MariaDb, using Maxwell and getting some errors (

Trigl commented

+1
Eager for MariaDB GTID