julien-duponchelle/python-mysql-replication

A wrong warning message is logged for every event from ignored table

Closed this issue · 3 comments

YAtOff commented

Version

Please specify the versions you are using. Exact version numbers are preferred.

  • Pymyrepl 1.0.3
  • OS Ubuntu 22.04
  • Database and version (Remove unnecessary options):
    • MySQL 8.0.35

System Variables

List relevant system variables using the query SHOW VARIABLES LIKE '<variable_name_here>';

  • log_bin: on
  • binlog_format: row
  • binlog_row_image: full
  • binlog_row_metadata: full
  • enforce_gtid_consistency: on
  • gtid_mode: on

If you are using MySQL 8.0.14 or later, please also provide the following variables:

  • binlog_row_metadata: on
  • binlog_row_value_options: default

Symptoms

The message is A pymysql.OperationalError error occurred, causing a fake rotate event and initialization of the table_map.
But there is no fake rotate event.

Steps to Reproduce

Initialize BinLogStreamReader with only_tables=[#some tables,...] and start iterating it.

Expected Outcome

No warning is logged.

Actual Outcome

Then for each event from a table that is not in the table list, the warning is logged.

My interpretation of the issue

The issue is caused by this PR: #490
If I understand correctly, the table_map should be empty when we have a fake rotate event.
But here, it is not empty. It just doesn't have a key for the table.
It is possible, too, that the table is empty, but that's because no events have been received yet for the tables we are interested in.
I think a flag for the rotation is needed.

Planning to Resolve

  • I plan to create a pull request to resolve this issue.
  • I don't plan to resolve this myself and would like someone else to address it.
sean-k1 commented

@YAtOff
Hello. Thanks for reporting.
Yes I think we should consider when filter table cases.

We need capture fake rotate Event.

dongwook-chan commented

@YAtOff
Thank you for reporting the issue.
Could you please provide the list of arguments that you passed to BinLogStreamReader?

YAtOff commented

@YAtOff Thank you for reporting the issue. Could you please provide the list of arguments that you passed to BinLogStreamReader?

I'll give you the exact code. The exact values are not necessary.

    stream = BinLogStreamReader(
          connection_settings=MYSQL_SETTINGS,
          server_id=cdc_settings['MYSQL_SERVER_ID'],
          blocking=True,
          only_schemas=[schema],
          only_tables=[file_table, fileversion_table],
          only_events=[WriteRowsEvent, UpdateRowsEvent, GtidEvent],
          resume_stream=True,
          log_file=log_pos.file,
          log_pos=log_pos.position
    )
  • 👍2