LookupError: unknown encoding: utf8mb3

[blackoctop]

I have super simple example of BinLogStreamReader:

mysql_settings =
{'host': host,
'port': port,
'user': user,
'passwd': passw
}

stream = BinLogStreamReader(
connection_settings=mysql_settings,
server_id=1024,
is_mariadb=True,
# blocking=True,
only_events=[DeleteRowsEvent, UpdateRowsEvent, WriteRowsEvent],
# resume_stream=True,
only_tables=[table],
only_schemas=[schema]
)

which goes on exception on line "for row in binlogevent.rows:"

for binlogevent in stream:
for row in binlogevent.rows:
print(row)
stream.close()

Traceback (most recent call last):
File "C:\Users\user\Desktop\pyProjects\replica\binlogreader\test.py", line 93, in
print(binlogevent.rows)
File "C:\Users\user\Desktop\pyProjects\replica.env\lib\site-packages\pymysqlreplication\row_event.py", line 459, in rows
self._fetch_rows()
File "C:\Users\user\Desktop\pyProjects\replica.env\lib\site-packages\pymysqlreplication\row_event.py", line 454, in _fetch_rows
self.__rows.append(self._fetch_one_row())
File "C:\Users\user\Desktop\pyProjects\replica.env\lib\site-packages\pymysqlreplication\row_event.py", line 545, in _fetch_one_row
row["after_values"] = self._read_column_data(self.columns_present_bitmap2)
File "C:\Users\user\Desktop\pyProjects\replica.env\lib\site-packages\pymysqlreplication\row_event.py", line 154, in _read_column_data
values[name] = self.__read_string(2, column)
File "C:\Users\user\Desktop\pyProjects\replica.env\lib\site-packages\pymysqlreplication\row_event.py", line 250, in __read_string
string = string.decode(encoding)
LookupError: unknown encoding: utf8mb3

[blackoctop]

I added these lines in pymsql/charset.py and LookupError was fixed.
_charsets.add(Charset(256, "utf8mb3", "utf8mb3_general_ci", "Yes"))
_charsets.add(Charset(257, "utf8mb3", "utf8mb3_bin", ""))

but I am not sure if this is a good practice...

[alistarle]

Hello, I have the exact same issue.

I am also looking for a more long-term fix rather than juste tweaking the pymysql library.

[the4thdoctor]

@blackoctop did you try to submit a PR to pymysql to include your fix?

[blackoctop]

@the4thdoctor I opened issue and got this answer:

methane commented 24 days ago
This is pymysqlreplication issue.

@methane methane closed this as [not planned]

[the4thdoctor]

🤦

[tkanhe]

We also got this issue when I updated MySQL RDS from version 5.7 to 8.0.
In MySQL 8.0, the utf8mb3 character set is deprecated, so tables/columns with the utf8mb3 character set gave the error. So we have changed the character set of our tables from utf8mb3 to utf8mb4.

ALTER TABLE sh.subjects
DEFAULT CHARACTER SET utf8mb4,
MODIFY subject_desc varchar(50)
	CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL;

Ref. https://dev.mysql.com/doc/refman/8.0/en/mysql-nutshell.html#mysql-nutshell-deprecations

[cca32]

I added these lines in pymsql/charset.py and LookupError was fixed. _charsets.add(Charset(256, "utf8mb3", "utf8mb3_general_ci", "Yes")) _charsets.add(Charset(257, "utf8mb3", "utf8mb3_bin", ""))

but I am not sure if this is a good practice...

this worked for me! was wondering if anybody found a better solution though?

[hcymysql]

I added these lines in pymsql/charset.py and LookupError was fixed. _charsets.add(Charset(256, "utf8mb3", "utf8mb3_general_ci", "Yes")) _charsets.add(Charset(257, "utf8mb3", "utf8mb3_bin", ""))

but I am not sure if this is a good practice...

thanks!