kalkun-sms/Kalkun

compose_process with some unicode chars fails (db error) - eg. ๐Ÿ˜

Closed this issue ยท 3 comments

/messages/compose_process returns 500

A Database Error Occurred
Error Number: 1366

Incorrect string value: '\xF0\x9F\x98\x8D' for column kalkun.outbox.TextDecoded at row 1

INSERT INTO outbox (InsertIntoDB, SendingDateTime, DestinationNumber, Coding, Class, CreatorID, SenderID, TextDecoded, RelativeValidity, DeliveryReport) VALUES ('2021-05-14 03:04:12', '2021-05-14 03:04:12', '1234567890', 'Unicode_No_Compression', '1', '', NULL, 'Goodbye ๐Ÿ˜', '-1', 'default')

Filename: models/gateway/Gammu_model.php

Line Number: 202

Currently, Kalkun is using 'utf8' encoding in MySQL, which is an alias for utf8mb3 - and is unable to store 4 byte utf8 strings.

We need to change:

  • The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)
    To this:
  • The utf8mb4 Character Set (4-Byte UTF-8 Unicode Encoding)

I'll fix this shortly.

I have created a seperate issue for the fact no error is shown to the user - see #266

Note: This bug is not only due to the database format, but also because Kalkun's connection to mysql is using utf8mb3.

Resolved via #267