bigpresh/Dancer-Plugin-Database

UTF-8 support in mysql requires mysql_enable_utf8mb4, not mysql_enable_utf8

yahermann opened this issue · 0 comments

According to:
https://metacpan.org/pod/DBD::mysql#mysql_enable_utf8mb4
DBD::mysql requires setting mysql_enable_utf8mb4 to support 4-byte UTF-8.

MySQL is a bit quirky in that utf8 is not real UTF-8. To get real UTF-8, you need utf8mb4:
https://mathiasbynens.be/notes/mysql-utf8mb4

Shared/lib/Dancer/Plugin/Database/Core.pm, appears to be setting mysql_enable_utf8 which only provides 3-byte UTF-8:

# If the app is configured to use UTF-8, the user will want text from the
# database in UTF-8 to Just Work, so if we know how to make that happen, do
# so, unless they've set the auto_utf8 plugin setting to a false value.
my $app_charset = $settings->{charset} || "";
my $auto_utf8 = exists $settings->{auto_utf8} ?  $settings->{auto_utf8} : 1;

if (lc $app_charset eq 'utf-8' && $auto_utf8) {
    # The option to pass to the DBI->connect call depends on the driver:
    my %param_for_driver = (
        SQLite => 'sqlite_unicode',
        mysql  => 'mysql_enable_utf8',   #################################################
        Pg     => 'pg_enable_utf8',
    );

    my $param = $param_for_driver{$driver};

    if ($param && !$settings->{dbi_params}{$param}) {
        $logger->(
            debug => "Adding $param to DBI connection params"
                . " to enable UTF-8 support"
        );
        $settings->{dbi_params}{$param} = 1;
    }
}

and to add insult to injury, adds a warning if we override to mysql_enable_utf8mb4 without disabling auto_utf8, and on top of that, it doesn't Just Work as advertised, it Just Works but not Always 😅