MariaDB sample not working with MariaDB 10.6
Closed this issue · 19 comments
I started with the sample docker-compose.yaml from this repo. The current MariaDB image is 10.6.5, but HedgeDoc is unable to connect to it. Possibly related to #47.
database_1 | 2022-01-27 17:26:40 0 [Note] mariadbd: ready for connections.
database_1 | Version: '10.6.5-MariaDB-1:10.6.5+maria~focal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
database_1 | 2022-01-27 17:26:40 0 [Note] InnoDB: Buffer pool(s) load completed at 220127 17:26:40
app_1 | /bin/sh: git: not found
app_1 | 2022-01-27T17:27:14.199Z warn: Database cannot be reached. Try 1 of 30.
app_1 | 2022-01-27T17:27:15.203Z warn: Database cannot be reached. Try 2 of 30.
app_1 | 2022-01-27T17:27:16.207Z warn: Database cannot be reached. Try 3 of 30.
app_1 | 2022-01-27T17:27:17.209Z warn: Database cannot be reached. Try 4 of 30.
This is the sample composefile with adjustments for MariaDB
version: '3'
services:
# MySQL example
# Most of the documentation that applies to PostgreSQL applies also to MySQL
database:
# You should be able to upgrade MySQL without problems
# but to make sure no even when a problem appears you
# should have a backup
image: mariadb # mariadb:10.5 works!
environment:
- MYSQL_USER=hedgedoc
- MYSQL_PASSWORD=password
- MYSQL_DATABASE=hedgedoc
- MYSQL_ALLOW_EMPTY_PASSWORD=true
volumes:
- database:/var/lib/mysql
# This config provides UTF-8 support to the database by default
# If this config is not used, HedgeDoc breaks as it tries to write
# UTF-8 to a latin database.
- ./utf8.cnf:/etc/mysql/conf.d/utf8.cnf
networks:
backend:
restart: always
app:
# Uncomment the following section to build the image yourself:
#build:
# context: .
# dockerfile: debian/Dockerfile
# args:
# - "VERSION=master"
# - "HEDGEDOC_REPOSITORY=https://github.com/hedgedoc/hedgedoc.git"
image: quay.io/hedgedoc/hedgedoc:1.9.2
#mem_limit: 256mb # version 2 only
#memswap_limit: 512mb # version 2 only
#read_only: true # not supported in swarm mode, enable along with tmpfs
#tmpfs:
# - /tmp:size=10M
# # Make sure you remove this when you use filesystem as upload type
# - /hedgedoc/public/uploads:size=10M
environment:
# **DB_URL** is formatted like: <databasetype>://<username>:<password>@<hostname>:<port>/<database>
# Other examples are:
# - mysql://hedgedoc:password@database:3306/hedgedoc
# - sqlite:///data/sqlite.db (NOT RECOMMENDED)
# - For details see the official sequelize docs: http://docs.sequelizejs.com/en/v3/
- CMD_DB_URL=mysql://hedgedoc:password@database:3306/hedgedoc
# **DOMAIN** is the domain under which HedgeDoc will be available. Just the (sub)domain, no protocol or paths.
# You MUST change this if your instance should be available under another domain than "localhost", otherwise your
# instance may be broken.
# You can define only ONE domain.
- CMD_DOMAIN=localhost
# **PROTOCOL_USESSL** defines if generated links should be HTTPS URLs.
# This variable won't activate encryption on the listen port.
# Set this to true if your reverse proxy exposes your instance via HTTPS.
- CMD_PROTOCOL_USESSL=false
# **HSTS_ENABLE** defines if HSTS headers should be sent. Set this to true if you use HTTPS.
- CMD_HSTS_ENABLE=false
# **URL_ADDPORT** defines if generated links should contain the port.
# Set this to false if your reverse uses the default ports (443 for HTTPS or 80 for HTTP).
- CMD_URL_ADDPORT=true
volumes:
- uploads:/hedgedoc/public/uploads
ports:
# Ports that are published to the outside.
# The latter port is the port inside the container. It should always stay on 3000
# If you only specify a port it'll published on all interfaces. If you want to use a
# local reverse proxy, you may want to listen on 127.0.0.1.
# Example:
# - "127.0.0.1:3000:3000"
- "3000:3000"
networks:
backend:
restart: always
depends_on:
- database
# Define networks to allow best isolation
networks:
# Internal network for communication with PostgreSQL/MySQL
backend:
# Define named volumes so data stays in place
volumes:
# Volume for PostgreSQL/MySQL database
database:
uploads:
A different error occurs with MariaDB 10.6.5 and mariadb://
as the connection string as per the docs.
database_1 | 2022-01-27 17:45:27 0 [Note] mariadbd: ready for connections.
database_1 | Version: '10.6.5-MariaDB-1:10.6.5+maria~focal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
app_1 | /bin/sh: 1: git: not found
app_1 | 2022-01-27T17:45:31.606Z warn: Session secret not set. Using random generated one. Please set `sessionSecret` in your config.json file. All users will be logged out.
app_1 | 2022-01-27T17:45:32.294Z error: uncaughtException: this.throwError is not a function
app_1 | TypeError: this.throwError is not a function
app_1 | at Function.parseOkPacket (/hedgedoc/node_modules/mariadb/lib/cmd/command.js:144:26)
app_1 | at Query.readOKPacket (/hedgedoc/node_modules/mariadb/lib/cmd/resultset.js:141:30)
app_1 | at Query.readResponsePacket (/hedgedoc/node_modules/mariadb/lib/cmd/resultset.js:40:21)
app_1 | at PacketInputStream.receivePacketBasic (/hedgedoc/node_modules/mariadb/lib/io/packet-input-stream.js:104:9)
app_1 | at PacketInputStream.onData (/hedgedoc/node_modules/mariadb/lib/io/packet-input-stream.js:169:20)
app_1 | at Socket.emit (node:events:390:28)
app_1 | at addChunk (node:internal/streams/readable:315:12)
app_1 | at readableAddChunk (node:internal/streams/readable:289:9)
app_1 | at Socket.Readable.push (node:internal/streams/readable:228:10)
app_1 | at TCP.onStreamRead (node:internal/stream_base_commons:199:23)
app_1 | 2022-01-27T17:45:32.294Z error: An uncaught exception has occured.
app_1 | 2022-01-27T17:45:32.294Z error: this.throwError is not a function
app_1 | 2022-01-27T17:45:32.294Z error: Process will exit now.
database_1 | 2022-01-27 17:45:32 3 [Warning] Aborted connection 3 to db: 'hedgedoc' user: 'hedgedoc' host: '192.168.32.3' (Got an error reading communication packets)
I just tried to reproduce this using
version: '3'
services:
# MySQL example
# Most of the documentation that applies to PostgreSQL applies also to MySQL
database:
# You should be able to upgrade MySQL without problems
# but to make sure no even when a problem appears you
# should have a backup
image: mariadb:10.6 # mariadb:10.5 works!
environment:
- MYSQL_USER=hedgedoc
- MYSQL_PASSWORD=password
- MYSQL_DATABASE=hedgedoc
- MYSQL_ALLOW_EMPTY_PASSWORD=true
volumes:
- database:/var/lib/mysql
# This config provides UTF-8 support to the database by default
# If this config is not used, HedgeDoc breaks as it tries to write
# UTF-8 to a latin database.
- ./utf8.cnf:/etc/mysql/conf.d/utf8.cnf
networks:
backend:
restart: always
app:
# Uncomment the following section to build the image yourself:
#build:
# context: .
# dockerfile: debian/Dockerfile
# args:
# - "VERSION=master"
# - "HEDGEDOC_REPOSITORY=https://github.com/hedgedoc/hedgedoc.git"
image: quay.io/hedgedoc/hedgedoc:1.9.2
#mem_limit: 256mb # version 2 only
#memswap_limit: 512mb # version 2 only
#read_only: true # not supported in swarm mode, enable along with tmpfs
#tmpfs:
# - /tmp:size=10M
# # Make sure you remove this when you use filesystem as upload type
# - /hedgedoc/public/uploads:size=10M
environment:
# **DB_URL** is formatted like: <databasetype>://<username>:<password>@<hostname>:<port>/<database>
# Other examples are:
# - mysql://hedgedoc:password@database:3306/hedgedoc
# - sqlite:///data/sqlite.db (NOT RECOMMENDED)
# - For details see the official sequelize docs: http://docs.sequelizejs.com/en/v3/
- CMD_DB_URL=mariadb://hedgedoc:password@database:3306/hedgedoc
# **DOMAIN** is the domain under which HedgeDoc will be available. Just the (sub)domain, no protocol or paths.
# You MUST change this if your instance should be available under another domain than "localhost", otherwise your
# instance may be broken.
# You can define only ONE domain.
- CMD_DOMAIN=localhost
# **PROTOCOL_USESSL** defines if generated links should be HTTPS URLs.
# This variable won't activate encryption on the listen port.
# Set this to true if your reverse proxy exposes your instance via HTTPS.
- CMD_PROTOCOL_USESSL=false
# **HSTS_ENABLE** defines if HSTS headers should be sent. Set this to true if you use HTTPS.
- CMD_HSTS_ENABLE=false
# **URL_ADDPORT** defines if generated links should contain the port.
# Set this to false if your reverse uses the default ports (443 for HTTPS or 80 for HTTP).
- CMD_URL_ADDPORT=true
volumes:
- uploads:/hedgedoc/public/uploads
ports:
# Ports that are published to the outside.
# The latter port is the port inside the container. It should always stay on 3000
# If you only specify a port it'll published on all interfaces. If you want to use a
# local reverse proxy, you may want to listen on 127.0.0.1.
# Example:
# - "127.0.0.1:3000:3000"
- "3000:3000"
networks:
backend:
restart: always
depends_on:
- database
# Define networks to allow best isolation
networks:
# Internal network for communication with PostgreSQL/MySQL
backend:
# Define named volumes so data stays in place
volumes:
# Volume for PostgreSQL/MySQL database
database:
uploads:
This is the file you posted with mariadb
pinned to 10.6
and using the mariadb://
driver.
Everything seems to work fine using this config.
EDIT: Also works using mysql://
in the database URI.
Very strange indeed. I can reproduce this on macOS and Linux:
mysql
connection string:
$ docker-compose config && docker-compose up
networks:
backend: {}
services:
app:
depends_on:
database:
condition: service_started
environment:
CMD_DB_URL: mysql://hedgedoc:password@database:3306/hedgedoc
CMD_DOMAIN: localhost
CMD_HSTS_ENABLE: "false"
CMD_PROTOCOL_USESSL: "false"
CMD_URL_ADDPORT: "true"
image: quay.io/hedgedoc/hedgedoc:1.9.2
networks:
backend: null
ports:
- published: 3000
target: 3000
restart: always
volumes:
- uploads:/hedgedoc/public/uploads:rw
database:
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: "true"
MYSQL_DATABASE: hedgedoc
MYSQL_PASSWORD: password
MYSQL_USER: hedgedoc
image: mariadb:10.6
networks:
backend: null
restart: always
volumes:
- database:/var/lib/mysql:rw
- /home/agross/hedge/utf8.cnf:/etc/mysql/conf.d/utf8.cnf:rw
version: '3'
volumes:
database: {}
uploads: {}
Creating network "hedge_backend" with the default driver
Creating hedge_database_1 ... done
Creating hedge_app_1 ... done
Attaching to hedge_database_1, hedge_app_1
database_1 | 2022-01-27 19:53:05+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
database_1 | 2022-01-27 19:53:05+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
database_1 | 2022-01-27 19:53:05+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
database_1 | 2022-01-27 19:53:05 0 [Note] mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal) starting as process 1 ...
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Number of pools: 1
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
database_1 | 2022-01-27 19:53:05 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Using Linux native AIO
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Completed initialization of buffer pool
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: 128 rollback segments are active.
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Creating shared tablespace for temporary tables
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: 10.6.5 started; log sequence number 42347; transaction id 14
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
database_1 | 2022-01-27 19:53:05 0 [Note] Plugin 'FEEDBACK' is disabled.
database_1 | 2022-01-27 19:53:05 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
database_1 | 2022-01-27 19:53:05 0 [Note] Server socket created on IP: '0.0.0.0'.
database_1 | 2022-01-27 19:53:05 0 [Note] Server socket created on IP: '::'.
database_1 | 2022-01-27 19:53:05 0 [Warning] 'proxies_priv' entry '@% root@4288b977ad8f' ignored in --skip-name-resolve mode.
database_1 | 2022-01-27 19:53:05 0 [Note] InnoDB: Buffer pool(s) load completed at 220127 19:53:05
database_1 | 2022-01-27 19:53:05 0 [Note] mariadbd: ready for connections.
database_1 | Version: '10.6.5-MariaDB-1:10.6.5+maria~focal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
app_1 | /bin/sh: 1: git: not found
app_1 | 2022-01-27T19:53:09.310Z warn: Session secret not set. Using random generated one. Please set `sessionSecret` in your config.json file. All users will be logged out.
app_1 | 2022-01-27T19:53:09.649Z warn: Database cannot be reached. Try 1 of 30.
app_1 | 2022-01-27T19:53:10.714Z warn: Database cannot be reached. Try 2 of 30.
app_1 | 2022-01-27T19:53:11.716Z warn: Database cannot be reached. Try 3 of 30.
^CGracefully stopping... (press Ctrl+C again to force)
Stopping hedge_app_1 ... done
Stopping hedge_database_1 ... done
mariadb
connection string:
$ docker-compose config && docker-compose up
networks:
backend: {}
services:
app:
depends_on:
database:
condition: service_started
environment:
CMD_DB_URL: mariadb://hedgedoc:password@database:3306/hedgedoc
CMD_DOMAIN: localhost
CMD_HSTS_ENABLE: "false"
CMD_PROTOCOL_USESSL: "false"
CMD_URL_ADDPORT: "true"
image: quay.io/hedgedoc/hedgedoc:1.9.2
networks:
backend: null
ports:
- published: 3000
target: 3000
restart: always
volumes:
- uploads:/hedgedoc/public/uploads:rw
database:
environment:
MYSQL_ALLOW_EMPTY_PASSWORD: "true"
MYSQL_DATABASE: hedgedoc
MYSQL_PASSWORD: password
MYSQL_USER: hedgedoc
image: mariadb:10.6
networks:
backend: null
restart: always
volumes:
- database:/var/lib/mysql:rw
- /home/agross/hedge/utf8.cnf:/etc/mysql/conf.d/utf8.cnf:rw
version: '3'
volumes:
database: {}
uploads: {}
Creating network "hedge_backend" with the default driver
Creating hedge_database_1 ... done
Creating hedge_app_1 ... done
Attaching to hedge_database_1, hedge_app_1
database_1 | 2022-01-27 19:55:56+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
database_1 | 2022-01-27 19:55:56+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
database_1 | 2022-01-27 19:55:56+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.6.5+maria~focal started.
database_1 | 2022-01-27 19:55:56 0 [Note] mariadbd (server 10.6.5-MariaDB-1:10.6.5+maria~focal) starting as process 1 ...
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Number of pools: 1
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
database_1 | 2022-01-27 19:55:56 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Using Linux native AIO
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Completed initialization of buffer pool
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: 128 rollback segments are active.
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Creating shared tablespace for temporary tables
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: 10.6.5 started; log sequence number 42359; transaction id 14
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
database_1 | 2022-01-27 19:55:56 0 [Note] Plugin 'FEEDBACK' is disabled.
database_1 | 2022-01-27 19:55:56 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
database_1 | 2022-01-27 19:55:56 0 [Note] Server socket created on IP: '0.0.0.0'.
database_1 | 2022-01-27 19:55:56 0 [Note] Server socket created on IP: '::'.
database_1 | 2022-01-27 19:55:56 0 [Note] InnoDB: Buffer pool(s) load completed at 220127 19:55:56
database_1 | 2022-01-27 19:55:56 0 [Warning] 'proxies_priv' entry '@% root@4288b977ad8f' ignored in --skip-name-resolve mode.
database_1 | 2022-01-27 19:55:56 0 [Note] mariadbd: ready for connections.
database_1 | Version: '10.6.5-MariaDB-1:10.6.5+maria~focal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
app_1 | /bin/sh: 1: git: not found
app_1 | 2022-01-27T19:55:59.720Z warn: Session secret not set. Using random generated one. Please set `sessionSecret` in your config.json file. All users will be logged out.
app_1 | 2022-01-27T19:56:00.042Z error: uncaughtException: this.throwError is not a function
app_1 | TypeError: this.throwError is not a function
app_1 | at Function.parseOkPacket (/hedgedoc/node_modules/mariadb/lib/cmd/command.js:144:26)
app_1 | at Query.readOKPacket (/hedgedoc/node_modules/mariadb/lib/cmd/resultset.js:141:30)
app_1 | at Query.readResponsePacket (/hedgedoc/node_modules/mariadb/lib/cmd/resultset.js:40:21)
app_1 | at PacketInputStream.receivePacketBasic (/hedgedoc/node_modules/mariadb/lib/io/packet-input-stream.js:104:9)
app_1 | at PacketInputStream.onData (/hedgedoc/node_modules/mariadb/lib/io/packet-input-stream.js:169:20)
app_1 | at Socket.emit (node:events:390:28)
app_1 | at addChunk (node:internal/streams/readable:315:12)
app_1 | at readableAddChunk (node:internal/streams/readable:289:9)
app_1 | at Socket.Readable.push (node:internal/streams/readable:228:10)
app_1 | at TCP.onStreamRead (node:internal/stream_base_commons:199:23)
app_1 | 2022-01-27T19:56:00.042Z error: An uncaught exception has occured.
app_1 | 2022-01-27T19:56:00.042Z error: this.throwError is not a function
app_1 | 2022-01-27T19:56:00.042Z error: Process will exit now.
database_1 | 2022-01-27 19:56:00 3 [Warning] Aborted connection 3 to db: 'hedgedoc' user: 'hedgedoc' host: '192.168.48.3' (Got an error reading communication packets)
^CGracefully stopping... (press Ctrl+C again to force)
Stopping hedge_app_1 ... done
Stopping hedge_database_1 ... done
I tried your docker-compose files on my linux system and they work. 🤔
Also works on my MBP.
@agross Do you have some kind of firewall or an unusual network setup that may interfere with the Docker network? Your stack trace containing PacketInputStream.receivePacketBasic
and TCP.onStreamRead
may point to networking issues.
Nothing special, just Fedora Linux. I just tried again on a Hetzner server (also Fedora). I also ran tcpdump in the MariaDB container, there are packets flowing between the app and the database.
What is really strange is that MariaDB 10.5 works.
Which Linux distribution did you test on?
I tested on Fedora 35 with Docker 20.10.12
Same here 🤔
Can you please post your SSH public keys here or send them to me (agross at therightstuff.de). I'll set up a a server that shows the behavior.
Hint: You can append .keys
to the URL of a GitHub profile to get that user's public keys.
@ErikMichelson Nice, didn’t know that! Thanks!
@davidmehren @mrdrogdrog I've added your keys to the test server. Please see for yourself:
$ ssh root@162.55.216.35
Last login: Mon Feb 7 12:00:50 2022 from 79.216.188.169
[root@hedgedoc ~]# cd hedgedoc
[root@hedgedoc hedgedoc]# docker-compose up
Creating network "hedgedoc_backend" with the default driver
Creating hedgedoc_database_1 ... done
Creating hedgedoc_app_1 ... done
Attaching to hedgedoc_database_1, hedgedoc_app_1
...
app_1 | /bin/sh: 1: git: not found
app_1 | 2022-02-07T12:07:14.600Z warn: Session secret not set. Using random generated one. Please set `sessionSecret` in your config.json file. All users will be logged out.
app_1 | 2022-02-07T12:07:15.443Z warn: Database cannot be reached. Try 1 of 30.
app_1 | 2022-02-07T12:07:16.461Z warn: Database cannot be reached. Try 2 of 30.
...
Using image: quay.io/hedgedoc/hedgedoc-nightly:debian
which prints out the database error, I get
Database cannot be reached. Try 1 of 30. (Error: Encoding not recognized: 'undefined' (searched as: 'undefined'))
After commenting out the utf8.cnf
volume for the maria container, HedgeDoc starts correctly. So there seems to be an issue with the encoding config we provide.
I can reproduce locally that our config stops working with MariaDB 10.6.
There is a fix in #287, but I'm not sure if we can merge it, as it will break old MariaDB installations.
Thank you for your time researching this issue. I'm going to take down the test server now.