Please downgrade keydb to 6.3.3

[sergiodj]

Hi,

keydb is currently affected by Snapchat/KeyDB#802 and can fail to start on some machines (mine included) when using the latest version 6.3.4. I have checked that version 6.3.3 works fine, and I'm manually downgrading the image for now, but I believe this should be done officially because other people might be impacted by this problem and it's a bit tricky to determine its root cause.

Thanks.

[estribiyo]

Downgrading to 6.3.3 worked for me.

[UehgxEQbFiw4Tlw8]

Hello everyone,

could you please provide information about how did you downgrade?

I made this way in my vars.yml:

keydb_enabled: true
keydb_version: 6.3.3
keydb_arch: arm64
keydb_container_image_force_pull: "{{ keydb_container_image.endswith(':arm64_v6.3.3') }}"

but it didn't work and I'm not 100% sure about the last line, which in main.yml has :latest and not the particular version in it.
As you can see, I'm on arm64 (should be... Raspi5 with last Raspberry OS installed and regularly updated).

I encountered the problem trying to install Funkwhale on it, which succeed some time ago on a Raspberry 4 with same OS but previous Debian basis (was Bookworm? I don't remeber).

Now I have an error message (following Ansible's hints):

systemctl status mash-keydb.service
● mash-keydb.service - keydb (mash-keydb)
Loaded: loaded (/etc/systemd/system/mash-keydb.service; enabled; preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Thu 2024-11-21 18:24:22 CET; 21s ago
Process: 859620 ExecStartPre=/usr/bin/env sh -c /usr/bin/env docker stop --time=30 mash-keydb 2>/dev/null || true (>
Process: 859630 ExecStartPre=/usr/bin/env sh -c /usr/bin/env docker rm mash-keydb 2>/dev/null || true (code=exited,>
Process: 859639 ExecStartPre=/usr/bin/env docker create --rm --name=mash-keydb --log-driver=none --user=995:1002 -->
Process: 859648 ExecStart=/usr/bin/env docker start --attach mash-keydb (code=exited, status=133)
Main PID: 859648 (code=exited, status=133)
CPU: 85ms

Nov 21 18:24:22 XXXXXX systemd[1]: mash-keydb.service: Failed with result 'exit-code'.

And:

user@hostname:~ $ journalctl -fu mash-keydb.service
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: : Unsupported system page size
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: : Unsupported system page size
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: : Unsupported system page size
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: : Unsupported system page size
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: Error in GnuTLS initialization: ASN1 parser: Element was not found.
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: : Unsupported system page size
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: : Unsupported system page size
Nov 21 18:26:34 RPiMASH5 mash-keydb[863271]: zmalloc: Out of memory trying to allocate 48 bytes
Nov 21 18:26:34 RPiMASH5 systemd[1]: mash-keydb.service: Main process exited, code=exited, status=133/n/a
Nov 21 18:26:34 RPiMASH5 systemd[1]: mash-keydb.service: Failed with result 'exit-code'.

These are waters to deep for me...
Help will be highly appreciated! Thank you!

Best regards,
M.

[spantaleev]

You only need to add keydb_version: 6.3.3 to your existing vars.yml configuration to pin the role to this lower version.

You don't really need to override keydb_arch, keydb_container_image_force_pull, etc.

[UehgxEQbFiw4Tlw8]

Done, thank you, I modified the vars.yml:

keydb_enabled: true
keydb_version: 6.3.3
#keydb_arch: arm64
#keydb_container_image_force_pull: "{{ keydb_container_image.endswith(':arm64_v6.3.3') }}"

rerun with a just setup-all, but it didn't succed, same errors.
I've also tried to (re)start the service manually via systemctl (re)start but it doesn't work.

Btw. and FYI all of the Process lines in the comment above are truncated but terminated with a status0/SUCCESS - except the last:
Process: 859648 ExecStart=/usr/bin/env docker start --attach mash-keydb (code=exited, status=133)

I will try to better understand the meaning of those errors, but it could require quite some time...

Failed at:
TASK [galaxy/systemd_service_manager : Fail if service isn't detected to be running] ***********************************
skipping: [mash.example.com] => (item=mash-postgres.service)
failed: [mash.example.com] (item=mash-keydb.service) => changed=false
ansible_loop_var: item
item: mash-keydb.service
msg: mash-keydb.service was not detected to be running. It's possible that there's a configuration problem or another service on your server interferes with it (uses the same ports, etc.). Try running systemctl status mash-keydb.service and journalctl -fu mash-keydb.service on the server to investigate. If you're on a slow or overloaded server, it may be that services take a longer time to start and that this error is a false-positive. You can consider raising the value of the devture_systemd_service_manager_up_verification_delay_seconds variable. See /home/pi/mash-playbook/roles/galaxy/systemd_service_manager/defaults/main.yml for more details about that.

The server is not overloaded, in parallel I have your matrix-docker-ansible-deploy, but very few local users, working very smooth ...

Thanks a lot for help!

Best, M.

[spantaleev]

Please try the new Valkey service instead of using KeyDB.

It should be as simple as:

• updating the playbook and its roles (git pull && just roles)
• replacing keydb references in your vars.yml file with valkey
• re-running the playbook

Hopefully Valkey would work much better across servers with various CPU capabilities.

If you find the mash-keydb service is not automatically uninstalled from your server, you can also do it manually:

• systemctl disable --now mash-keydb
• rm -rf /mash/keydb

[UehgxEQbFiw4Tlw8]

Hello,

switching to Valkey has solved all my problems. I have now a functioning funkwhale (on a RasPi 5)
Thank you so much for your efforts!
As for my part, this can be closed, but maybe the other users would like to comment.
Wishing you all a very nice weekend,

Best
M.