Postgres driver not found in ARM64 and ARM32 Docker images
Kexkey opened this issue · 6 comments
Issue and Steps to Reproduce
Using CLN with PostgreSQL works with the AMD64 Docker image, but not with the ARM32 and ARM64 ones.
In the following outputs, we can see Unable to find DB driver for postgres://... for ARM32/ARM64 vs Error calling DB setup: Could not connect to postgres://... for AMD64.
Running the CLN Docker image using Postgres on ARM32 (on a RPi3):
docker run --rm -it --entrypoint bash elementsproject/lightningd:v24.08
root@56aea353d694:/# lightningd --wallet=postgres://whatever:CHANGEME@whatever:5432/whatever
2024-09-06T13:20:44.926Z INFO lightningd: v24.08
2024-09-06T13:20:48.540Z INFO lightningd: Creating configuration directory /root/.lightning/bitcoin
2024-09-06T13:20:54.470Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
2024-09-06T13:20:54.470Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
2024-09-06T13:20:54.642Z **BROKEN** lightningd: Unable to find DB driver for postgres://whatever:CHANGEME@whatever:5432/whatever
Unable to find DB driver for postgres://whatever:CHANGEME@whatever:5432/whatever
Running the CLN Docker image using Postgres on ARM64 (on a Mac M1):
docker run --rm -it --entrypoint bash elementsproject/lightningd:v24.08
root@8e781ccf42ca:/# lightningd --wallet=postgres://whatever:CHANGEME@whatever:5432/whatever
2024-09-06T13:19:59.859Z INFO lightningd: v24.08
2024-09-06T13:20:00.153Z INFO lightningd: Creating configuration directory /root/.lightning/bitcoin
2024-09-06T13:20:00.458Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
2024-09-06T13:20:00.458Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
2024-09-06T13:20:00.469Z **BROKEN** lightningd: Unable to find DB driver for postgres://whatever:CHANGEME@whatever:5432/whatever
Unable to find DB driver for postgres://whatever:CHANGEME@whatever:5432/whatever
Running the CLN Docker image using Postgres on AMD64 (on a PC):
docker run --rm -it --entrypoint bash elementsproject/lightningd:v24.08
root@6ae50e3bfb84:/# lightningd --wallet=postgres://whatever:CHANGEME@whatever:5432/whatever
2024-09-06T13:21:26.683Z INFO lightningd: v24.08
2024-09-06T13:21:28.844Z INFO lightningd: Creating configuration directory /root/.lightning/bitcoin
2024-09-06T13:21:35.513Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
2024-09-06T13:21:35.513Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
2024-09-06T13:21:36.379Z **BROKEN** lightningd: Error calling DB setup: Could not connect to postgres://whatever:CHANGEME@whatever:5432/whatever: could not translate host name "whatever" to address: Name or service not known?
Error calling DB setup: Could not connect to postgres://whatever:CHANGEME@whatever:5432/whatever: could not translate host name "whatever" to address: Name or service not known
Version
Using the Docker hub's elementsproject/lightningd:v24.08 image. Same error happens with elementsproject/lightningd:v24.05.
More info
I also built from sources locally (on AMD64) and got the same results. I also tried adding postgresql package in the base-builder stage as well as postgresql and libpq-dev packages in the final stage, with the same results.
I noticed the problem first on a working setup with the Postgres server up and running, so the error above is not due to the pg server not running but the driver libraries not found.
@Kexkey Can you please confirm your last working CLN version setup on arm64 with postgress?
Hi @ShahanaFarooqui,
I've always used Docker images that I wrote myself because the ones provided in the CLN repo were not working on ARM32. For Cyphernode, I want to make sure all the images work on AMD64, ARM32 and ARM64.
I think the last version I successfully built was v23.11.2: https://github.com/SatoshiPortal/dockers/blob/342a94bc9d1b2b219731aed7d802bcc87f7aaee4/c-lightning/Dockerfile
I failed to build v24+ for RPi2-3, unfortunately. The Python dependancies are a pita when building for multiple platforms. When I saw your work on the newer Dockerfiles (thanks btw!) I decided to give them a try. Docker buildx is pretty neat, I've never used it before. :) I used to build each arch on its respective hardware. Building on an RPi2 takes forever, lol!
PS: I managed to use your Dockerfile to cross-compile on my Mac M1... Got the same results as when I compile on AMD64, re. no postgres drivers on ARM64.
@Kexkey I completely agree on buildx—it made my life much easier too! Credit for the new Dockerfile goes to @NicolasDorier.
As for PostgreSQL support, it appears that it is already available for both ARM64 and ARM32 architectures (according to ChatGPT). I am adding this as an enhancement for a future release, as it was not previously supported in the CLN images.
So just to hop in. Long time ago, I got same problem where some client tooling weren't available on arm32 (only arm64 and amd64).
The way I hacked around it is by extracting the CLI tooling from a postgres docker image, then downloading it rather than passing by apt-get.
I would have expected ARM64 have the apt packages though.
I am not sure if you are experiencing the same problem or not.
Thanks Nicolas! After reading your comment, I thought I'd try to base the final stage on the postgres image instead of the current ${BASE_DISTRO}.
It worked! We can obviously expect Postgres' official images to have all the Postgres drivers. :)
More specifically, changing
FROM ${BASE_DISTRO} AS final
to
FROM postgres:17.0-bullseye AS final
I don't think it's the right solution though, since there are a lot of stuff in the Postgres image that we don't need for CLN. At least for now I'll do this. And it gives hints for a better way to make it work.
PS: haven't tried it on ARM32 yet.
This is on a Mac M1:
Before (Unable to find DB driver for postgres):
docker run --rm -it elementsproject/lightningd:v24.08 --wallet=postgres://whatever:CHANGEME@whatever:5432/whatever
Core-Lightning starting
Couldn't watch /root/.lightning/bitcoin: No such file or directory
lightningd --network="${LIGHTNINGD_NETWORK}" "$@"
2024-10-03T15:46:12.901Z INFO lightningd: v24.08
2024-10-03T15:46:13.053Z INFO lightningd: Creating configuration directory /root/.lightning/bitcoin
2024-10-03T15:46:13.400Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
2024-10-03T15:46:13.400Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
2024-10-03T15:46:13.416Z **BROKEN** lightningd: Unable to find DB driver for postgres://whatever:CHANGEME@whatever:5432/whatever
Unable to find DB driver for postgres://whatever:CHANGEME@whatever:5432/whatever
After (Error calling DB setup: Could not connect to postgres):
docker run --rm -it cln-pg-based:v24.08 --wallet=postgres://whatever:CHANGEME@whatever:5432/whatever
Core-Lightning starting
Couldn't watch /root/.lightning/bitcoin: No such file or directory
lightningd --network="${LIGHTNINGD_NETWORK}" "$@"
2024-10-03T15:46:31.860Z INFO lightningd: v24.08-modded
2024-10-03T15:46:31.965Z INFO lightningd: Creating configuration directory /root/.lightning/bitcoin
2024-10-03T15:46:32.162Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
2024-10-03T15:46:32.162Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
2024-10-03T15:46:32.175Z **BROKEN** lightningd: Error calling DB setup: Could not connect to postgres://whatever:CHANGEME@whatever:5432/whatever: could not translate host name "whatever" to address: Name or service not known?
Error calling DB setup: Could not connect to postgres://whatever:CHANGEME@whatever:5432/whatever: could not translate host name "whatever" to address: Name or service not known
EDIT: I forgot to tell you that you need to install postgresql in the base-builder stage for the configure to pick it up... for the ARM64 arch.
Bounty available on this: https://community.sphinx.chat/bounty/2628