SSL support is not compiled (libpq)
manifest opened this issue ยท 13 comments
I'm getting an error from the application with Diesel dependency that should have compiled statically with SSL
[2019-01-22T06:11:14Z ERROR r2d2] sslmode value "require" invalid when SSL support is not compiled in
I've compiled application with the following command
OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include cargo build --release
Entire Dockerfile
## -----------------------------------------------------------------------------
## Build
## -----------------------------------------------------------------------------
FROM clux/muslrust as build-stage
WORKDIR "/build"
COPY . .
RUN OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include cargo build --release
## -----------------------------------------------------------------------------
## Package
## -----------------------------------------------------------------------------
FROM alpine
COPY --from=build-stage "/build/target/x86_64-unknown-linux-musl/release/example" "/app/example"
WORKDIR "/app"
ENTRYPOINT ["/app/example"]
App's Cargo.toml
[dependencies]
diesel = { version = "1.3.3", features = ["postgres", "uuid", "serde_json", "r2d2"] }
openssl = "*"
[patch.crates-io]
diesel = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
diesel_derives = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
The same error with the command:
OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include PQ_LIB_STATIC=yes PQ_LIB_DIR=/musl/lib/ cargo build --release
And Cargo.toml
[dependencies]
diesel = { version = "1.3.3", features = ["postgres", "uuid", "serde_json", "r2d2"] }
openssl = "*"
pq-sys = "*"
[patch.crates-io]
diesel = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
diesel_derives = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
I've played around libpg compilation adding --with-openssl
to postgresql`s configure script and start getting an error on compilation
= note: /build/target/x86_64-unknown-linux-musl/release/deps/libpq_sys-784bf4f1583faa91.rlib(fe-secure-openssl.o): In function `pgtls_init':
fe-secure-openssl.c:(.text+0xf48): undefined reference to `OPENSSL_config'
collect2: error: ld returned 1 exit status
cd postgresql-$PQ_VER && \
CC="musl-gcc -fPIE -pie" LDFLAGS="-L$PREFIX/lib" CFLAGS="-I$PREFIX/include" ./configure \
+ --with-openssl \
Hey, thanks for raising this! This should indeed be supported. curl goes through the same dance.
It's probably a bit annoying to get it to work though. I'll try to have a look on the weekend, but can't promise anything.
In the mean time, any chance you can commit what you have as a test case in a PR?
@clux I've published a test case for the issue here.
Steps to reproduce:
- Create a database
## Database conenction URL must contain `?sslmode=require`
## query string parameter to reproduce the issue
export DATABASE_URL='postgres://localhost:5432/diesel-muslrust-example?sslmode=require'
diesel setup --database-url ${DATABASE_URL}
- Build and run a docker container
docker build -t diesel-muslrust-example -f docker/Dockerfile .
docker run --rm -ti --entrypoint=/bin/sh diesel-muslrust-example
- Run the example app from within the container
## The app reads 'DATABASE_URL' from environment variable, so it should be exported
export DATABASE_URL=''
RUST_LOG=diesel_muslrust_example=info /app/diesel-muslrust-example
- You should see an error after connection will fail with timeout
thread 'main' panicked at 'Error creating a database pool: Error(Some("sslmode value \"require\" invalid when SSL support is not compiled in\n"))', src/libcore/result.rs:999:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
Ah, sorry not yet. Busy weeks ๐
Had a bit of a go today and added failing tests from your code in 1677445
Unfortunately, no matter what permutations i tried libpq
will build, but cargo build will fail to link due to missing SSL_ symbols. Leaving a comment for the future with current status.
Testing method now is:
- tweak
Dockerfile
make clean
make build
make test-dieselpgssl
There's quite a lot of flags in libpq's ./configure
though:
./configure -h
Usage: ./configure [OPTION]... [VAR=VALUE]...
To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE. See below for descriptions of some of the useful variables.
Defaults for the options are specified in brackets.
Configuration:
-h, --help display this help and exit
--help=short display options specific to this package
--help=recursive display the short help of all the included packages
-V, --version display version information and exit
-q, --quiet, --silent do not print `checking ...' messages
--cache-file=FILE cache test results in FILE [disabled]
-C, --config-cache alias for `--cache-file=config.cache'
-n, --no-create do not create output files
--srcdir=DIR find the sources in DIR [configure dir or `..']
Installation directories:
--prefix=PREFIX install architecture-independent files in PREFIX
[/usr/local/pgsql]
--exec-prefix=EPREFIX install architecture-dependent files in EPREFIX
[PREFIX]
By default, `make install' will install all the files in
`/usr/local/pgsql/bin', `/usr/local/pgsql/lib' etc. You can specify
an installation prefix other than `/usr/local/pgsql' using `--prefix',
for instance `--prefix=$HOME'.
For better control, use the options below.
Fine tuning of the installation directories:
--bindir=DIR user executables [EPREFIX/bin]
--sbindir=DIR system admin executables [EPREFIX/sbin]
--libexecdir=DIR program executables [EPREFIX/libexec]
--sysconfdir=DIR read-only single-machine data [PREFIX/etc]
--sharedstatedir=DIR modifiable architecture-independent data [PREFIX/com]
--localstatedir=DIR modifiable single-machine data [PREFIX/var]
--libdir=DIR object code libraries [EPREFIX/lib]
--includedir=DIR C header files [PREFIX/include]
--oldincludedir=DIR C header files for non-gcc [/usr/include]
--datarootdir=DIR read-only arch.-independent data root [PREFIX/share]
--datadir=DIR read-only architecture-independent data [DATAROOTDIR]
--infodir=DIR info documentation [DATAROOTDIR/info]
--localedir=DIR locale-dependent data [DATAROOTDIR/locale]
--mandir=DIR man documentation [DATAROOTDIR/man]
--docdir=DIR documentation root [DATAROOTDIR/doc/postgresql]
--htmldir=DIR html documentation [DOCDIR]
--dvidir=DIR dvi documentation [DOCDIR]
--pdfdir=DIR pdf documentation [DOCDIR]
--psdir=DIR ps documentation [DOCDIR]
System types:
--build=BUILD configure for building on BUILD [guessed]
--host=HOST cross-compile to build programs to run on HOST [BUILD]
Optional Features:
--disable-option-checking ignore unrecognized --enable/--with options
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
--disable-integer-datetimes
obsolete option, no longer supported
--enable-nls[=LANGUAGES]
enable Native Language Support
--disable-rpath do not embed shared library search path in
executables
--disable-spinlocks do not use spinlocks
--disable-atomics do not use atomic operations
--disable-strong-random do not use a strong random number source
--enable-debug build with debugging symbols (-g)
--enable-profiling build with profiling enabled
--enable-coverage build with coverage testing instrumentation
--enable-dtrace build with DTrace support
--enable-tap-tests enable TAP tests (requires Perl and IPC::Run)
--enable-depend turn on automatic dependency tracking
--enable-cassert enable assertion checks (for debugging)
--disable-thread-safety disable thread-safety in client libraries
--disable-largefile omit support for large files
--disable-float4-byval disable float4 passed by value
--disable-float8-byval disable float8 passed by value
Optional Packages:
--with-PACKAGE[=ARG] use PACKAGE [ARG=yes]
--without-PACKAGE do not use PACKAGE (same as --with-PACKAGE=no)
--with-extra-version=STRING
append STRING to version
--with-template=NAME override operating system template
--with-includes=DIRS look for additional header files in DIRS
--with-libraries=DIRS look for additional libraries in DIRS
--with-libs=DIRS alternative spelling of --with-libraries
--with-pgport=PORTNUM set default port number [5432]
--with-blocksize=BLOCKSIZE
set table block size in kB [8]
--with-segsize=SEGSIZE set table segment size in GB [1]
--with-wal-blocksize=BLOCKSIZE
set WAL block size in kB [8]
--with-wal-segsize=SEGSIZE
set WAL segment size in MB [16]
--with-CC=CMD set compiler (deprecated)
--with-icu build with ICU support
--with-tcl build Tcl modules (PL/Tcl)
--with-tclconfig=DIR tclConfig.sh is in DIR
--with-perl build Perl modules (PL/Perl)
--with-python build Python modules (PL/Python)
--with-gssapi build with GSSAPI support
--with-krb-srvnam=NAME default service principal name in Kerberos (GSSAPI)
[postgres]
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
--with-bonjour build with Bonjour support
--with-openssl build with OpenSSL support
--with-selinux build with SELinux support
--with-systemd build with systemd support
--without-readline do not use GNU Readline nor BSD Libedit for editing
--with-libedit-preferred
prefer BSD Libedit over GNU Readline
--with-uuid=LIB build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
--with-ossp-uuid obsolete spelling of --with-uuid=ossp
--with-libxml build with XML support
--with-libxslt use XSLT support when building contrib/xml2
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
CC C compiler command
CFLAGS C compiler flags
LDFLAGS linker flags, e.g. -L<lib dir> if you have libraries in a
nonstandard directory <lib dir>
LIBS libraries to pass to the linker, e.g. -l<library>
CPPFLAGS (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
you have headers in a nonstandard directory <include dir>
CPP C preprocessor
PKG_CONFIG path to pkg-config utility
PKG_CONFIG_PATH
directories to add to pkg-config's search path
PKG_CONFIG_LIBDIR
path overriding pkg-config's built-in search path
ICU_CFLAGS C compiler flags for ICU, overriding pkg-config
ICU_LIBS linker flags for ICU, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.
There might be one in there that'll fix it.
Otherwise, there might be some magic in https://github.com/sgrif/pq-sys/blob/master/build.rs that we can use.
oh, and saw this once during compilation of libpq:
configure: WARNING: openssl/ssl.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: openssl/ssl.h: proceeding with the compiler's result
got rid of this warning by passing:
--with-libraries=$PREFIX/lib \
--with-includes=$PREFIX/include \
to ./configure
. didn't seem to help any further though.
Another diff that may or may not help bring this closer:
diff --git a/Dockerfile b/Dockerfile
index 9899a4f..46448c1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -45,7 +45,7 @@ RUN curl https://sh.rustup.rs -sSf | \
ENV SSL_VER="1.0.2q" \
CURL_VER="7.64.0" \
ZLIB_VER="1.2.11" \
- PQ_VER="10.6" \
+ PQ_VER="9.6.8" \
SQLITE_VER="3270200" \
CC=musl-gcc \
PREFIX=/musl \
@@ -91,10 +91,14 @@ RUN curl -sSL https://curl.haxx.se/download/curl-$CURL_VER.tar.gz | tar xz && \
# Build libpq
# TODO: fix so that --with-openssl works with pqssl tests
-RUN curl -sSL https://ftp.postgresql.org/pub/source/v$PQ_VER/postgresql-$PQ_VER.tar.gz | tar xz && \
- cd postgresql-$PQ_VER && \
+RUN curl -sSL https://ftp.postgresql.org/pub/source/v$PQ_VER/postgresql-$PQ_VER.tar.gz | tar xz
+RUN cd postgresql-$PQ_VER && \
CC="musl-gcc -fPIE -pie" LDFLAGS="-L$PREFIX/lib" CFLAGS="-I$PREFIX/include" ./configure \
--without-readline \
+ --with-openssl \
+ --with-libraries=$PREFIX/lib \
+ --with-includes=$PREFIX/include \
+ --oldincludedir=/usr/include \
--prefix=$PREFIX --host=x86_64-unknown-linux-musl && \
cd src/interfaces/libpq make -s -j$(nproc) all-static-lib && make -s install install-lib-static && \
cd ../../bin/pg_config && make -j $(nproc) && make install && \
@@ -121,13 +125,19 @@ ENV PATH=$PREFIX/bin:$PATH \
PKG_CONFIG_ALLOW_CROSS=true \
PKG_CONFIG_ALL_STATIC=true \
PQ_LIB_STATIC_X86_64_UNKNOWN_LINUX_MUSL=true \
+ PQ_LIB_STATIC=true \
+ PQ_LIB_DIR=$PREFIX/lib/ \
+ OPENSSL_LIB_DIR=$PREFIX/lib/ \
+ OPENSSL_INCLUDE_DIR=$PREFIX/include \
+ DEP_OPENSSL_INCLUDE=$PREFIX/include/ \
PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig \
PG_CONFIG_X86_64_UNKNOWN_LINUX_GNU=/usr/bin/pg_config \
OPENSSL_STATIC=true \
OPENSSL_DIR=$PREFIX \
SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
SSL_CERT_DIR=/etc/ssl/certs \
- LIBZ_SYS_STATIC=1
+ LIBZ_SYS_STATIC=1 \
+ TARGET=musl
# Allow ditching the -w /volume flag to docker run
WORKDIR /volume
the issue might be libpq
configuration based on this output from its detection:
checking for CRYPTO_new_ex_data in -lcrypto... yes
checking for SSL_new in -lssl... yes
checking for SSL_get_current_compression... yes
checking for OPENSSL_init_ssl... no
checking for BIO_get_data... no
checking for BIO_meth_new... no
checking for ASN1_STRING_get0_data... no
checking for RAND_OpenSSL... no
checking for CRYPTO_lock... yes
but maybe that's just openssl 1.1 stuff it's testing for ๐ค
Hello! I'm the maintainer of rust-musl-builder
, and we ran into this problem lately. You might be interested in the following issues:
TL;dr: It's possible to work around this by adding this to main.rs
:
extern crate openssl;
#[macro_use]
extern crate diesel;
...but the workaround might break with certain projects or releases of Rust. I've filed a bug upstream to investigate further: sgrif/pq-sys#25
Oh wow, thanks a lot @emk ! It does look like the order of the includes actually fixes this!
Had a quick check compiling pg with --with-openssl
and make test-dieselpgssl
now both links and runs in this repo without ssl errors.
Have made openssl required now, since there's now way of having the support without pulling in the crate, but it's documented in the main README. Will keep an eye on pq-sys#25 for a potential future improvement on this.