redis-rb/redis-client

does not compile on musl system (alpinelinux)

Closed this issue · 7 comments

tomhub commented

In my case, gem install hiredis-client failed. As per chatgpt:
On Alpine Linux, the strnlen function is not included in musl by default, as it is part of the GNU C Library (glibc).

And solution given:

#include <string.h>

size_t strnlen(const char *s, size_t maxlen) {
    return (strlen(s) < maxlen) ? strlen(s) : maxlen;
}
casperisfine commented

I wouldn't trust this ChatGPT answer one bit.

First because musl does have strnlen: https://github.com/cloudius-systems/musl/blob/00733dd1cf791d13ff6155509cf139a5f7b2eecb/src/string/strnlen.c

Second because the implementation provided by ChatGPT has a glaring read out of bound issue...

So please provide the error given to you by the compiler, or a way to reproduce the issue (e.g. a Dockerfile).

  • ❤️5
tomhub commented

Sorry, I should have been more specific. So I am running:

$ gem install hiredis-client

On Linux version 6.11.2-arch1-1 (linux@archlinux) (gcc (GCC) 14.2.1 20240910, GNU ld (GNU Binutils) 2.43.0) #1 SMP PREEMPT_DYNAMIC Fri, 04 Oct 2024 21:51:11 +0000

Outcome:

Building native extensions. This could take a while...
ERROR:  Error installing hiredis-client:
        ERROR: Failed to build gem native extension.

    current directory: /home/reduser/.local/share/gem/ruby/3.3.0/gems/hiredis-client-0.22.2/ext/redis_client/hiredis
/usr/bin/ruby extconf.rb
checking for openssl/ssl.h... yes
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb alloc.c
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb net.c
net.c: In function 'redisKeepAlive':
net.c:165:41: warning: unused parameter 'ttl' [-Wunused-parameter]
  165 | int redisKeepAlive(redisContext *c, int ttl, int interval) {
      |                                     ~~~~^~~
net.c:165:50: warning: unused parameter 'interval' [-Wunused-parameter]
  165 | int redisKeepAlive(redisContext *c, int ttl, int interval) {
      |                                              ~~~~^~~~~~~~
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb hiredis.c
In file included from read.h:35,
                 from hiredis.h:36,
                 from hiredis.c:41:
In function 'sprintf',
    inlined from 'redisvFormatCommand' at hiredis.c:498:11:
/usr/include/fortify/stdio.h:300:23: warning: '__orig_snprintf' specified size 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
  300 |                 __r = __orig_snprintf(__s, __b, __f, __builtin_va_arg_pack());
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/fortify/string.h:26,
                 from hiredis.c:35:
/usr/include/fortify/stdio.h: In function 'redisvFormatCommand':
/usr/include/fortify/stdio.h:274:1: note: in a call to function '__orig_snprintf' declared with attribute 'access (read_only, 3)'
  274 | _FORTIFY_FN(snprintf) int snprintf(char *__s, size_t __n,
      | ^~~~~~~~~~~
In function 'sprintf',
    inlined from 'redisvFormatCommand' at hiredis.c:500:16:
/usr/include/fortify/stdio.h:300:23: warning: '__orig_snprintf' specified size 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
  300 |                 __r = __orig_snprintf(__s, __b, __f, __builtin_va_arg_pack());
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/fortify/stdio.h: In function 'redisvFormatCommand':
/usr/include/fortify/stdio.h:274:1: note: in a call to function '__orig_snprintf' declared with attribute 'access (read_only, 3)'
  274 | _FORTIFY_FN(snprintf) int snprintf(char *__s, size_t __n,
      | ^~~~~~~~~~~
In function 'sprintf',
    inlined from 'redisFormatCommandArgv' at hiredis.c:647:11:
/usr/include/fortify/stdio.h:300:23: warning: '__orig_snprintf' specified size 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
  300 |                 __r = __orig_snprintf(__s, __b, __f, __builtin_va_arg_pack());
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/fortify/stdio.h: In function 'redisFormatCommandArgv':
/usr/include/fortify/stdio.h:274:1: note: in a call to function '__orig_snprintf' declared with attribute 'access (read_only, 3)'
  274 | _FORTIFY_FN(snprintf) int snprintf(char *__s, size_t __n,
      | ^~~~~~~~~~~
In function 'sprintf',
    inlined from 'redisFormatCommandArgv' at hiredis.c:650:16:
/usr/include/fortify/stdio.h:300:23: warning: '__orig_snprintf' specified size 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
  300 |                 __r = __orig_snprintf(__s, __b, __f, __builtin_va_arg_pack());
      |                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/fortify/stdio.h: In function 'redisFormatCommandArgv':
/usr/include/fortify/stdio.h:274:1: note: in a call to function '__orig_snprintf' declared with attribute 'access (read_only, 3)'
  274 | _FORTIFY_FN(snprintf) int snprintf(char *__s, size_t __n,
      | ^~~~~~~~~~~
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb sds.c
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb async.c
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb read.c
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb sockcompat.c
ar rcs libhiredis.a alloc.o net.o hiredis.o sds.o async.o read.o sockcompat.o
cc -std=c99 -pedantic -c -O3 -fPIC  -fvisibility=hidden -Wall -W -Wstrict-prototypes -Wwrite-strings -Wno-missing-field-initializers -g -ggdb ssl.c
In file included from ssl.c:38:
/usr/include/fortify/string.h: In function 'strncat':
/usr/include/fortify/string.h:297:36: error: implicit declaration of function 'strnlen'; did you mean 'strlen'? [-Wimplicit-function-declaration]
  297 |                 __fh_size_t __sl = strnlen(__s, __n);
      |                                    ^~~~~~~
      |                                    strlen
make: *** [Makefile:223: ssl.o] Error 1
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include=${opt-dir}/include
        --without-opt-include
        --with-opt-lib=${opt-dir}/lib
        --without-opt-lib
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/bin/$(RUBY_BASE_NAME)
        --with-openssl-dir
        --without-openssl-dir
        --with-openssl-include=${openssl-dir}/include
        --without-openssl-include
        --with-openssl-lib=${openssl-dir}/lib
        --without-openssl-lib
        --with-make-prog
        --without-make-prog
extconf.rb:51:in `block in build_hiredis': Building hiredis failed (RuntimeError)
        from extconf.rb:49:in `chdir'
        from extconf.rb:49:in `build_hiredis'
        from extconf.rb:20:in `configure_extension'
        from extconf.rb:12:in `configure'
        from extconf.rb:108:in `<main>'

To see why this extension failed to compile, please check the mkmf.log which can be found here:

  /home/reduser/.local/share/gem/ruby/3.3.0/extensions/x86_64-linux-musl/3.3.0/hiredis-client-0.22.2/mkmf.log

extconf failed, exit code 1

Gem files will remain installed in /home/reduser/.local/share/gem/ruby/3.3.0/gems/hiredis-client-0.22.2 for inspection.
Results logged to /home/reduser/.local/share/gem/ruby/3.3.0/extensions/x86_64-linux-musl/3.3.0/hiredis-client-0.22.2/gem_make.out

mkmf.log:

have_header: checking for openssl/ssl.h... -------------------- yes

LD_LIBRARY_PATH=.:/usr/lib "gcc -o conftest -I/usr/include/ruby-3.3.0/x86_64-linux-musl -I/usr/include/ruby-3.3.0/ruby/backward -I/usr/include/ruby-3.3.0 -I.  -fno-omit-frame-pointer -fno-strict-aliasing   -O2 -fstack-clash-protection -Wformat -Werror=format-security -fno-plt -g -fno-omit-frame-pointer -fno-strict-aliasing -fPIC conftest.c  -L. -L/usr/lib -L. -Wl,--as-needed,-O1,--sort-common -Wl,-z,pack-relative-relocs -fstack-protector-strong -rdynamic -Wl,-export-dynamic -Wl,--no-as-needed     -lruby  -lm -lpthread -lucontext -lc"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: int main(int argc, char **argv)
4: {
5:   return !!argv[argc];
6: }
/* end */

LD_LIBRARY_PATH=.:/usr/lib "gcc -I/usr/include/ruby-3.3.0/x86_64-linux-musl -I/usr/include/ruby-3.3.0/ruby/backward -I/usr/include/ruby-3.3.0 -I.  -fno-omit-frame-pointer -fno-strict-aliasing   -O2 -fstack-clash-protection -Wformat -Werror=format-security -fno-plt -g -fno-omit-frame-pointer -fno-strict-aliasing -fPIC   -c conftest.c"
checked program was:
/* begin */
1: #include "ruby.h"
2: 
3: #include <openssl/ssl.h>
/* end */

--------------------

Let me know if you need anything further.

casperisfine commented

Thanks for the extra information. Is there any way I can run the same environment? Alpine is generally used for docker images, is that what you are doing? If so could I know which base image you are using?

If not, how did you install alpine? From which .iso etc.

tomhub commented

I have it in lxd container.

$lxc launch images:alpine/edge test1
$lxc exec test1 /bin/ash

I guess you can easily get iso for VM (https://www.alpinelinux.org/downloads/)
Updated: use edge and create test1 container

tomhub commented

upon further investigation, I noticed that for some reasons string.h is taken from /usr/include/fortify/string.h which does not have strnlen() but /usr/include/string.h has strnlen().

casperisfine commented

Alright, I tried to repro using the alpine:edge docker image on aarch64.

$ apk add ruby ruby-dev gcc fortify-headers make git musl-dev openssl-dev
$ bundle
$ bundle exec rake compile

And it worked fine.

I specifically added fortify-headers so that /usr/include/fortify/string.h would be present, but yet it didn't break it.

That said:

In file included from ssl.c:38:
/usr/include/fortify/string.h: In function 'strncat':
/usr/include/fortify/string.h:297:36: error: implicit declaration of function 'strnlen'; did you mean 'strlen'? [-Wimplicit-function-declaration]
  297 |                 __fh_size_t __sl = strnlen(__s, __n);
      |                                    ^~~~~~~
      |

That error doesn't seem to actually be in redis-client. The error is in that fortify header thing, and when I look at the header in the alpine:edge image it's only 192 lines long, so I suspect the version you are using got some recent changes and introduced a bug.