openresty/lua-nginx-module

ngx_http_lua_module.so: undefined symbol: pcre_dfa_exec when dynamic module compiled

Closed this issue · 15 comments

centminmod commented

tried latest compile with Nginx 1.9.12 and 0.10.2 and get the following error only when lua nginx is compiled as dynamic module, if non-dynamic it compiles fine

nginx -t
nginx: [emerg] dlopen() "/usr/local/nginx/modules/ngx_http_lua_module.so" failed (/usr/local/nginx/modules/ngx_http_lua_module.so: undefined symbol: pcre_dfa_exec) in /usr/local/nginx/conf/dynamic-modules.conf:8
nginx: configuration file /usr/local/nginx/conf/nginx.conf test failed

/usr/local/nginx/conf/dynamic-modules.conf contents

load_module "modules/ngx_http_brotli_filter_module.so";
load_module "modules/ngx_http_brotli_static_module.so";
load_module "modules/ngx_http_image_filter_module.so";
load_module "modules/ngx_http_xslt_filter_module.so";
load_module "modules/ngx_http_headers_more_filter_module.so";
load_module "modules/ngx_http_set_misc_module.so";
load_module "modules/ngx_http_echo_module.so";
load_module "modules/ngx_http_lua_module.so";
load_module "modules/ngx_pagespeed.so";
load_module "modules/ngx_http_geoip_module.so";
load_module "modules/ngx_stream_module.so";

./configure --with-ld-opt="-ljemalloc -Wl,-z,relro -Wl,-rpath,/usr/local/lib" --with-cc-opt="-m64 -mtune=native -mfpmath=sse -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2" --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --with-openssl-opt="enable-tlsext" --add-module=../nginx-module-vts --with-libatomic --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.10.33.6-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module=dynamic --with-http_realip_module --add-module=../ngx-fancyindex-0.3.6 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0rc1 --add-dynamic-module=../set-misc-nginx-module-0.30 --add-dynamic-module=../echo-nginx-module-master --add-module=../redis2-nginx-module-0.12 --add-module=../ngx_http_redis-0.3.7 --add-dynamic-module=../lua-nginx-module-0.10.2 --add-module=../nginx_upstream_check_module-0.3.0 --add-module=../openresty-memc-nginx-module-4f6f78f --add-module=../srcache-nginx-module-0.30 --add-dynamic-module=../headers-more-nginx-module-master --with-http_xslt_module=dynamic --with-pcre=../pcre-8.38 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../openssl-1.0.2g
checking for OS
 + Linux 2.6.32-042stab112.15 x86_64
checking for C compiler ... found
 + using GNU C compiler
 + gcc version: 4.8.5 20150623 (Red Hat 4.8.5-4) (GCC) 

ls -lah /usr/local/nginx/modules
total 23M
drwxr-xr-x  2 root root 4.0K Mar 20 15:53 .
drwxr-xr-x 10 root root 4.0K Mar 18 10:33 ..
-rwxr-xr-x  1 root root 210K Mar 20 15:53 ngx_http_brotli_filter_module.so
-rwxr-xr-x  1 root root 109K Mar 20 15:53 ngx_http_brotli_static_module.so
-rwxr-xr-x  1 root root 671K Mar 20 15:53 ngx_http_echo_module.so
-rwxr-xr-x  1 root root 126K Mar 20 15:53 ngx_http_geoip_module.so
-rwxr-xr-x  1 root root 286K Mar 20 15:53 ngx_http_headers_more_filter_module.so
-rwxr-xr-x  1 root root 146K Mar 20 15:53 ngx_http_image_filter_module.so
-rwxr-xr-x  1 root root 3.3M Mar 20 15:53 ngx_http_lua_module.so
-rwxr-xr-x  1 root root 770K Mar 20 15:53 ngx_http_set_misc_module.so
-rwxr-xr-x  1 root root 157K Mar 20 15:53 ngx_http_xslt_filter_module.so
-rwxr-xr-x  1 root root  17M Mar 20 15:53 ngx_pagespeed.so
-rwxr-xr-x  1 root root 574K Mar 20 15:53 ngx_stream_module.so
agentzh commented

@centminmod I guess you need to dynamically link against to PCRE in your nginx build because other dynamic modules like ngx_http_lua_module.so may depend on PCRE.

centminmod commented

thanks @agentzh looks like for now probably easiest to just compile as non-dynamic module

portablesoft commented

Try to complile lua-nginx-module as dynamic module under Nginx 1.11.4 and still get the following error:

Starting nginx: nginx: [emerg] dlopen() "/usr/local/nginx/modules/ngx_http_lua_module.so" failed (/usr/local/nginx/modules/ngx_http_lua_module.so: undefined symbol: pcre_dfa_exec)

and with the following configure:

./configure --prefix=/usr/local/nginx --user=www --group=www --with-http_v2_module --with-http_ssl_module --with-http_gzip_static_module --with-http_realip_module --with-openssl=../openssl/ --with-pcre=../pcre/ --with-pcre-jit --add-dynamic-module=../ngx_devel_kit/ --add-dynamic-module=../srcache-nginx-module/ --add-dynamic-module=../set-misc-nginx-module/ --add-dynamic-module=../lua-nginx-module/ --with-cc-opt="-I /opt/pcrejit/include" --with-ld-opt="-L /opt/pcrejit/lib -Wl,-rpath,/opt/pcrejit/lib"

Then, how to config the pcre for the dynamic lua-nginx-module correctly? Thanks.

agentzh commented

Try adding the --with-ld-opt="-E" to your ./configure command line.

centminmod commented

I managed to get lua nginx module to compile with dynamic module just by adding to --with-ld-opt just -lpcre as i have custom compiled pcre at /usr/local/lib

nginx -V
nginx version: nginx/1.11.4
built by gcc 4.9.2 20150212 (Red Hat 4.9.2-6) (GCC)
built with LibreSSL 2.4.3
TLS SNI support enabled
configure arguments: --with-ld-opt='-lrt -ljemalloc -lpcre -Wl,-z,relro -Wl,-rpath,/usr/local/lib' --with-cc-opt='-m64 -march=native -g -O3 -fstack-protector-strong --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2' --sbin-path=/usr/local/sbin/nginx --conf-path=/usr/local/nginx/conf/nginx.conf --with-http_stub_status_module --with-http_secure_link_module --add-module=../nginx-module-vts --with-libatomic --with-http_gzip_static_module --add-dynamic-module=../ngx_brotli --add-dynamic-module=../ngx_pagespeed-release-1.11.33.4-beta --with-http_sub_module --with-http_addition_module --with-http_image_filter_module=dynamic --with-http_geoip_module --with-stream_geoip_module --with-stream_realip_module --with-threads --with-stream=dynamic --with-stream_ssl_module --with-http_realip_module --add-dynamic-module=../ngx-fancyindex-0.4.0 --add-module=../ngx_cache_purge-2.3 --add-module=../ngx_devel_kit-0.3.0 --add-module=../set-misc-nginx-module-0.31 --add-module=../echo-nginx-module-0.60 --add-module=../redis2-nginx-module-0.13 --add-module=../ngx_http_redis-0.3.7 --add-dynamic-module=../lua-nginx-module-0.10.6 --add-module=../memc-nginx-module-0.17 --add-module=../srcache-nginx-module-0.31 --add-module=../headers-more-nginx-module-0.31 --with-pcre=../pcre-8.39 --with-pcre-jit --with-http_ssl_module --with-http_v2_module --with-openssl=../libressl-2.4.3

seems okay

nginx -t
nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful

dynamic modules include file /usr/local/nginx/conf/dynamic-modules.conf

load_module "modules/ngx_http_brotli_filter_module.so";
load_module "modules/ngx_http_brotli_static_module.so";
load_module "modules/ngx_http_image_filter_module.so";
load_module "modules/ngx_http_lua_module.so";
load_module "modules/ngx_http_fancyindex_module.so";
load_module "modules/ngx_pagespeed.so";
load_module "modules/ngx_stream_module.so";

dynamic module directory at /usr/local/nginx/modules

total 44M
drwxr-xr-x.  2 root root 4.0K Oct  7 17:57 .
drwxr-xr-x. 10 root root 4.0K Sep 23 12:36 ..
-rwxr-xr-x   1 root root 119K Oct  7 17:57 ngx_http_brotli_filter_module.so
-rwxr-xr-x   1 root root 119K Oct  7 17:50 ngx_http_brotli_filter_module.so.old
-rwxr-xr-x   1 root root 106K Oct  7 17:57 ngx_http_brotli_static_module.so
-rwxr-xr-x   1 root root 106K Oct  7 17:50 ngx_http_brotli_static_module.so.old
-rwxr-xr-x   1 root root 150K Oct  7 17:57 ngx_http_fancyindex_module.so
-rwxr-xr-x   1 root root 150K Oct  7 17:50 ngx_http_fancyindex_module.so.old
-rwxr-xr-x   1 root root 149K Oct  7 17:57 ngx_http_image_filter_module.so
-rwxr-xr-x   1 root root 149K Oct  7 17:50 ngx_http_image_filter_module.so.old
-rwxr-xr-x   1 root root 3.5M Oct  7 17:57 ngx_http_lua_module.so
-rwxr-xr-x   1 root root 3.5M Oct  7 17:50 ngx_http_lua_module.so.old
-rwxr-xr-x   1 root root  17M Oct  7 17:57 ngx_pagespeed.so
-rwxr-xr-x   1 root root  17M Oct  7 17:50 ngx_pagespeed.so.old
-rwxr-xr-x   1 root root 1.1M Oct  7 17:57 ngx_stream_module.so
-rwxr-xr-x   1 root root 1.1M Oct  7 17:50 ngx_stream_module.so.old
wangwei1237 commented

add --with-ld-opt='-lpcre -Wl' to the ./configure command.

nginx version: nginx/1.10.2
built by clang 8.0.0 (clang-800.0.38)
built with OpenSSL 1.0.2j 26 Sep 2016
TLS SNI support enabled
configure arguments:
--with-ld-opt='-lpcre -Wl'
--prefix=/Users/wangwei/local/nginx
--with-pcre=/Users/wangwei/Documents/Resources/CodeSource/pcre-8.39
--with-zlib=/Users/wangwei/Documents/Resources/CodeSource/zlib-1.2.8
--with-http_ssl_module
--with-openssl=/Users/wangwei/Documents/Resources/CodeSource/openssl-1.0.2j
--with-http_v2_module
--add-dynamic-module=/Users/wangwei/Documents/Resources/CodeSource/lua-nginx-module

and the nginx seems ok.
nginx: the configuration file /Users/wangwei/local/nginx/conf/nginx.conf syntax is ok
nginx: configuration file /Users/wangwei/local/nginx/conf/nginx.conf test is successful

benbro commented

Is this fixed because the README still recommends using a dynamic module

agentzh commented

@benbro The document does not recommend building this module as a dynamic module. It simply states that this is possible and supported. If you statically link PCRE into your nginx, then you have to specify the ./configure option --with-ld-opt="-Wl,-E" to export those PCRE symbols for dynamically loaded nginx C modules to see them.

agentzh commented

@wangwei1237 Seems like you are linking two different PCRE libraries into the same nginx executable, one is statically linked via --with-pcre=PATH, and the other is dynamically linked via --with-ld-opt='-lpcre -Wl'. And you are asking for troubles with two PCREs in the same image.

agentzh commented

The recommended way is to always dynamically link against PCRE in your nginx instead of using the --with-pcre=PATH option to link statically against PCRE if any of your dynamically loaded nginx C modules do use PCRE, not just this ngx_http_lua_module BTW.

On the other hand, mixing statically linking and dynamic linking often requires explicit symbol exporting using linker options like -Wl,-E.

agentzh commented

To conclude, this is not really an issue in this module.

playaround88 commented

@agentzh sorry, I am new, as u mentioned, how to dynamically link against PCRE in nginx，instead using the --with-pcre=PATH option?

agentzh commented

@playaround88 See how we do that in our packaging scripts here:

https://github.com/openresty/openresty-packaging

Or better, just use our prebuilt Linux packages through our yum/apt repositories:

https://openresty.org/en/linux-packages.html

playaround88 commented

@agentzh thanks, but first links is 404.
I kown how to do! just install pcre to linux standard dynamic lib dir.

./configure --prefix=/usr/local/pcre-8.4 --libdir=/usr/local/lib/pcre --includedir=/usr/local/include/pcre
make
make install

and nginx lua work well!
Thank you anyway.

agentzh commented

@playaround88 Sorry, there was a typo there. Just updated my comment above.