varnish/varnish-modules

configure: error: Missing Varnish development files.

Closed this issue · 7 comments

RandyCupic commented

Hi,

I'm trying to install Varnish modules for Varnish 6.0.6 but ./configure keeps failing with configure: error: Missing Varnish development files..

varnishd -V

varnishd (varnish-6.0.6 revision 29a1a8243dbef3d973aec28dc90403188c1dc8e7)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2019 Varnish Software AS

dpkg -l | grep varnish

ii  varnish                                    6.0.6-1~bionic                                                 amd64        state of the art, high-performance web accelerator
ii  varnish-dev                                6.0.6-1~bionic                                                 amd64        development files for Varnish Cache

I tried tag 0.15.0 and branch 6.0-lts which are supposed to be for Varnish 6.0. My procedure for compiling varnish modules:

sudo apt install libtool automake docutils-common
git clone https://github.com/varnish/varnish-modules.git
cd varnish-modules
git checkout 0.15.0
./bootstrap
./configure

./bootstrap

+ mkdir -p m4
+ aclocal -I m4 -I /usr/local/share/aclocal
+ libtoolize --copy --force
libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'build-aux'.
libtoolize: copying file 'build-aux/ltmain.sh'
libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'.
libtoolize: copying file 'm4/libtool.m4'
libtoolize: copying file 'm4/ltoptions.m4'
libtoolize: copying file 'm4/ltsugar.m4'
libtoolize: copying file 'm4/ltversion.m4'
libtoolize: copying file 'm4/lt~obsolete.m4'
+ autoheader
+ automake --add-missing --copy --foreign
configure.ac:14: installing 'build-aux/compile'
configure.ac:14: installing 'build-aux/config.guess'
configure.ac:14: installing 'build-aux/config.sub'
configure.ac:10: installing 'build-aux/install-sh'
configure.ac:10: installing 'build-aux/missing'
src/Makefile.am: installing 'build-aux/depcomp'
parallel-tests: installing 'build-aux/test-driver'
+ autoconf

./configure

checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether make supports nested variables... (cached) yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking how to print strings... printf
checking whether make supports the include directive... yes (GNU style)
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking for shl_load... no
checking for shl_load in -ldld... no
checking for dlopen... no
checking for dlopen in -ldl... yes
checking whether a program can dlopen itself... yes
checking whether a statically linked program can dlopen itself... no
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking for the pthreads library -lpthreads... no
checking whether pthreads work without any flags... no
checking whether pthreads work with -Kthread... no
checking whether pthreads work with -kthread... no
checking for the pthreads library -llthread... no
checking whether pthreads work with -pthread... yes
checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
checking if more special flags are required for pthreads... no
checking whether to build with code coverage support... no
checking for rst2man... rst2man
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for pkg-config... (cached) /usr/bin/pkg-config
checking pkg-config is at least version 0.21... yes
checking for VARNISHAPI... yes
checking for explicit_bzero... yes
checking for Varnish... 6.0.6
checking vsha256.h usability... no
checking vsha256.h presence... no
checking for vsha256.h... no
configure: error: Missing Varnish development files.

And then of course make fails with make: *** No targets specified and no makefile found. Stop.

I tried many solutions that I've found online, I tried to reinstall varnish and varnish-dev, but no help. I'm running out of ideas. The weirdest thing is that it worked before, with the same modules tag and same Varnish version, but I had some other issues so I decided to remove and clean everything and then start from scratch. My colleague also managed to install this on his virtual machine (the same Ubuntu 20.04 version as mine) with the same procedure without issues.

What am I missing?

rezan commented

It looks like it cannot find the dev headers. Can you provide the output of these commands:

find / -type f -name vsha256.h 2> /dev/null
dpkg-query -L varnish-dev
grep -A4 vsha256 config.log

Run the last one in the varnish-modules directory.

RandyCupic commented

Hi @rezan. Here you go:

➜  varnish-modules git:(19b479a) find / -type f -name vsha256.h 2> /dev/null
/usr/include/varnish/vsha256.h

➜  varnish-modules git:(19b479a) dpkg-query -L varnish-dev
/.
/usr
/usr/include
/usr/include/varnish
/usr/include/varnish/cache
/usr/include/varnish/cache/cache.h
/usr/include/varnish/cache/cache_backend.h
/usr/include/varnish/cache/cache_director.h
/usr/include/varnish/cache/cache_filter.h
/usr/include/varnish/cache/cache_varnishd.h
/usr/include/varnish/common
/usr/include/varnish/common/common_param.h
/usr/include/varnish/miniobj.h
/usr/include/varnish/tbl
/usr/include/varnish/tbl/acct_fields_bereq.h
/usr/include/varnish/tbl/acct_fields_req.h
/usr/include/varnish/tbl/backend_poll.h
/usr/include/varnish/tbl/ban_vars.h
/usr/include/varnish/tbl/bo_flags.h
/usr/include/varnish/tbl/boc_state.h
/usr/include/varnish/tbl/body_status.h
/usr/include/varnish/tbl/cli_cmds.h
/usr/include/varnish/tbl/debug_bits.h
/usr/include/varnish/tbl/feature_bits.h
/usr/include/varnish/tbl/h2_error.h
/usr/include/varnish/tbl/h2_frames.h
/usr/include/varnish/tbl/h2_settings.h
/usr/include/varnish/tbl/h2_stream.h
/usr/include/varnish/tbl/htc.h
/usr/include/varnish/tbl/http_headers.h
/usr/include/varnish/tbl/http_response.h
/usr/include/varnish/tbl/locks.h
/usr/include/varnish/tbl/obj_attr.h
/usr/include/varnish/tbl/oc_exp_flags.h
/usr/include/varnish/tbl/oc_flags.h
/usr/include/varnish/tbl/params.h
/usr/include/varnish/tbl/req_body.h
/usr/include/varnish/tbl/req_flags.h
/usr/include/varnish/tbl/sess_attr.h
/usr/include/varnish/tbl/sess_close.h
/usr/include/varnish/tbl/steps.h
/usr/include/varnish/tbl/symbol_kind.h
/usr/include/varnish/tbl/vcc_types.h
/usr/include/varnish/tbl/vcl_returns.h
/usr/include/varnish/tbl/vcl_states.h
/usr/include/varnish/tbl/vhd_fsm.h
/usr/include/varnish/tbl/vhd_fsm_funcs.h
/usr/include/varnish/tbl/vhd_return.h
/usr/include/varnish/tbl/vhp_huffman.h
/usr/include/varnish/tbl/vhp_static.h
/usr/include/varnish/tbl/vrt_stv_var.h
/usr/include/varnish/tbl/vsc_levels.h
/usr/include/varnish/tbl/vsl_tags.h
/usr/include/varnish/tbl/vsl_tags_http.h
/usr/include/varnish/tbl/waiters.h
/usr/include/varnish/vapi
/usr/include/varnish/vapi/vapi_options.h
/usr/include/varnish/vapi/voptget.h
/usr/include/varnish/vapi/vsc.h
/usr/include/varnish/vapi/vsl.h
/usr/include/varnish/vapi/vsl_int.h
/usr/include/varnish/vapi/vsm.h
/usr/include/varnish/vas.h
/usr/include/varnish/vav.h
/usr/include/varnish/vbm.h
/usr/include/varnish/vcl.h
/usr/include/varnish/vcli.h
/usr/include/varnish/vcs.h
/usr/include/varnish/vdef.h
/usr/include/varnish/vmod_abi.h
/usr/include/varnish/vqueue.h
/usr/include/varnish/vre.h
/usr/include/varnish/vrnd.h
/usr/include/varnish/vrt.h
/usr/include/varnish/vrt_obj.h
/usr/include/varnish/vsa.h
/usr/include/varnish/vsb.h
/usr/include/varnish/vsha256.h
/usr/include/varnish/vtcp.h
/usr/include/varnish/vtim.h
/usr/include/varnish/vtree.h
/usr/include/varnish/vut.h
/usr/include/varnish/vut_options.h
/usr/include/varnish/waiter
/usr/include/varnish/waiter/waiter.h
/usr/lib
/usr/lib/pkgconfig
/usr/lib/pkgconfig/varnishapi.pc
/usr/share
/usr/share/aclocal
/usr/share/aclocal/varnish-legacy.m4
/usr/share/aclocal/varnish.m4
/usr/share/doc
/usr/share/doc/varnish-dev
/usr/share/doc/varnish-dev/changelog.Debian.gz
/usr/share/doc/varnish-dev/copyright
/usr/share/varnish
/usr/share/varnish/vmodtool.py
/usr/share/varnish/vsctool.py

➜  varnish-modules git:(19b479a) grep -A4 vsha256 config.log
configure:10496: checking vsha256.h usability
configure:10496: gcc -c -g -O2 -I/usr/local/include/varnish conftest.c >&5
conftest.c:58:10: fatal error: vsha256.h: No such file or directory
   58 | #include <vsha256.h>
      |          ^~~~~~~~~~~
compilation terminated.
configure:10496: $? = 1
configure: failed program was:
--
| #include <vsha256.h>
configure:10496: result: no
configure:10496: checking vsha256.h presence
configure:10496: gcc -E -I/usr/local/include/varnish conftest.c
conftest.c:25:10: fatal error: vsha256.h: No such file or directory
   25 | #include <vsha256.h>
      |          ^~~~~~~~~~~
compilation terminated.
configure:10496: $? = 1
configure: failed program was:
--
| #include <vsha256.h>
configure:10496: result: no
configure:10496: checking for vsha256.h
configure:10496: result: no
configure:10503: error: Missing Varnish development files.

## ---------------- ##
--
ac_cv_header_vsha256_h=no
ac_cv_host=x86_64-pc-linux-gnu
ac_cv_lib_dl_dlopen=yes
ac_cv_lib_dld_shl_load=no
ac_cv_objext=o
rezan commented

Well, the problem is that configure is trying to find the headers in /usr/local/include/varnish and not /usr/include/varnish.

Can you provide the contents of: /usr/lib/pkgconfig/varnishapi.pc

RandyCupic commented

Here:

➜  ~ cat /usr/lib/pkgconfig/varnishapi.pc
prefix=/usr
exec_prefix=${prefix}
bindir=${exec_prefix}/bin
sbindir=${exec_prefix}/sbin
libdir=/usr/lib
sysconfdir=/etc
pkgsysconfdir=${sysconfdir}/varnish
includedir=${prefix}/include
pkgincludedir=${includedir}/varnish
datarootdir=${prefix}/share
datadir=${datarootdir}
pkgdatadir=${datadir}/varnish
vcldir=${pkgdatadir}/vcl
vmoddir=${libdir}/varnish/vmods
vmodtool=${pkgdatadir}/vmodtool.py
vsctool=${pkgdatadir}/vsctool.py

Name: VarnishAPI
Description: Varnish API
Version: 6.0.6
Cflags: -I${includedir}/varnish
Libs: -L${libdir} -lvarnishapi
rezan commented

Well, that looks correct. Did you ever compile and install varnish from source? That could explain the local injection... If so, you would have to remove that installation, ex: make uninstall.

Can I see this command:

find / -type f -name varnishapi.pc 2> /dev/null
RandyCupic commented

@rezan That was it, thank you :) I tried to install Varnish on many ways, including compiling it from source and ended on installing it from repositories (through some adjustments because Ubuntu 20.04 comes with Varnish 6.2 and I needed 6.0 explicitly). The last command that you provided showed me that I had two varnishapi.pc files; one in /usr/lib/pkgconfig and the other one in /usr/local/lib/pkgconfig. Deleting this other one solved the problem.

I actually tried to find all files containing varnish on the filesystem and delete them before but I obviously messed up the regex and somehow overlooked this one.

Thank you so much for your help :) now everything works as it should.

rezan commented

Excellent!