Static compilation?

Question

Static compilation?

BeeeWall opened this issue 7 years ago · 27 comments

I want to be able to compile this statically, so I can include it in an app on Android. However, when I try to do so, a bunch of files give the error

/usr/bin/ld: /usr/lib/arm-linux-gnueabihf/libc.a(libc-start.o): relocation R_ARM_THM_MOVW_ABS_NC against `_dl_starting_up' can not be used when making a shared object; recompile with -fPIC            /usr/lib/arm-linux-gnueabihf/libc.a: error adding symbols: Bad value

Googling the issue tells me that it is trying to compile dynamically, despite me specifically saying to compile statically only (the full configure was

./configure --enable-static --disable-shared CPPFLAGS="-fPIC -fPIE -static" LDFLAGS="-static /usr/lib/arm-linux-gnueabihf/libz.a /usr/lib/arm-linux-gnueabihf/libxml2.a /usr/lib/gcc/arm-linux-gnueabihf/6/libstdc++.a /usr/lib/arm-linux-gnueabihf/libm.a /usr/lib/gcc/arm-linux-gnueabihf/6/libgcc.a /usr/lib/arm-linux-gnueabihf/libc.a /usr/lib/arm-linux-gnueabihf/libdl.a /usr/lib/arm-linux-gnueabihf/libicui18n.a /usr/lib/arm-linux-gnueabihf/libicuuc.a /usr/lib/arm-linux-gnueabihf/libicudata.a /usr/lib/arm-linux-gnueabihf/liblzma.a /usr/lib/arm-linux-gnueabihf/libpthread.a"

). Is something wrong with my configure, does it just not build statically, or what? I am building on an Android tablet in Termux, using Debian For Termux.

Answer 1 · 2017-11-15T16:20:45.000Z

Hmm, I'm not sure. Statically compiling xoreos-tools used to work, but I haven't done it for a while now (and I don't have static versions of all the libraries at hand at the moment).

Could you post your config.log and a full log of a compile in verbose mode (i.e. with make V=1)?

(Also, you might want to try to add --enable-static-boost to the configure call, in case boost.m4 picks up dynamic versions?)

Answer 2 · 2017-11-16T03:17:30.000Z

config.log (hastebin)
make.log (hastebin)

Configure slightly modified since first post:
./configure --enable-static --disable-shared CPPFLAGS=-fPIC -fPIE -static LDFLAGS=-static /usr/lib/arm-linux-gnueabihf/libz.a /usr/lib/arm-linux-gnueabihf/libxml2.a /usr/lib/gcc/arm-linux-gnueabihf/6/libstdc++.a /usr/lib/arm-linux-gnueabihf/libm.a /usr/lib/gcc/arm-linux-gnueabihf/6/libgcc.a /usr/lib/arm-linux-gnueabihf/libc.a /usr/lib/arm-linux-gnueabihf/libdl.a /usr/lib/arm-linux-gnueabihf/libicui18n.a /usr/lib/arm-linux-gnueabihf/libicuuc.a /usr/lib/arm-linux-gnueabihf/libicudata.a /usr/lib/arm-linux-gnueabihf/liblzma.a /usr/lib/arm-linux-gnueabihf/libpthread.a --disable-rpath --with-pic --disable-silent-rules

Answer 3 · 2017-11-16T03:18:18.000Z

Adding --enable-static-boost tells me it is an unrecognized option.

Answer 4 · 2017-11-16T07:28:30.000Z

Errm, you're compiling the xoreos-tools 0.0.4 release package, it seems? Could you compile the current git version instead? There's been quite some changes since then, including the build system (and the dependency on boost wasn't added until after 0.0.4 too).

Answer 5 · 2017-11-16T16:08:26.000Z

How do I build the git version? There's no configure file.

Answer 6 · 2017-11-16T16:19:49.000Z

https://wiki.xoreos.org/index.php?title=Git
https://wiki.xoreos.org/index.php?title=Compiling_xoreos-tools

In short:

To get the sources: git clone git://github.com/xoreos/xoreos-tools.git
Then you can either use the autotools build system (i.) or CMake (ii.)
1. auttools:
1. 1. ./autogen.sh to create the configure scripts
1. 1. ./configure (with options, optionally from a different directory for out-of-tree builds)
1. 1. make
1. CMake:
1. 1. cmake . (with options, optionally from a different directory for out-of-tree builds)
1. 1. make

Basically, to get from the git checkout to the state of the release tarball, you only need to execute the autogen.sh script.

Answer 7 · 2017-11-16T23:01:43.000Z

I'm now trying to static compile with
./configure --enable-static --disable-shared --disable-rpath --with-pic --disable-silent-rules --enable-static-boost \ CPPFLAGS="-fPIC -fPIE" \ LDFLAGS="-static \ /usr/lib/arm-linux-gnueabihf/libz.a \ /usr/lib/arm-linux-gnueabihf/libxml2.a \ /usr/lib/arm-linux-gnueabihf/libboost_filesystem.a \ /usr/lib/arm-linux-gnueabihf/libboost_system.a \ /usr/lib/arm-linux-gnueabihf/libboost_regex.a \ /usr/lib/arm-linux-gnueabihf/libboost_atomic.a \ /usr/lib/arm-linux-gnueabihf/libboost_locale.a \ /usr/lib/gcc/arm-linux-gnueabihf/6/libstdc++.a \ /usr/lib/arm-linux-gnueabihf/libm.a \ /usr/lib/gcc/arm-linux-gnueabihf/6/libgcc.a \ /usr/lib/arm-linux-gnueabihf/libc.a \ /usr/lib/arm-linux-gnueabihf/libdl.a \ /usr/lib/arm-linux-gnueabihf/libicui18n.a \ /usr/lib/arm-linux-gnueabihf/libicuuc.a \ /usr/lib/arm-linux-gnueabihf/libicudata.a \ /usr/lib/arm-linux-gnueabihf/liblzma.a \ /usr/lib/arm-linux-gnueabihf/librt.a \ /usr/lib/arm-linux-gnueabihf/libpthread.a \ /usr/lib/arm-linux-gnueabihf/libboost_chrono.a \ /usr/lib/arm-linux-gnueabihf/libboost_thread.a"
However, I'm getting the error cannot find the flags to link with Boost regex. config.log here.
The error seems to be /usr/bin/ld: cannot find -lboost_regex-d.

Answer 8 · 2017-11-16T23:14:08.000Z

EDIT: shouldn't, not should, in the second sentence.

Could you add -L/usr/lib/arm-linux-gnueabihf/ to the LDFLAGS?

You shouldn't need to add the .a files in LDFLAGS, the configure script should find them automatically.

In the log there, it does find at least one version of boost, but then it doesn't link because the other stuff in the LDFLAGS can't find pthread. That might be an issue with the linking order, pthread.a might need to go earlier or later (I can't ever remember which way).

In either case, it would be best if the configure script just picked them up itself. I.e. take them out, and just add the -L flag to the correct directory. Likewise, --enable-static for configure should already add -static to the LDFLAGS.

Answer 9 · 2017-11-19T02:39:38.000Z

It didn't work.
root@localhost:~/xoreos-tools# ldd src/gff2xml libz.so.1 => /lib/arm-linux-gnueabihf/libz.so.1 (0xb6ea0000) libxml2.so.2 => /usr/lib/arm-linux-gnueabihf/libxml2.so.2 (0xb6d7d000) libboost_filesystem.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_filesystem.so.1.62.0 (0xb6d5c000) libboost_system.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_system.so.1.62.0 (0xb6d49000) libboost_regex.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_regex.so.1.62.0 (0xb6c86000) libboost_atomic.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_atomic.so.1.62.0 (0xb6c74000) libboost_locale.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_locale.so.1.62.0 (0xb6bc7000) libstdc++.so.6 => /usr/lib/arm-linux-gnueabihf/libstdc++.so.6 (0xb6abb000) libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0xb6a43000) libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0xb6a19000) libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0xb692b000) /lib/ld-linux-armhf.so.3 (0x0f000000) libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0xb6918000) libicui18n.so.57 => /usr/lib/arm-linux-gnueabihf/libicui18n.so.57 (0xb676e000) libicuuc.so.57 => /usr/lib/arm-linux-gnueabihf/libicuuc.so.57 (0xb6646000) libicudata.so.57 => /usr/lib/arm-linux-gnueabihf/libicudata.so.57 (0xb4db9000) liblzma.so.5 => /lib/arm-linux-gnueabihf/liblzma.so.5 (0xb4d8f000) librt.so.1 => /lib/arm-linux-gnueabihf/librt.so.1 (0xb4d79000) libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0 (0xb4d53000) libboost_chrono.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_chrono.so.1.62.0 (0xb4d3e000) libboost_thread.so.1.62.0 => /usr/lib/arm-linux-gnueabihf/libboost_thread.so.1.62.0 (0xb4d12000)
It seems to build libraries statically (I saw a few references), but not binaries. That may be intended, however, based on the help line--enable-static[=PKGS] build static libraries [default=yes].
make.log
config.log

Answer 10 · 2017-11-19T02:56:48.000Z

Also, the error seems to be caused by me adding -static to LDFLAGS. I tried adding it to the configure in the previous post and got the same lboost_regex error.

Answer 11 · 2017-11-19T03:01:58.000Z

If I remove the -static from my post two days ago, it gives the same error as in the original issue (for obvious reasons, without -static it is making a shared object), so it is definitely the -static causing the regex error.

Answer 12 · 2017-11-19T18:13:33.000Z

Ah, yes, the issue seems to be that libtool hijacks -static for static libraries. See https://debbugs.gnu.org/cgi/bugreport.cgi?bug=11064 .

A workaround is to use --static (note the two hyphens) in the LDFLAGS.

I've put installed static versions of the build dependency libraries into the prefix /opt/static/, and then the folllowing configure line works for me, to produce statically linked xoreos-tools binaries:

./configure --enable-static-boost --with-boost=/opt/static LDFLAGS="--static -L/opt/static/libs/ -lpthread" XML2_CONFIG="/opt/static/bin/xml2-config"

--enable-static-boost --with-boost=/opt/static is necessary so that boost.m4 finds the correct libraries
--static -L/opt/static/libs/ in the LDFLAGS is necessary for static linking in general
-pthread in the LDFLAGS is necessary for several boost libraries (regex among them)
XML2_CONFIG, so that the correct libxml2 version is found

Answer 13 · 2017-11-19T20:30:52.000Z

Still getting the same error.
config.log

Answer 14 · 2017-11-19T20:50:19.000Z

-with-boost needs to be /home/xoreos-tools-deps, not /home/xoreos-tools-deps/include
It needs to be -pthread, not -lpthread. Subtle difference, but important

I.e. your line should be
./configure --enable-static-boost --with-boost=/home/xoreos-tools-deps LDFLAGS="--static -L/home/xoreos-tools-deps/lib -pthread" XML2_CONFIG=/home/xoreos-tools-deps/bin/xml2-config

Answer 15 · 2017-11-20T00:02:06.000Z

Now I get a Boost locale error, config.log. Also, thanks for your help so far.

Answer 16 · 2017-11-20T00:07:19.000Z

Seems like you're missing ICU now. You might need to put the ICU libs into the LDFLAGS as well.

I didn't catch that, because I compiled the static boost libs without ICU.

Answer 17 · 2017-11-20T00:48:08.000Z

Where do I find those? I have the libs from ldd, do I need all of them? And do I need to specify some sort of Include path for headers (if so, are they the headers from the unicode include subdirectory)?

Answer 18 · 2017-11-23T12:29:06.000Z

You'll need the ones shown by ldd if linking dynamically, yes.

Looking at my Boost include directory, I don't think you'll need the ICU headers (which would be the ones in the unicode directory, yes) when compiling xoreos. In fact, a program using Boost would only need them when using regex/icu.hpp, which xoreos does not. All other uses of ICU in Boost seems to be internal only.

Answer 19 · 2017-11-23T18:05:25.000Z

I already had all libraries from the ldd command. I then tried with all ICU libs, and then including the headers. Still got the same error.

Answer 20 · 2017-11-24T23:40:21.000Z

Just to make sure, you do have static versions of the ICU libs, right?

Otherwise, I'm currently out of guesses, too. I'll need to see for myself, compile a static version of Boost with ICU. Not tonight, though, sorry. :P

Answer 21 · 2017-11-24T23:42:39.000Z

Yes, I do have static ICU libs. Thanks for the help so far.

Answer 22 · 2017-11-25T17:29:07.000Z

Could it be related to the fact that I don't have libibverbs installed? It can never configure, and keeps throwing an error, so it isn't fully installed.

Answer 23 · 2017-11-25T17:33:40.000Z

It really shouldn't...

Answer 24 · 2017-11-25T20:49:12.000Z

Ah, I know why now. The problem is the linking order.

When putting the ICU libraries into LDFLAGS, they're put before boost_locale. But to correctly link, they need to come after boost_locale.

Unfortunately, there's currently no way to do that with xoreos' build system. As a workaround, you might want to recompile your static boost libraries without ICU? Not the best solution, I know.

There's two potential future fixes for that, both in our copy of boost.m4:

Have the Boost.Locale check (BOOST_LOCALE) also try to link with the ICU libraries
Extend BOOST_FIND_LIBS to see if BOOST_$1_LDFLAGS (which for Boost.Locale expands to BOOST_LOCALE_LDFLAGS) is already defined, and if so, use that instead of trying to find the flags itself

Both solutions can work in tandem, too.

Answer 25 · 2017-11-25T22:03:04.000Z

I'm using prebuild binaries from the package libboost-dev-all (or something along those lines). Currently, my chroot got messed up and I'm redoing it, but after that I'll try listing the libs in LDFLAGS again, now that we've (well, mostly you've) figured out the error I had with it. Maybe that will solve the incorrect order? Or will it just append my LDFLAGS to the incorrect one?

Answer 26 · 2017-11-25T22:39:09.000Z

No, that won't help. The LDFLAGS are always pasted before the library it tries to link.

See https://github.com/xoreos/xoreos-tools/blob/master/m4/boost.m4#L484 . The original LDFLAGS value (which is temporary saved in $boost_save_LDFLAGS) is put before the extra value found by boost.m4. Likewise, the original value of LIBS is put before the boost library it tries.

And the order of LDFLAGS and LIBS among each other is thus that LDFLAGS comes before LIBS (which is done by _BOOST_AC_LINK_IFELSE through the use of autoconf's ac_link, so not something that can or should be changed).

As far as I can see, there isn't really any way to do what is necessary for properply linking the ICU libs with boost_locale here. One (or both) of my solutions above is necessary for that.

Answer 27 · 2017-11-27T03:23:03.000Z

If I took object files for the binaries (by configuring and making without the --static LDFLAG), would I then be able to manually run linker with the libraries, or would I have to search through configure for a bunch of flags as well?