build shared libraries on Windows
svigerske opened this issue · 13 comments
Issue created by migration from Trac.
Original creator: @svigerske
Original creation time: 2018-10-17 15:34:04
Assignee: @svigerske
Version: 0.7
Keywords: autotools-update
Make sure autotools can build proper DLLs on Windows.
Probably restrict third-party codes (like Glpk), where we do not have control over the code, to static libraries, though.
COIN-OR package should make their API explicit by using
__declspec(dllexport/dllimport) (MSVC) and __attribute ((visibility(...))) (GCC).
Then we should pass option win32-dll to LT_INIT (https://www.gnu.org/software/libtool/manual/html_node/LT_005fINIT.html).
Comment by @svigerske created at 2018-10-17 15:34:36
Changing status from new to assigned.
Comment by @svigerske created at 2018-10-17 15:34:36
Version: Version changed from 0.7 to trunk
Comment by @svigerske created at 2018-10-17 20:14:09
Should read chapter 25.4 DLLs with Libtool in Autobook.
Comment by @svigerske created at 2018-10-17 20:20:15
Building a CoinUtils DLL with a static GLPK library (libcoinglpk.lib) is not possible:
*** Warning: This system cannot link to static lib archive /home/stefan/build-msvc/lib/libcoinglpk.la.
*** I have the capability to make that library automatically link in when
*** you link to this library. But I can only do this if you have a
*** shared version of the library, which you do not appear to have.
So doing static lib only for third-party code will not work. Building a shared lib for Glpk could work if not setting the win32-dll flag of LT_INIT, since that is only C code. But it essentially means that using Glpk's configure will not work here, too.
Comment by @svigerske created at 2018-10-18 21:36:43
To have the OsiCommonTest DLL correctly link against the core OSI DLL, these changes where helpful: https://projects.coin-or.org/Osi/changeset/2152/
Comment by @svigerske created at 2018-10-18 21:45:41
Another problem that came up: After building CoinUtils, automake only installs the static CoinUtils lib and a CoinUtils DLL, but it doesn't install the small LIB that would load the DLL.
Thus, when building the Osi DLL, it pulls in the whole CoinUtils LIB, instead of linking against the CoinUtils DLL.
Comment by @svigerske created at 2018-10-19 05:13:30
Further, if disabling static libs, thus building only DLLs, libtool decides to do a cp libCoinUtils-0.dll libCoinUtils.lib...
Comment by @svigerske created at 2018-10-21 22:41:36
Regarding the previous comment. This is caused by the entry
# Names of this library.
library_names='libCoinUtils-0.dll libCoinUtils.lib'
in libCoinUtils.la. libtool --install installs libCoinUtils-0.dll as file and then creates for each following entry a symlink (which is cp on some Windows) to the first file.
The corresponding libtool code is
if test "$#" -gt 0; then
# Delete the old symlinks, and create new ones.
# Try 'ln -sf' first, because the 'ln' binary might depend on
# the symlink we replace! Solaris /bin/ln does not understand -f,
# so we also need to try rm && ln -s.
for linkname
do
test "$linkname" != "$realname" \
&& func_show_eval "(cd $destdir && { $LN_S -f $realname $linkname || { $RM $linkname && $LN_S $realname $linkname; }; })"
done
fi
It seems wrong that libCoinUtils.lib is mentioned in library_names in libCoinUtils.la at the first place, which happens even if static libs are disabled.
library_names is setup early in libtool:
# List of archive names. First name is the real one, the rest are links.
# The last name is the one that the linker finds with -lNAME
library_names_spec="\$libname\`echo \$release | \$SED -e s/[.]/-/g\`\$versuffix\$shared_ext \$libname.lib"
Thus, \$libname.lib at the end of the last line seems to be wrong here.
It is added by share/aclocal/libtool.m4. Line 2530 and following have
case $GCC,$cc_basename in
yes,*)
# gcc
library_names_spec='$libname.dll.a'
[...]
*,cl*)
# Native MSVC
libname_spec='$name'
soname_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext'
library_names_spec='$libname.dll.lib'
[...]
*)
# Assume MSVC wrapper
library_names_spec='$libname`echo $release | $SED -e 's/[[.]]/-/g'`$versuffix$shared_ext $libname.lib'
dynamic_linker='Win32 ld.exe'
So we seem to get into the last case instead of the "Native MSVC" one, which I presume is because we use the compile wrapper around cl.
Comment by @svigerske created at 2018-10-21 23:49:22
This presumption is wrong. The compile wrapper is taken into consideration and cl is correctly detected. This results in the line
checking dynamic linker characteristics... Win32 link.exe
in the output of configure.
However, there is another round of checks coming right after, which seems to reset things to "Win32 ld.exe":
checking whether the /home/stefan/Osi-au/Osi/CoinUtils/compile cl linker (/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... Win32 link.exe
checking how to hardcode library paths into programs... immediate
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking for unavailable option to produce PIC... -DDLL_EXPORT
checking if unavailable PIC flag -DDLL_EXPORT works... no
checking if unavailable static flag works... no
checking if unavailable supports -c -o file.obj... no
checking if unavailable supports -c -o file.obj... (cached) no
checking whether the unavailable linker (/bin/ld) supports shared libraries... yes
checking dynamic linker characteristics... Win32 ld.exe
Maybe this is due to the Fortran compiler, which I do not have available on my setup.
Comment by @svigerske created at 2018-10-27 20:55:46
So, I made some progress on the DLL builds.
I threw out everything Fortran related in configure of CoinUtils. As CoinUtils does not build Fortran code, it should not have to check for a Fortran compiler. The check for Lapack doesn't assume anymore that the Lapack library is written in Fortran and so also does not assume anymore that the name mangling scheme of Fortran would apply to Lapack. Instead, we now check explicitly what name mangling scheme is used for the Lapack lib and set a corresponding C macro.
Also AC_COIN_FINALIZE_FLAGS now introduces a XYZ_EXPORT macro that is set to __declspec(dllimport) if building DLLs. For the build of XYZ itself, we have to redefine this as __declspec(dllexport), though (happens in CoinUtilsConfig.h).
With that, it seems that a proper CoinUtils DLL is build and installed. The CoinUtils tests manage to build, link, and run against it.
Osi is more of the same, but more complex, because it is not just one lib. I've set things up for the main OsiLib and OsiCommonTestsLib. This seems to build and link, also using a CoinUtils DLL.
Someone had the glorious idea a long while ago to declare Osi-dependent constructors for Coin{Pre|Post}SolveMatrix in CoinUtils and implement them in Osi! Such dirty stuff does not work easily for DLLs due to the distinction into dllimport and dllexport. So I removed these declarations from CoinUtils and made them ordinary C++ functions in OsiPresolve.
Comment by @svigerske created at 2018-10-28 15:09:56
Osi tests now also seems to work. Problem was that the main Osi DLL was passing a filepointer (FILE*) to the CoinUtils DLL for reading an .lp file. This does not seem to work (fscanf was crashing). The internet warns on problems when DLLs were build with different MSVC versions, but here it also seems to crash when using the same compilers. In this particular case, it was easy to work around, but this problem could pop up somewhere again.
Comment by @svigerske created at 2018-10-21 23:49:22
This presumption is wrong. The compile wrapper is taken into consideration and cl is correctly detected. This results in the line
checking dynamic linker characteristics... Win32 link.exein the output of configure.
However, there is another round of checks coming right after, which seems to reset things to "Win32 ld.exe":
checking whether the /home/stefan/Osi-au/Osi/CoinUtils/compile cl linker (/bin/ld) supports shared libraries... yes checking dynamic linker characteristics... Win32 link.exe checking how to hardcode library paths into programs... immediate checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... no checking for unavailable option to produce PIC... -DDLL_EXPORT checking if unavailable PIC flag -DDLL_EXPORT works... no checking if unavailable static flag works... no checking if unavailable supports -c -o file.obj... no checking if unavailable supports -c -o file.obj... (cached) no checking whether the unavailable linker (/bin/ld) supports shared libraries... yes checking dynamic linker characteristics... Win32 ld.exeMaybe this is due to the Fortran compiler, which I do not have available on my setup.
Yes. I could get around this by ensuring that F77 is unset instead of set to unavailable if no Fortran compiler is available: 4e1fe54
On the BuildTools side, this seems to be possible now, see also some recent commits that patch up libtool and compile.
F77 is now unset instead of set to "unavailable".