Scikit-build-core is failing to link .dll in Windows
JamesMakela-NOAA opened this issue · 3 comments
We've recently ported a pretty complex project with C, C++ and Cython used as Python extensions to scikit-build-core.
We got it all working on Mac an Linux, but on Windows, we're having some problems linking a .dll when compiling a Cython extension. I will try not to be too long-winded, but this requires some detail.
This is related to issue #668.
Basically, in the CMakeLists.txt file, we compile our C sources into a library called gnome. Here is a snippet:
add_library(gnome SHARED ${LIBGNOME_CPP_FILES})
target_link_libraries(gnome PRIVATE NetCDF::NetCDF)
target_compile_definitions(gnome PUBLIC pyGNOME=1)
target_include_directories(gnome PUBLIC lib_gnome)
install(TARGETS gnome DESTINATION gnome/cy_gnome)
Here, we want to compile a bunch of .cpp sources into a dynamic library. This seems to work, and the compiler command seems sane to me. At least I don't immediately see anything wrong.
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64\link.exe
/ERRORREPORT:QUEUE /OUT:"C:\Users\james.makela\AppData\Local\Temp\3\tmpt18bixqf\build\Release\gnome.dll"
/INCREMENTAL:NO /NOLOGO C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\netcdf.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\mfhdf.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\hdf.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\hdf5.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\hdf5_hl.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\z.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\zip.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\blosc.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\zstd.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\bzip2.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\libcurl.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\Library\lib\xml2.lib
kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib
/MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed
/PDB:"C:/Users/james.makela/AppData/Local/Temp/3/tmpt18bixqf/build/Release/gnome.pdb" /SUBSYSTEM:CONSOLE
/TLBID:1 /DYNAMICBASE /NXCOMPAT
/IMPLIB:"C:/Users/james.makela/AppData/Local/Temp/3/tmpt18bixqf/build/Release/gnome.lib" /MACHINE:X64 /machine:x64
/DLL
I think this looks normal for Windows. We create a dynamic library (gnome.dll) and an associated import library (gnome.lib), and the output seems to indicate success. So far, so good.
Then, in CMakeLists.txt, we try to build the Cython extensions. The code is a bit verbose to include here, but here is a snippet:
add_custom_command(
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/gnome/cy_gnome/${ext}.cpp"
DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/gnome/cy_gnome/${ext}.pyx"
VERBATIM
COMMAND "${CYTHON}" --cplus -3
"${CMAKE_CURRENT_SOURCE_DIR}/gnome/cy_gnome/${ext}.pyx"
--output-file
"${CMAKE_CURRENT_BINARY_DIR}/gnome/cy_gnome/${ext}.cpp"
)
python_add_library(
${ext}
MODULE "${CMAKE_CURRENT_BINARY_DIR}/gnome/cy_gnome/${ext}.cpp"
WITH_SOABI
)
target_link_libraries(${ext} PRIVATE gnome Python::NumPy)
if(APPLE)
set_target_properties(${ext} PROPERTIES INSTALL_RPATH "@loader_path/.")
elseif(UNIX)
set_target_properties(${ext} PROPERTIES INSTALL_RPATH "$ORIGIN/.")
endif()
install(TARGETS ${ext} DESTINATION gnome/cy_gnome)
An example of the resulting link command when compiling the extension is:
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.39.33519\bin\HostX64\x64\link.exe
/ERRORREPORT:QUEUE
/OUT:"C:\Users\james.makela\AppData\Local\Temp\3\tmpt18bixqf\build\Release\cy_basic_types.cp311-win_amd64.pyd"
/INCREMENTAL:NO /NOLOGO Release\gnome.lib
C:\Users\james.makela\AppData\Local\miniconda3\envs\pygnome\libs\python311.lib kernel32.lib user32.lib gdi32.lib
winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib
/MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed
/PDB:"C:/Users/james.makela/AppData/Local/Temp/3/tmpt18bixqf/build/Release/cy_basic_types.pdb"
/SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT
/IMPLIB:"C:/Users/james.makela/AppData/Local/Temp/3/tmpt18bixqf/build/Release/cy_basic_types.lib"
/MACHINE:X64 /machine:x64 /DLL cy_basic_types.dir\Release\cy_basic_types.obj
LINK : fatal error LNK1181: cannot open input file 'Release\gnome.lib'
[C:\Users\james.makela\AppData\Local\Temp\3\tmpt18bixqf\build\cy_basic_types.vcxproj]
So here's what I see. I think gnome.lib is the correct file to reference if you want to link external symbols to the dynamic library we previously created, but it isn't referenced by a full path like the other files. Why would that be?
It bears repeating that this is working on MacOSX and Linux.
I wonder if this issueis related to .dll files needing to be in PATH in order to be callable. Maybe it's the aame for searching for dll. I have this suspicion because you are using a custom_command which might nothave the automation that cmake has for its compiler.
You can try adding to the path environment $<TARGET_RUNTIME_DLL_DIRS:tgt> to PATH, where tgt is ${ext}. It can be tricky to pass and escape the list variable, hopefully this link helps with any such pitfalls
@JamesMakela-NOAA I am building a dll and Python bindings to it on Windows using very similar commands to yours, although I am not using Cython, and everything works. My equivalent link output looks like:
C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\MSVC\14.29.30133\bin\HostX86\x64\link.exe /ERRORREPORT:QUEUE /OUT:"H:\Python\oblqtran\build\cp312-cp312-win_amd64\src\python\Release\_oblqtran.cp312-
win_amd64.pyd" /INCREMENTAL:NO /NOLOGO "C:\Program Files\Python312\libs\python312.lib" ..\cpp\Release\oblqtran.lib kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi
32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:"H:/Python/oblqtran/build/cp312-cp312-win_amd64/src/python/Release/_oblqtran.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:"H:/Python/oblqtran/build/cp312-cp312-win_amd64/src/python/Release/_oblqtran.lib" /MACHINE:X64 /machine:x64 /DLL _oblqtran.dir\Release\oblqtranp.obj
and you can see that the lib is given with a relative path and is found correctly. The only obvious difference that I can see between your situation and mine is that I am using Visual Studio 2019 and you are using 2022, I don't know if that could be causing any problems?
One thing that I have found very helpful is to set
[tool.scikit-build]
build-dir = "build/{wheel_tag}"
so that the CMake build stuff ends up in the local directory (or wherever you choose that is readily accessible). I find this enormously speeds up my builds by reusing a lot of the CMake configuration steps, and also makes inspecting the folders to see what files are being generated, and where, much easier. Have you checked that the Release folder and .lib file are definitely where they should be relative to the CMakeLists.txt where the Python library is being compiled? Without seeing the entire of your complex CMakeLists.txt files it is hard to tell where anything could be being copied around prematurely or something like that.
Sorry not to be able to solve your problem, but hopefully it is helpful to know that the basic paradigm is valid on Windows.
Thanks guys for the advice. While not "the" solution, it definitely pointed me in the right direction.
Also this SO question was very similar to my problem and provided some answers as well. Basically here are the changes that I added to get it to build on my Windows environment.
CMakeLists.txt:
+# On Windows, we need to set these environmental variables so that the linker
+# can properly find the .lib file associated with a dynamic library (.dll).
+# And they must be set before any add_library() commands
+if(CMAKE_SYSTEM_NAME STREQUAL "Windows")
+ set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS TRUE)
+ set(BUILD_SHARED_LIBS TRUE)
+endif()
-install(TARGETS gnome DESTINATION gnome/cy_gnome)
+install(TARGETS gnome LIBRARY DESTINATION gnome/cy_gnome)
I am in fact using Visual Studio 22, but this will probably work with 2019 as well.