Work with official CMake
huangqinjin opened this issue · 11 comments
The README shows that a custom version CMake 844ccd2280d11ada286d0e2547c0fa5ff22bd4db is needed. That version has 4 commit on top of official CMake.
-
Don't set CMAKE_NINJA_CMCLDEPS_RC=1 if cross compiling.
This commit is already upstreamed and is avaliable since CMake 3.23. -
Define _ARM_WINAPI_PARTITION_DESKTOP_SDK_AVAILABLE=1 when building for arm
I believe this is avaliable since Windows 10 October 2018 Update.
huangqinjin/ucrt@10.0.17134.0...10.0.17763.0#diff-783e189238639bd1391011cae2be302cf6da473b146e7dc59b3c7edc306d33f4R242-R244 -
Use /Z7 instead of /Zi, which doesn't work in wine
Since CMake 3.25, we can use the new CMake variable for the same purpose.BTW, @mstorsjo any known reason can share why it is needed?Edit:Ziis supported, see below discussion and PR #65.set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT "$<$<CONFIG:Debug,RelWithDebInfo>:Embedded>")
-
Don't use notify_update with mt.exe
This is still needed (why). @mstorsjo are you interested in a PR that strip outEdit:/notify_updatefrommtwrapper script, such that we can use the official CMake?/notify_updateis supported, see PR #63.
This is a CMakePresets.json expected to work after mt.exe issue fixed
{
"version": 3,
"cmakeMinimumRequired": {
"major": 3,
"minor": 23,
"patch": 0
},
"configurePresets": [
{
"name": "msvc-wine",
"generator": "Ninja",
"cacheVariables": {
"CMAKE_C_COMPILER": "/opt/msvc/bin/x64/cl.exe",
"CMAKE_CXX_COMPILER": "/opt/msvc/bin/x64/cl.exe",
"CMAKE_RC_COMPILER": "/opt/msvc/bin/x64/rc.exe",
"CMAKE_SYSTEM_NAME": "Windows",
"CMAKE_SYSTEM_VERSION": "10",
"CMAKE_CROSSCOMPILING_EMULATOR": "env;WINEDEBUG=-all;wine64",
"CMAKE_BUILD_TYPE": "RelWithDebInfo"
},
"condition": {
"string": "${hostSystemName}",
"type": "notInList",
"list" : [ "Windows" ]
}
}
]
}
- Use /Z7 instead of /Zi, which doesn't work in wine
Since CMake 3.25, we can use the new CMake variable for the same purpose. BTW, @mstorsjo any known reason can share why it is needed?set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT "$<$<CONFIG:Debug,RelWithDebInfo>:Embedded>")
Without this, there's various issues when building with debug info enabled (and even if configuring with -DCMAKE_BUILD_TYPE=Release, some of the initial build tests are done in debug mode, if I remember correctly). In some cases, I remember some people saying that it works for them (installing the package winbind can help), but even with that, I have seen quite mixed success (including cases where cmake hangs waiting for a cl.exe process that is stuck).
So without this change, the whole experience is much more unreliable.
- Don't use notify_update with mt.exe
This is still needed (why). @mstorsjo are you interested in a PR that strip out/notify_updatefrommtwrapper script, such that we can use the official CMake?
Oh, right, if that works, what would be helpful. It would of course be interesting to look into what /notify_update really does and if Wine could be fixed so that this would work, but stripping it out in the wrapper probably would be good enough too.
This is a
CMakePresets.jsonexpected to work aftermt.exeissue fixed.
I'm not familiar with cmake presets json files before - how does one use that? Does the changes to debug info format apply from the very first few invocations of the compiler at the detection stages too?
Does the changes to debug info format apply from the very first few invocations of the compiler at the detection stages too?
Yes, the change is applied at same place with your patch.
A CMakePresets.json is put at the root of project. And we can simply run cmake --preset=msvc-wine instead of many -D options from command line. Many IDEs support CMakePresets.json, e.g. Visual Studio, Visual Studio Code, CLion.
Does the changes to debug info format apply from the very first few invocations of the compiler at the detection stages too?
Yes, the change is applied at same place with your patch.
A
CMakePresets.jsonis put at the root of project. And we can simply runcmake --preset=msvc-wineinstead of many-Doptions from command line. Many IDEs supportCMakePresets.json, e.g. Visual Studio, Visual Studio Code, CLion.
Ok, fair enough.
FYI, I tested this, and tested setting up building with CMake in the Github Actions CI configuration. I didn't add detailed checking of whether the manifest is merged or so, just basic testing to see that building works in general; see the branch https://github.com/mstorsjo/msvc-wine/commits/cmake-unmodified.
Overall it works fine, but there's one gotcha: The CMAKE_MSVC_DEBUG_INFORMATION_FORMAT setting is new, and only CMake projects that set cmake_minimum_required(VERSION 3.25.0) react to it, unless they explicitly add cmake_policy(SET CMP0141 NEW). So that kinda limits the usability - but I guess it's still better than requiring a custom CMake build. (Using the custom CMake build still can of course be useful if building random/older projects.)
FWIW I did test without this setting (and/or with a CMake project that doesn't set the right version or policy), and without it (without having winbind installed) we get an immediate error about fatal error C1902: Program database manager mismatch; please check your installation. I also tested installing winbind in the Github Actions environment, and that does make it work somewhat, but the CMake compiler detection phase takes 10-20 minutes, so it's pretty much broken.
I digged into the PDB hang-up issue. A single wine invocation may spawn several long-living Win32 processes automatically. These processes will inherit at least the stderr handle of current wine command and keep it open (not sure about stdin and stdout). Waiting for the wine command's stderr will hang. A maybe related bug report is here https://bugs.winehq.org/show_bug.cgi?id=15462 .
Launch any exe (e.g. wineboot, cl)
export BINDIR=/opt/msvc/vc/tools/msvc/14.36.32522/bin/Hostx64/x64/
export WINEDEBUG=-all
echo "int main() {}" > main.c
rm -fr wine.log.* *.obj *.pdb
wineserver -k
wineserver -p
wine64 ${BINDIR}cl.exe /c main.c > wine.log.stdout 2> wine.log.stderr
lsof +c0 wine.log.*COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
wineserver 23396 huangqinjin 24w REG 0,45 131 107492576 wine.log.stderr
services.exe 23403 huangqinjin 2w REG 0,45 131 107492576 wine.log.stderr
winedevice.exe 23406 huangqinjin 2w REG 0,45 131 107492576 wine.log.stderr
winedevice.exe 23415 huangqinjin 2w REG 0,45 131 107492576 wine.log.stderr
plugplay.exe 23436 huangqinjin 2w REG 0,45 131 107492576 wine.log.stderr
svchost.exe 23442 huangqinjin 2w REG 0,45 131 107492576 wine.log.stderr
rpcss.exe 23449 huangqinjin 2w REG 0,45 131 107492576 wine.log.stderrLaunch cl with PDB
export BINDIR=/opt/msvc/vc/tools/msvc/14.36.32522/bin/Hostx64/x64/
export WINEDEBUG=-all
echo "int main() {}" > main.c
rm -fr wine.log.* *.obj *.pdb
wineserver -k
wineserver -p
wine64 ${BINDIR}cl.exe /c /Zi /FS main.c > wine.log.stdout 2> wine.log.stderr
lsof +c0 wine.log.*COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
wineserver 23471 huangqinjin 24w REG 0,45 131 107492630 wine.log.stderr
services.exe 23478 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderr
winedevice.exe 23481 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderr
winedevice.exe 23490 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderr
explorer.exe 23499 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderr
explorer.exe 23499 huangqinjin 8w REG 0,45 131 107492630 wine.log.stderr
plugplay.exe 23511 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderr
svchost.exe 23517 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderr
rpcss.exe 23524 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderr
mspdbsrv.exe 23539 huangqinjin 2w REG 0,45 131 107492630 wine.log.stderrCMake hangup when PDB enabled
CMake waits for launched commands until their stdout and stderr EOF.
https://gitlab.kitware.com/cmake/cmake/-/blob/0991023c30ed5b83bcb1446b5bcc9c1eae028835/Source/kwsys/ProcessUNIX.c#L1076
So if long-living Win32 processes spawned by wine keep the compilation command's stdout or stderr open, CMake will hang.
With /Zi /FS options for cl.exe, wine will spawn the extra mspdbsrv.exe. That is why running arbitrary wine commands, e.g. wine64 wineboot, before CMake invocation, still causes CMake hang.
We can simply Launch cl with PDB once before CMake invocation to workaround the issue.
I proposed PR #65 to fix the hang-up issue, and do not need to manually launch any wine command.
After the recent merged patches (thanks!), I now updated my branch https://github.com/mstorsjo/msvc-wine/commits/cmake-unmodified - which now should be more straightforward. I tried to explain the various cases and the reasons for it in the readme (primarily suggesting using winbind and letting the compiler use mspdbsrv.exe, but explaining the various ways one can avoid it too). Does that branch seem reasonable to you?
-
Run wine at least oncein no longer needed.
msvc-wine/.github/workflows/build.yml
Line 13 in a80b5b8
Lines 55 to 58 in a80b5b8
-
It would be good to mention that the actual compile option causing
mspdbsrvget spawned is /FS.
Lines 139 to 140 in a80b5b8
Run wine at least oncein no longer needed.
msvc-wine/.github/workflows/build.yml
Line 13 in a80b5b8
Lines 55 to 58 in a80b5b8
Hmm, right. It's usually still good to do that anyway (depending on the wine setup, it may print a bunch of messages and take a little time to set up the wineprefix the first time), so I'd prefer to keep it in the CI setups. I adjusted the readme to make its role clearer though.
Amended, thanks!
@mstorsjo LGTM!
@mstorsjo LGTM!
Thanks, this is merged now. I guess that pretty much concludes this issue?
Yes, let's close this as completed! 🎉 🎉 🎉