mstorsjo/msvc-wine

Trying to get MSBuild working in a Fedora container

johnsonjh opened this issue · 22 comments

johnsonjh commented

@mstorsjo @ravenexp    [ Moved from #85 ]

Thanks for the work on this in #103 - I'm trying it out in our project, however, and I'm not having luck with it yet. Our build containers are based on Fedora Rawhide rather than Ubuntu, so I'm assuming that we're likely missing a dependency, or some other user error on my part.

We're been successfully using MSVC, Clang-CL, (as well as many other native Windows compilers like Embarcadero and OrangeC) in our CI process: see our latest pipeline and our CI configuration.

The update that adds your latest work is in the 20240125/gccsa branch (our PR #388), specifically the .gitlab-ci/msvc directory. We're now using a git subtree to bring the msvc-wine repo into our tree, and this Dockerfile to build the MSVC container, along with the Dockerfile that builds our base container image.

The behavior I'm seeing trying to run your tests is the following:

$ docker run --rm -it registry.gitlab.com/libsir/libsir/msvc:latest  # enter new container
$ BIN=/opt/msvc/bin/x64 . /opt/msvc/msvcenv-native.sh
$ export PATH="/opt/msvc/bin/x64:${PATH:?}"
$ cd && git clone https://github.com/mstorsjo/msvc-wine && cd msvc-wine/test
$ ./test-msbuild.sh

EXEC: msbuild /p:UseEnv=true /p:Configuration=Debug /p:IntDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-true/Debug/ /p:OutDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-true/Debug/ /home/user/msvc-wine/test/HelloWorld.vcxproj 
Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: Path not found.

EXEC: msbuild /p:UseEnv=false /p:Configuration=Debug /p:IntDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-false/Debug/ /p:OutDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-false/Debug/ /home/user/msvc-wine/test/HelloWorld.vcxproj 
Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: Path not found.

EXEC: msbuild /p:UseEnv=true /p:Configuration=Release /p:IntDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-true/Release/ /p:OutDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-true/Release/ /home/user/msvc-wine/test/HelloWorld.vcxproj 
Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: Path not found.

EXEC: msbuild /p:UseEnv=false /p:Configuration=Release /p:IntDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-false/Release/ /p:OutDir=Z:/tmp/msvc-wine.tmp.vTQs/useenv-false/Release/ /home/user/msvc-wine/test/HelloWorld.vcxproj 
Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: Path not found.

EXIT: test-msbuild.sh   total tests: 4    failed tests: 4   ............. Failed

Trying to just run msbuild gives me the same error:

$ msbuild
Application could not be started, or no application associated with the specified file.
ShellExecuteEx failed: Path not found.

I'd love to figure this out, and I look forward to getting MSBuild working and I switching our CI invocations that manually call CL.EXE over to the MSVS solutions, which will more closely match an end-user's build experience.

The container is using Wine 9.0 and Wine-Mono is also installed. Using WineHQ 9.1-Staging didn't change anything. Everything else from MSVC-Wine (CL.EXE for all platforms, Clang-CL, etc.) is working fine.

I've tried using bash -x to see some debugging traces, but it seems OK to me there, and the error is that MSBuild is running OK in Wine, but it cannot execute something it needs.

Any tips on debugging this before I tear into it in earnest tonight?

Would you prefer I open a new issue?

ravenexp commented

Any tips on debugging this before I tear into it in earnest tonight?

It looks like the main MSBuild binary is not found in your installation.
Can you verify that there is an MSBuild.exe file in <prefix>/MSBuild/Current/Bin/amd64?
Can you run it directly as wine <prefix>/MSBuild/Current/Bin/amd64/MSBuild.exe?

UPD: You say that Wine-Mono is also installed, but I couldn't find anything about instaling Wine-Mono in your Dockerfiles. Did you verify that Wine from the Fedora repos is configured to install Wine-Mono on demand? Because this is not the case with both Debian and Ubuntu.

johnsonjh commented

@ravenexp

Hi, thanks for helping. It looks like that's the case. I only have the following:

[user@65ae7dbba5bc /]$ find /opt | grep -i msbuild
/opt/msvc/bin/arm/msbuild.exe
/opt/msvc/bin/arm/msbuild
/opt/msvc/bin/arm64/msbuild.exe
/opt/msvc/bin/arm64/msbuild
/opt/msvc/bin/x64/msbuild.exe
/opt/msvc/bin/x64/msbuild
/opt/msvc/bin/x86/msbuild.exe
/opt/msvc/bin/x86/msbuild

[user@65ae7dbba5bc /]$ file /opt/msvc/bin/x64/msbuild*
/opt/msvc/bin/x64/msbuild:     Bourne-Again shell script, ASCII text executable
/opt/msvc/bin/x64/msbuild.exe: Bourne-Again shell script, ASCII text executable

Seems it wasn't installed or extracted.

I can quickly rebuild the container and capture the build output.

Anything else I should look for?

ravenexp commented

This is strange. Does the /opt/msvc/MSBuild tree exist at all in your container? If not, it looks like the vsdownload.py has failed to download it for some reason.

The only change I did in the downloader script was this: 30ae8b9

johnsonjh commented

UPD: You say that Wine-Mono is also installed, but I couldn't find anything about instaling Wine-Mono in your Dockerfiles. Did you verify that Wine from the Fedora repos is configured to install Wine-Mono on demand? Because this is not the case with both Debian and Ubuntu.

I have wine installed via the base container, and the following Wine packages are installed in the image:

$ rpm -qa | grep -i '^wine' | sort
wine-9.0-3.fc40.x86_64
wine-alsa-9.0-3.fc40.i686
wine-alsa-9.0-3.fc40.x86_64
wine-arial-fonts-9.0-3.fc40.noarch
wine-cms-9.0-3.fc40.i686
wine-cms-9.0-3.fc40.x86_64
wine-common-9.0-3.fc40.noarch
wine-core-9.0-3.fc40.i686
wine-core-9.0-3.fc40.x86_64
wine-courier-fonts-9.0-3.fc40.noarch
wine-desktop-9.0-3.fc40.noarch
wine-dxvk-1.10.3-5.fc40.i686
wine-dxvk-1.10.3-5.fc40.x86_64
wine-dxvk-d3d9-1.10.3-5.fc40.i686
wine-dxvk-d3d9-1.10.3-5.fc40.x86_64
wine-dxvk-dxgi-1.10.3-5.fc40.i686
wine-dxvk-dxgi-1.10.3-5.fc40.x86_64
wine-filesystem-9.0-3.fc40.noarch
wine-fixedsys-fonts-9.0-3.fc40.noarch
wine-fonts-9.0-3.fc40.noarch
wine-ldap-9.0-3.fc40.i686
wine-ldap-9.0-3.fc40.x86_64
wine-marlett-fonts-9.0-3.fc40.noarch
wine-mono-8.1.0-1.fc40.noarch
wine-ms-sans-serif-fonts-9.0-3.fc40.noarch
wine-opencl-9.0-3.fc40.i686
wine-opencl-9.0-3.fc40.x86_64
wine-pulseaudio-9.0-3.fc40.i686
wine-pulseaudio-9.0-3.fc40.x86_64
wine-small-fonts-9.0-3.fc40.noarch
wine-symbol-fonts-9.0-3.fc40.noarch
wine-system-fonts-9.0-3.fc40.noarch
wine-systemd-9.0-3.fc40.noarch
wine-tahoma-fonts-9.0-3.fc40.noarch
wine-times-new-roman-fonts-9.0-3.fc40.noarch
wine-twain-9.0-3.fc40.i686
wine-twain-9.0-3.fc40.x86_64
wine-webdings-fonts-9.0-3.fc40.noarch
wine-wingdings-fonts-9.0-3.fc40.noarch

There isn't any MSBuild directory, only:

$ ls -l /opt/msvc/
total 28
drwxr-xr-x 7 root root 4096 Jan 30 02:59 'DIA SDK'
drwxr-xr-x 5 root root 4096 Jan 30 02:59  VC
lrwxrwxrwx 1 root root    4 Jan 30 02:59 'Windows Kits' -> kits
drwxr-xr-x 6 root root 4096 Jan 30 02:59  bin
drwxr-xr-x 3 root root 4096 Jan 30 02:59  kits
-rwxr-xr-x 1 root root 2047 Jan 29 19:12  msvcenv-native.sh
-rw-r--r-- 1 root root 4522 Jan 29 19:12  msvctricks.cpp
lrwxrwxrwx 1 root root    2 Jan 30 02:59  vc -> VC
johnsonjh commented

Here is the build log.

I verified I'm up to date with current master:

$ ./update-msvc-wine.sh
From https://github.com/mstorsjo/msvc-wine
 * branch              master     -> FETCH_HEAD
Subtree is already at commit 4175b6a7c7f707b92c56d9405528ede951e63347.
ravenexp commented

You have a stale vsdownload.py file in your repository at: https://github.com/aremmell/libsir/tree/johnsonjh/20240125/gccsa/.gitlab-ci/msvc

Check the file update dates. It looks like git subtree has failed you.

johnsonjh commented

@ravenexp Hrrm ... crazy! Let me wipe away that file, and I'll modify my scripts to use the full paths and get rid of the symlinks! I hate the way git handles symlinks anyway.

johnsonjh commented

Thanks, that was 99% of the problem! I feel like an idiot :)

However, it's not working yet - the MSBuild wrapper isn't working - it shows no output at all. Running the real MSBuild.exe does at least somewhat work.

I think I see the culprit, manually running wine64 /opt/msvc/bin/x64/../msvctricks.exe /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe:

0148:err:module:import_dll Library iconv.dll (which is needed by L"Z:\\usr\\share\\wine\\mono\\wine-mono-8.1.0\\bin\\libmono-2.0-x86_64.dll") not found
0148:err:mscoree:load_mono Could not load Mono into this process
johnsonjh commented

Let me try to install the mingw64-win-iconv and mingw32-win-iconv packages into my base container.

ravenexp commented

However, it's not working yet - the MSBuild wrapper isn't working - it shows no output at all. Running the real MSBuild.exe does at least somewhat work.

I think I see the culprit, manually running wine64 /opt/msvc/bin/x64/../msvctricks.exe /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe:

The msbuild wrapper does not run msvctricks.exe though. It runs MSBuild.exe directly like this: wine64 /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe.

Installing mingw64-win-iconv should not be needed. Wine Mono works without it on both Debian and Ubuntu.

johnsonjh commented

Odd, I ran bash -x $(command -v msbuild) and the last executed line was:

/opt/msvc/bin/x64/wine-msvc.sh /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe

... and running bash -x /opt/msvc/bin/x64/wine-msvc.sh /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe does show it using msvctricks:

[user@f3443f7324f9 /]$ bash -x /opt/msvc/bin/x64/wine-msvc.sh /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe
++ dirname /opt/msvc/bin/x64/wine-msvc.sh
+ MSVCTRICKS_EXE=/opt/msvc/bin/x64/../msvctricks.exe
+ EXE=/opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe
+ shift
+ ARGS=()
+ '[' 0 -gt 0 ']'
+ WINE=wine64
+ export WINEDEBUG=-all
+ WINEDEBUG=-all
+ '[' -n '' ']'
+ WINE_MSVC_STDOUT_SED='s/\r//;'
+ WINE_MSVC_STDERR_SED='s/\r//;'
+ '[' '!' -f /opt/msvc/bin/x64/../msvctricks.exe ']'
+ export WINE_MSVC_STDOUT=/tmp/wine-msvc.stdout.122
+ WINE_MSVC_STDOUT=/tmp/wine-msvc.stdout.122
+ export WINE_MSVC_STDERR=/tmp/wine-msvc.stderr.122
+ WINE_MSVC_STDERR=/tmp/wine-msvc.stderr.122
+ trap cleanup EXIT
+ cleanup
+ wait
+ rm -f /tmp/wine-msvc.stdout.122 /tmp/wine-msvc.stderr.122
+ mkfifo /tmp/wine-msvc.stdout.122 /tmp/wine-msvc.stderr.122
+ pid=126
+ wine64 /opt/msvc/bin/x64/../msvctricks.exe /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe
+ wait 126
+ sed -E 's/\r//;'
+ sed -E 's/\r//;'
+ cleanup
+ wait
+ rm -f /tmp/wine-msvc.stdout.122 /tmp/wine-msvc.stderr.122

It's already almost done rebuilding with the extra iconv packages, so I'll see if that changes anything.

ravenexp commented

Odd, I ran bash -x $(command -v msbuild) and the last executed line was:

/opt/msvc/bin/x64/wine-msvc.sh /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe

... and running bash -x /opt/msvc/bin/x64/wine-msvc.sh /opt/msvc/MSBuild/Current/Bin/amd64/MSBuild.exe does show it using msvctricks:

Not if you set WINE_MSVC_RAW_STDOUT=1:
https://github.com/mstorsjo/msvc-wine/blob/master/wrappers/wine-msvc.sh#L66

And this is exactly what msbuild.exe script does.

johnsonjh commented

Ah, I see! Thanks.

Now, finally ...

This last issue seems to be a bug in Fedora Wine, at least in Rawhide.

I added the mingw iconv packages, entered the running container as root (docker exec -it -u 0...) and executed:

cp -f /usr/i686-w64-mingw32/sys-root/mingw/bin/iconv.dll ~user/.wine/drive_c/windows/system/
cp -f /usr/x86_64-w64-mingw32/sys-root/mingw/bin/iconv.dll ~user/.wine/drive_c/windows/system32/

and, lo and behold, back in my unprivileged container:

MSBuild version 17.8.5+b5265ef37 for .NET Framework
Build started 1/30/2024 4:37:00 AM.

... and all tests pass!

I'll report this upstream, but I'll also update my Dockerfile accordingly with the workaround in the meantime.

Sorry for the hassles and thanks for pointing me in the right direction!

While I have your attention, I do have a related question ... would it be possible to use the Linux native binary of MSBuild from the Microsoft .NET Core SDK (https://dotnet.microsoft.com/download or https://github.com/dotnet/installer/releases) instead of running it via Wine, but still have it calling the wrapped compilers? (I've not tried, but I thought I'd ask first before I start down a dead end.) Perhaps there isn't really any advantage to doing so, even if it's possible?

(I'm rebuilding my container again, and if everything works, I'll close this issue out, thanks again.)

johnsonjh commented

@ravenexp

It works now, but only after adding the iconv.dll files into the CI user's wine prefix.

Since that shouldn't be needed, I'll report that to the upstream packager.

Thanks again!

Edit: I'll rebuild all my containers to make sure I didn't miss anything, and start adding MSBuild-based CI jobs!

ravenexp commented

While I have your attention, I do have a related question ... would it be possible to use the Linux native binary of MSBuild from the Microsoft .NET Core SDK (https://dotnet.microsoft.com/download or https://github.com/dotnet/installer/releases) instead of running it via Wine, but still have it calling the wrapped compilers? (I've not tried, but I thought I'd ask first before I start down a dead end.) Perhaps there isn't really any advantage to doing so, even if it's possible?

I have not tried this either, but I think it's unlikely to work for existing Visual C++ project files for a bunch of reasons:

  • Most C++ .vcxproj files include targets and property files shipped with MSVC (you can find them in <prefix/>MSBuild/Microsoft/), and these files assume a Windows environment ('\' as path separators, access to the registry, etc.)
  • MSBuild target files can actually load DLL plugins shipped with MSVC and WinSDK, which are .NET 4.x assemblies incompatible with .NET Core.
  • MSBuild parses compiler output, and wrapper scripts for cl.exe and link.exe will interfere with this.

dotnet tool can be made work if you write your .vcxproj files from scratch and add all your C++ build targets yourself, but I think you'd be better off using CMake at this point.

johnsonjh commented

Thanks - items 1 and 3 could be overcome, but point 2 seems like it would be a showstopper.