Building the VMR fails on s390x with fsharp looking for crossgen for linux-s390x
omajid opened this issue · 5 comments
omajid commented
Describe the Bug
I am trying to build a recent commit from the VMR (commit 1f2801003dc6785938b6116bd3c7c1717b7c060d). This fails on s390x when building fsharp
/home/omajid/dotnet/repo-projects/Directory.Build.targets(428,5): error MSB3073: The command "/home/omajid/dotnet/src/fsharp/build.sh --pack --publish --ci --configuration Release -bl /p:DotNetBuildRepo=true /p:DotNetBuildOrchestrator=true /p:RestoreConfigFile=/home/omajid/dotnet/artifacts/obj/fsharp/NuGet.config /p:SourceBuiltAssetsDir=/home/omajid/dotnet/artifacts/assets/Release/ /p:SourceBuiltAssetManifestsDir=/home/omajid/dotnet/artifacts/obj/AssetManifests/fsharp/ /p:SourceBuiltSymbolsDir=/home/omajid/dotnet/artifacts/obj/Symbols/fsharp/ /p:ArcadeBuildFromSource=true /p:DotNetBuildSourceOnly=true /p:PreviouslySourceBuiltNupkgCacheDir="/home/omajid/dotnet/prereqs/packages/previously-source-built/" /p:ReferencePackageNupkgCacheDir="/home/omajid/dotnet/prereqs/packages/reference/" /p:TreatWarningsAsErrors=false /p:GenerateResourceUsePreserializedResources=true --sourcebuild --tfm net9.0 /p:SourceBuildBootstrapTfm=net9.0 /p:SourceBuiltShippingPackagesDir=/home/omajid/dotnet/artifacts/packages/Release/Shipping/fsharp/ /p:SourceBuiltNonShippingPackagesDir=/home/omajid/dotnet/artifacts/packages/Release/NonShipping/fsharp/ > /home/omajid/dotnet/artifacts/log/Release/fsharp.log 2>&1" exited with code 1. [/home/omajid/dotnet/repo-projects/fsharp.proj]
##vso[task.setvariable variable=Artifacts;isSecret=false;isOutput=true]/home/omajid/dotnet/src/fsharp/artifacts
##vso[task.setvariable variable=Artifacts.Toolset;isSecret=false;isOutput=true]/home/omajid/dotnet/src/fsharp/artifacts/toolset
##vso[task.setvariable variable=Artifacts.Log;isSecret=false;isOutput=true]/home/omajid/dotnet/src/fsharp/artifacts/log/Release
##vso[task.setvariable variable=Temp;isSecret=false;isOutput=true]/home/omajid/dotnet/src/fsharp/artifacts/tmp/Release
##vso[task.setvariable variable=TMP;isSecret=false;isOutput=true]/home/omajid/dotnet/src/fsharp/artifacts/tmp/Release
/home/omajid/dotnet/src/fsharp//FSharp.sln:
publish /home/omajid/dotnet/src/fsharp//proto.proj /bl:"/home/omajid/dotnet/src/fsharp/artifacts/log/Release/Build.binlog"+.proto.binlog /p:Configuration=Proto /p:ArcadeBuildFromSource=true /p:DotNetBuildRepo=true /p:DotNetBuildOrchestrator=true /p:RestoreConfigFile=/home/omajid/dotnet/artifacts/obj/fsharp/NuGet.config /p:SourceBuiltAssetsDir=/home/omajid/dotnet/artifacts/assets/Release/ /p:SourceBuiltAssetManifestsDir=/home/omajid/dotnet/artifacts/obj/AssetManifests/fsharp/ /p:SourceBuiltSymbolsDir=/home/omajid/dotnet/artifacts/obj/Symbols/fsharp/ /p:ArcadeBuildFromSource=true /p:DotNetBuildSourceOnly=true /p:PreviouslySourceBuiltNupkgCacheDir=/home/omajid/dotnet/prereqs/packages/previously-source-built/ /p:ReferencePackageNupkgCacheDir=/home/omajid/dotnet/prereqs/packages/reference/ /p:TreatWarningsAsErrors=false /p:GenerateResourceUsePreserializedResources=true /p:SourceBuildBootstrapTfm=net9.0 /p:SourceBuiltShippingPackagesDir=/home/omajid/dotnet/artifacts/packages/Release/Shipping/fsharp/ /p:SourceBuiltNonShippingPackagesDir=/home/omajid/dotnet/artifacts/packages/Release/NonShipping/fsharp/
##vso[task.setvariable variable=NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS;isSecret=false;isOutput=true]20
##vso[task.setvariable variable=NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS;isSecret=false;isOutput=true]20
/home/omajid/dotnet/src/fsharp//FSharp.sln:
publish /home/omajid/dotnet/src/fsharp//proto.proj /restore /bl:"/home/omajid/dotnet/src/fsharp/artifacts/log/Debug/Build.binlog"+.proto.binlog /p:Configuration=Proto /p:ArcadeBuildFromSource=false /p:SourceBuildUseMonoRuntime=
Determining projects to restore...
/home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj : error NU1100: Unable to resolve 'Microsoft.NETCore.App.Crossgen2.linux-s390x (= 9.0.0-rc.1.24410.5)' for 'net9.0'. PackageSourceMapping is enabled, the following source(s) were not considered: /home/omajid/dotnet/.dotnet/sdk/9.0.100-rc.1.24413.1/FSharp/library-packs, prebuilt, previously-source-built, reference-packages, source-built-transport-arcade. [/home/omajid/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
##vso[task.logissue type=error;sourcepath=/home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj;linenumber=0;columnnumber=0;code=NU1100;]Unable to resolve 'Microsoft.NETCore.App.Crossgen2.linux-s390x (= 9.0.0-rc.1.24410.5)' for 'net9.0'. PackageSourceMapping is enabled, the following source(s) were not considered: /home/omajid/dotnet/.dotnet/sdk/9.0.100-rc.1.24413.1/FSharp/library-packs, prebuilt, previously-source-built, reference-packages, source-built-transport-arcade.
Restored /home/omajid/dotnet/src/fsharp/src/FSharp.Core/FSharp.Core.fsproj (in 504 ms).
Failed to restore /home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj (in 507 ms).
real 0m3.425s
user 0m2.882s
sys 0m0.130s
/home/omajid/dotnet/src/fsharp/eng/SourceBuild.props(42,5): error MSB3073: The command "./build.sh --bootstrap --skipBuild -bl --tfm net9.0 /p:SourceBuildUseMonoRuntime=" exited with code -1. [/home/omajid/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
##vso[task.logissue type=error;sourcepath=/home/omajid/dotnet/src/fsharp/eng/SourceBuild.props;linenumber=42;columnnumber=5;code=MSB3073;]The command "./build.sh --bootstrap --skipBuild -bl --tfm net9.0 /p:SourceBuildUseMonoRuntime=" exited with code -1.
Build FAILED.
/home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj : error NU1100: Unable to resolve 'Microsoft.NETCore.App.Crossgen2.linux-s390x (= 9.0.0-rc.1.24410.5)' for 'net9.0'. PackageSourceMapping is enabled, the following source(s) were not considered: /home/omajid/dotnet/.dotnet/sdk/9.0.100-rc.1.24413.1/FSharp/library-packs, prebuilt, previously-source-built, reference-packages, source-built-transport-arcade. [/home/omajid/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
/home/omajid/dotnet/src/fsharp/eng/SourceBuild.props(42,5): error MSB3073: The command "./build.sh --bootstrap --skipBuild -bl --tfm net9.0 /p:SourceBuildUseMonoRuntime=" exited with code -1. [/home/omajid/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
0 Warning(s)
2 Error(s)
Steps to Reproduce
First, produce an SDK (I had dotnet-sdk-9.0.100-rc.1.24413.1-linux-s390x.tar.gz) and a previously-source-built-artifacts (I had Private.SourceBuilt.Artifacts.9.0.100-rc.1.24413.1.linux-s390x.tar.gz) by cross-building the VMR on an x64 machine using something like this:
$ git clone https://github.com/dotnet/dotnet
$ cd dotnet
$ export CROSS_ARCH=s390x
$ export CROSSCOMPILE=1
$ export ROOTFS_DIR=/ # or the appropriate value
$ ./prep-source-build.sh
$ ./build.sh --source-only --online --use-mono-runtime /p:TargetArchitecture=s390x /p:PortableBuild=true
Then use the just-built SDK and previously-source-built-artifacts on an s390x machine to build the VMR natively:
#!/bin/bash
set -ueo pipefail
set -x
if [ ! -d dotnet ]; then
git clone https://github.com/dotnet/dotnet
fi
cd dotnet
git clean -xdf
git checkout -- .
git checkout main
git clean -xdf
git checkout -- .
curl -sSLO http://example.com/dotnet-sdk-9.0.100-rc.1.24413.1-linux-s390x.tar.gz
mkdir -p .dotnet
pushd .dotnet
tar xf ../dotnet-sdk*.tar.gz
popd
sdk_version=$(ls -1 .dotnet/sdk/)
sed -i -E 's/"version": .*$/"version": "'$sdk_version'",/' global.json
sed -i -E 's/"dotnet": .*$/"dotnet": "'$sdk_version'"/' global.json
curl -sSLO http://example.com/Private.SourceBuilt.Artifacts.9.0.100-rc.1.24413.1.linux-s390x.tar.gz
pushd prereqs/packages/archive/
mv ../../../Private.SourceBuilt.Artifacts.*.tar.gz .
popd
./prep-source-build.sh
./build.sh --source-only
Other Information
This was working as of Preview 7. It seems to be a regression that landed after that time. It probably affects other mono-using-platforms (eg, ppc64le) as well.
tmds commented
I ran into this issue before and had fixed it with dotnet/fsharp#17300.
I think that worked well (preview7 includes that change).
Something seems to have regressed after that.
@mthalman @ViktorHofer may be you have an idea?
ViktorHofer commented
No idea, unfortunately. I was out for the last few weeks. @omajid do you have a last known good commit so that we can narrow down the search?
tmds commented
@omajid our CI jobs that build vmr main don't show the issue. Can you still reproduce it?
omajid commented
I can still reproduce this with the VMR using the steps above:
$ git rev-parse HEAD
5215fac03da3f4f9ac067e436485dc2401041fbd
<steps from reproducer>
##vso[task.setvariable variable=NUGET_PLUGIN_HANDSHAKE_TIMEOUT_IN_SECONDS;isSecret=false;isOutput=true]20
##vso[task.setvariable variable=NUGET_PLUGIN_REQUEST_TIMEOUT_IN_SECONDS;isSecret=false;isOutput=true]20
/home/omajid/dotnet/src/fsharp//FSharp.sln:
publish /home/omajid/dotnet/src/fsharp//proto.proj /restore /bl:"/home/omajid/dotnet/src/fsharp/artifacts/log/Debug/Build.binlog"+.proto.binlog /p:Configuration=Proto /p:ArcadeBuildFromSource=false /p:SourceB
uildUseMonoRuntime=false
Determining projects to restore...
/home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj : error NU1100: Unable to resolve 'Microsoft.NETCore.App.Crossgen2.linux-s390x (= 9.0.0-rc.1.24414.5)' for 'net9.0'. PackageSourceMapping is enabled,
the following source(s) were not considered: /home/omajid/dotnet/.dotnet/sdk/9.0.100-rc.1.24422.1/FSharp/library-packs, prebuilt, previously-source-built, reference-packages, source-built-transport-arcade. [/hom
e/omajid/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
##vso[task.logissue type=error;sourcepath=/home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj;linenumber=0;columnnumber=0;code=NU1100;]Unable to resolve 'Microsoft.NETCore.App.Crossgen2.linux-s390x (=
9.0.0-rc.1.24414.5)' for 'net9.0'. PackageSourceMapping is enabled, the following source(s) were not considered: /home/omajid/dotnet/.dotnet/sdk/9.0.100-rc.1.24422.1/FSharp/library-packs, prebuilt, previously-sou
rce-built, reference-packages, source-built-transport-arcade.
Restored /home/omajid/dotnet/src/fsharp/src/FSharp.Core/FSharp.Core.fsproj (in 625 ms).
Failed to restore /home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj (in 625 ms).
real 0m3.646s
user 0m3.788s
sys 0m0.183s
/home/omajid/dotnet/src/fsharp/eng/SourceBuild.props(43,5): error MSB3073: The command "./build.sh --bootstrap --skipBuild -bl --tfm net9.0 /p:SourceBuildUseMonoRuntime=false" exited with code -1. [/home/omajid
/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
##vso[task.logissue type=error;sourcepath=/home/omajid/dotnet/src/fsharp/eng/SourceBuild.props;linenumber=43;columnnumber=5;code=MSB3073;]The command "./build.sh --bootstrap --skipBuild -bl --tfm net9.0 /p:Sour
ceBuildUseMonoRuntime=false" exited with code -1.
Build FAILED.
/home/omajid/dotnet/src/fsharp/buildtools/fslex/fslex.fsproj : error NU1100: Unable to resolve 'Microsoft.NETCore.App.Crossgen2.linux-s390x (= 9.0.0-rc.1.24414.5)' for 'net9.0'. PackageSourceMapping is enabled,
the following source(s) were not considered: /home/omajid/dotnet/.dotnet/sdk/9.0.100-rc.1.24422.1/FSharp/library-packs, prebuilt, previously-source-built, reference-packages, source-built-transport-arcade. [/hom
e/omajid/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
/home/omajid/dotnet/src/fsharp/eng/SourceBuild.props(43,5): error MSB3073: The command "./build.sh --bootstrap --skipBuild -bl --tfm net9.0 /p:SourceBuildUseMonoRuntime=false" exited with code -1. [/home/omajid
/dotnet/artifacts/source-built-sdks/Microsoft.DotNet.Arcade.Sdk/tools/Build.proj]
0 Warning(s)
2 Error(s)
Time Elapsed 00:00:04.34
Build failed with exit code 1. Check errors above.
##vso[task.logissue type=error](NETCORE_ENGINEERING_TELEMETRY=Build) Error building solution (exit code '1').
real 0m6.753s
user 0m7.001s
sys 0m0.349s
'fsharp' failed during build.
tmds commented
/p:SourceBuildUseMonoRuntime=false
SourceBuildUseMonoRuntime is incorrect for these architectures.
In our CI, we pass --use-mono-runtime also on the native s390x/ppc64le builds, which is why they don't run into the issue. You can do this as a workaround.