Add Makefile and/or build instructions

Question

Add Makefile and/or build instructions

eduherminio opened this issue 3 years ago · 14 comments

As suggested in #31 (comment), it'd be good to facilitate building Lynx by people not necessarily familiar with .NET environment.

Answer 1 · 2021-09-20T10:53:08.000Z

OK ! Let me add this : i use Linux (Xubuntu 20.04) and luckily the .NET environment is supported, although it's a MicroSoft thingy .. to be complete in your docs, you could document how the .NET env can be installed on Linux (i already managed to do so) .. also you should detect the OS (Windows / Linux / Mac / Android) and compile accordingly .. and i see many chess engine developers are detecting the CPU type of the user and thus create an optimal binary, eg. popcnt, bmi2, avx2, etc. I'm looking forward to it !

Answer 2 · 2021-09-25T10:32:57.000Z

I may eventually follow-up on this topic and provide a way of building it using Docker and/or a build script, but I hope this suffices for now.

Answer 3 · 2021-09-25T12:02:43.000Z

at first my installation failed by using your Makefile, but i managed to compile your v0.4.1 and it runs fine in CuteChess !

the error said i need .NET v6.0 but i have v5 .. so i have to update .NET but it's not fully clear to me how dotnet is installed .. i used some snap installer also .. and some 'dotnet' is in my Path, but i'm not sure how it all works : SDK and Runtime ?

Determining projects to restore...
/usr/share/dotnet/sdk/5.0.401/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(141,5): error NETSDK1045: The current .NET SDK does not support targeting .NET 6.0.  Either target .NET 5.0 or lower, or use a version of the .NET SDK that supports .NET 6.0. [/home/tissatussa/Compiled/Lynx/src/Lynx/Lynx.csproj]
/usr/share/dotnet/sdk/5.0.401/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.TargetFrameworkInference.targets(141,5): error NETSDK1045: The current .NET SDK does not support targeting .NET 6.0.  Either target .NET 5.0 or lower, or use a version of the .NET SDK that supports .NET 6.0. [/home/tissatussa/Compiled/Lynx/src/Lynx.Cli/Lynx.Cli.csproj]
make: *** [Makefile:42: publish] Error 1

in your readme at "Building Lynx" you state :

Requirements: .NET 6 SDK. You can find instructions about how to install it in your preferred OS/Distro either here or here.

as Linux user i find the link to the Micro$oft page annoying .. installing and using their software on my OS seems difficult .. on the page of the other link i find "installation instructions" which leads to ".NET 6 installation instructions for Linux", where "Linux" leads to https://github.com/dotnet/core/blob/main/release-notes/6.0/install-linux.md and i used the code at the bottom "Installing from a binary archive" :

~# curl -Lo dotnet.tar.gz  https://download.visualstudio.microsoft.com/download/pr/5fcb98bb-21e1-47a5-bb8e-bb25f41a3e52/04811d5d05b7e694f040d2a13c1aae4c/dotnet-sdk-6.0.100-rc.1.21463.6-linux-x64.tar.gz
~# mkdir dotnet
~# tar -C dotnet -xf dotnet.tar.gz
~# rm dotnet.tar.gz
~# export DOTNET_ROOT=~/dotnet
~# export PATH=$PATH:~/dotnet
~# dotnet --version
6.0.100-rc.1.21463.6

this does not install anything, but gives a folder/file tree in dir 'dotnet' which shows to be a newer (v6.0) version of the folder tree in my /usr/share/dotnet ! So i just renamed that folder and replaced it by the v6.0 tree and it works : dotnet can now be found and used by your Makefile without error.

Answer 4 · 2021-09-25T12:10:02.000Z

you could point to the compiled binary in artifacts/Lynx/ called Lynx.Cli and appsettings.json is needed .. this is what i discovered and it runs with these 2 files in CuteChess.

Answer 5 · 2021-09-25T12:22:43.000Z

UPDATE: at this point i encounter problems with another chess engine, MinimalChess, which also uses .NET .. but it needs v5 !? How can this issue be solved ? Maybe you're not on Linux .. in the mean time i switch that dotnet folder by renaming for v5 / v6, but can an environment var be set which version / folder to use ? Or maybe i'm mistaken here ..

Answer 6 · 2021-09-25T13:01:05.000Z

Can you please:

Run dotnet --info and dotnet --version to see what you have installed locally
Paste here the issue you're encountering with MinimalChess, both at building and at running it?

dotnet --info should show something similar to:

.NET SDKs installed:
  5.0.301 [/usr/share/dotnet/sdk]
  6.0.100-rc.1.21463.6 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.7 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-rc.1.21452.15 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.7 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-rc.1.21451.13 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

(If probably won't, due to your chosen way of installing v5/v6)

I have setup my Linux as follows:

For .NET 5, since it's a GA/stable version, I used this instructions using APT.
For .NET 6, since it's still in preview version and apparently not available through APT yet, I used this other method using deb/rpm packages. I've run this script twice, first to install .NET 6 preview-5 and recently to install .NET 6 rc-1, and it worked flawlessly

I just tested that I'm able to build and run MinimalChess without any issues with that setup

Answer 7 · 2021-09-25T13:31:29.000Z

A little more in depth:

If you choose to install .NET from a binary archive as you did, you have to manage manually the versions and folders, etc since you're just downloading that, an executable.
If you choose to install .NET with the approach I suggested, using normally APT and that preview script for this specific case, you'll end up with 2 different SDKs (and 2 different runtimes) installed at the same time, with .NET ecosystem managing them.
- The latest SDK version will be chosen by default to build code.
- Normally, dotnet will be able to retro-compatibly compile code (that is, with .NET 6 is likely that you can compile .NET 5 code).
  If that wasn't the case, or you just want to compile it using a non-preview/rc version until .NET 6 stable is out, you can always add a global.json file to control which SDK is used under a certain directory.

dotnet --info shows you the SDK/runtimes installed, and dotnet --version the one that is currently being used (i.e., after taking into account global.json files in the current tree, etc.).

Let me know if you struggle with anything, happy to help you to have a local setup able to build all .NET/.NET Core engines out there (not .NET Framework ones, since that's oldie and Windows-only)

Answer 8 · 2021-09-25T14:13:17.000Z

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.401
 Commit:    4bef5f3dbf

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  20.04
 OS Platform: Linux
 RID:         ubuntu.20.04-x64
 Base Path:   /usr/share/dotnet/sdk/5.0.401/

Host (useful for support):
  Version: 5.0.10
  Commit:  e1825b4928

.NET SDKs installed:
  3.1.413 [/usr/share/dotnet/sdk]
  5.0.401 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.19 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.10 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.19 [/usr/share/dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.10 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

$ dotnet --version
5.0.401

i also supply my current PATH :

$ echo $PATH
/home/tissatussa/.cargo/bin:/home/tissatussa/.nimble/bin:/home/tissatussa/.local/bin:/usr/include/mkl:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/tissatussa/.dotnet/tools

because /usr/share/dotnet is referenced by dotnet --info, i can rename / switch the version of that folder and so i can (temporarily) use .NET v6.0 both SDK and runtimes without installing them .. you also see i had v3.1 once and i recently updated to v5 but v3.1 remained .. normally i don't install any software by hand, i only use common apt commands .. so this .NET versioning thingy annoys me .. why is v6.0 not 'downwards compatible' ? Why can MinimalChess no longer run after a .NET update ? It seems i can use some global.json as you pointed out, and i'm attracted to this solution because it keeps things simple and maintainable .. but it's not clear to me : i expect some path in the .json to distinguish versions !?

here's my log after switching / renaming the /usr/share/dotnet folder to v6.0 :

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.100-rc.1.21463.6
 Commit:    e627d556a1

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  20.04
 OS Platform: Linux
 RID:         ubuntu.20.04-x64
 Base Path:   /usr/share/dotnet/sdk/6.0.100-rc.1.21463.6/

Host (useful for support):
  Version: 6.0.0-rc.1.21451.13
  Commit:  d7619cd4b1

.NET SDKs installed:
  6.0.100-rc.1.21463.6 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.0-rc.1.21452.15 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.0-rc.1.21451.13 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

$ dotnet --version
6.0.100-rc.1.21463.6

Answer 9 · 2021-09-25T14:26:57.000Z

Why can MinimalChess no longer run after a .NET update ?

Without seeing both the compile and the runtime output, I cannot tell you for sure, but my bet is:

For MinimalChess to run, since it's not distributed as a self-contained application, .NET 5 runtime is required to be available.
By installing .NET 6 using the method you followed, when you switch to .NET 6, you can see that there's no .NET 5 runtime detected, therefore MinimalChess won't be able to run (it might be able to compile, though).

I can only suggest you to try the deb/rpm packages method rather than the binary executable one until the stable .NET 6 version is releaed (November) and we can install it via apt commands.

Regarding the global.json, once you have all sdks you want/need available, you can place one of those files wherever you want, and everything under the directory where that lives will be following the rules dictated by it. For instance, you can place one in the clone of MinimalChess so that it uses .NET 5 SDK to build.

But again, that's after reaching a state where dotnet --info shows both .NET 6 and 5 available.

Answer 10 · 2021-09-25T14:44:38.000Z

thanks .. so if i understand right, in November i can install / update my .NET to v6.0 by common apt and a third .NET version of both SDK and runtime will be shown in the --info lists .. then i guess the newest .NET version will be used by default for any app unless a global.json file in its working dir will point to another .NET version ? Please confirm. Your text is clear, but tell me more about downward compatibility as i stated : does it exist anyhow among .NET ?

Answer 11 · 2021-09-25T15:05:18.000Z

thanks .. so if i understand right, in November i can install / update my .NET to v6.0 by common apt and a third .NET version of both SDK and runtime will be shown in the --info lists .. then i guess the newest .NET version will be used by default for any app unless a global.json file in its working dir will point to another .NET version ? Please confirm.

That's right, and the same behavior can be achieved for v6.0 release candidate 1 using the solution I pointed you at, which is currently needed in order to build Lynx.

Your text is clear, but tell me more about downward compatibility as i stated : does it exist anyhow among .NET

".NET runtime updates are compatible within a major version band such as 1.x and 2.x" (...) "In general, you only need the latest SDK and latest patch version of the runtimes required for your application"
"Additionally, newer releases of the .NET SDK generally maintain the ability to build applications that target previous versions of the runtime in a compatible manner"

Source: https://docs.microsoft.com/en-us/dotnet/core/install/remove-runtime-sdk-versions

That is, you can probably build MinimalChess with .NET 6 SDK, but you won't be able to run it with .NET 6 runtime.

Having said that:

In order to run executables* (i.e. MinimalChess release artifacts), you only need to install the runtime. But you need the same major version of .NET runtime that was used to compile the project (2.1, 3.1, 5, 6).
In order to build projects, you need the SDK. When you install the SDK, the runtime is included since normally you build stuff in order to run it afterwards.
In your case, since you want to both build and run projects, I'd recommend you always keep around the latest patch SDK version of each major version: 3.1.x, 5.0.x and 6.0.x (which right now is still 6.0.0-rc1, and rc2 is expected soon). You'll get patch updates for those ones available through apt, after you've installed them using that method.

* Unless they're self-contained executables, such as Lynx release artifacts, in which case you don't need anything.

Answer 12 · 2021-09-25T15:34:33.000Z

great info ! I will read all this again and other users will too !
it does explain a lot about using .NET on Linux, apart from chess engines :-)
thanks

Answer 13 · 2021-11-09T23:36:29.000Z

.NET 6 GA is finally out 🥳

In order to install it, the preview/release candidate version may need/want to be removed first.
An issue where the package manager recognizes the installed preview/RC version as the latest one is already documented here:

When using a package manager to manage your installation of .NET, you may run into a conflict if you've previously installed a preview release.

More details about how to remove it manually can be found here, but if one followed the method recommended above, it's enough to do something like:

sudo apt-get remove dotnet-sdk-6.0 # Removes preview/rc version
sudo apt-get install -y dotnet-sdk-6.0 # Installs GA version
dotnet --info # Verifies that there's no preview/rc version left, and that .NET 6 GA is installed

(tested in Ubuntu 18.04)

Answer 14 · 2021-11-11T12:19:12.000Z

thanks ! Indeed, when using the last 3 command lines you mention, i managed to update / install .NET v6 and compile Lynx v0.6.0 !