open-simh/simh

Compiling for Windows

Closed this issue · 17 comments

deltecent commented

I have spent days trying to figure out how to compile programs on Windows.

There seems to be an infinite number of possibilities for setting up the build environment: Visual Studio (pick a year), Visual C++ (pick a year), MSYS, MingGW, CMake (multiple versions available from multiple sources), Project files, make, Git (multiple versions from multiple sources), Powershell, to vpkg or not vpkg, and on and on.

I've tried following the instructions in README-CMake, but it fails.

README.CMake jumps around from system to system making it very difficult to follow. It's no wonder Windows users are always looking for binaries.

Coming from apt install <what I need> && git clone open-simh && cd open-simh && make altairz80 && BIN/altairz80 (wow, is that complicated!), Windows software development is unbelievably confusing and I would appreciate if anyone has the secret steps to take to compile SIMH on Windows.

pkoning2 commented

Your best bet may be to pick up "assets" from the automatic builds, that way you're using whatever Github has done to make the windows nightmare work for them. I avoid Windows at all cost for exactly the reasons you mentioned.

Hm, I just remembered something interesting. I recently had to build Wireshark for some Windows colleagues, and found that they provide scripts to do that on Linux. Yes, even the Windows GUI bits. It would be interesting to see if it's straightforward to apply that to open-simh. I think the magic element is "MinGW" but I'd have to dig.

@bscottm do you have helpful suggestions? You're much better at this stuff.

deltecent commented

@pkoning2, I am right there with you regarding Windows. It is, by far, the worst operating system (and I use the term "operating system" loosely) ever created. I think there are a minimum of 3 ways to build Windows executables within the SIMH structure, and none of them are based on any sort of standard development environment, because none exist. What is really quite amusing is all these MingGW type tools seem to try to make Windows be more like Unix, but obviously have failed to do so because you can't fix a operating system that is broken from the ground up with apps.

Someday, hopefully, mankind will evolve and Windows will be eradicated from the planet. But in the mean time, it would be nice if there was a README-Windows.md that showed the "recommended" way to compile SIMH simulators for Windows.

I've copied some of the README-CMake below. My questions and comments are in ().

Why CMake?

[CMake][cmake] is a cross-platform meta-build system that provides similar functionality to GNU autotools within a more integrated and platform-agnostic framework. A sample of the supported build environments include:

  • Unix Makefiles
  • [MinGW Makefiles][mingw64]
  • [Ninja][ninja]
  • macOS XCode
  • MS Visual Studio solutions (2015, 2017, 2019, 2022)
  • IDE build wrappers ([Sublime Text][sublime] and [CodeBlocks][codeblocks])

Making the Windows build process less complex by automatically downloading, building and installing dependency feature libraries, and consistent cross platform support were the initial motivations behind a [CMake][cmake]-based build infrastructure.

(I can really get behind that objective!)

Since then, that motivation expanded to supporting a wider variety of platforms and compiler combinations, streamlining the overall compile process, enhanced IDE integration and SIMH packaging.

(Why? How many development environments does one need? Now it probably doesn't work well with any of them and negates the original objective of "making the Windows build process less complex". This is the law of software. Keep improving it until it no longer works.)

Before You Begin Building...

Toolchains and Tools

Before you begin building the simulators, you need the following:

  • A C/C++ compiler toolchain.

    • GNU C Compiler (gcc): gcc is the default compiler for Linux and
      Unix/Unix-like platforms. It can also be used for [Mingw-w64][mingw64]-based
      builds on Windows.

    • Microsoft Visual C/C++: Visual Studio 2022, 2019, 2017 and 2015 are
      supported. The [appveyor CI/CD][appveyor] pipeline builds using these four
      Microsoft toolchains in Release and Debug configurations.

      • VS 2022: The Community Edition can be downloaded from the
        [Microsoft Visual Studio Community][vs_community] page.

      • VS 2019 Community Edition, VS 2017 and 2015:
        [Microsoft's older Visual Studio Releases page][older_vs_community]
        hosts the installers for these previous releases.

    • CLang/LLVM: clang is the default compiler on MacOS. To use clang on
      Linux/Unix-like operating systems, [CMake][cmake] needs to be invoked
      manually (see here.)

    Success reports for additional platforms not listed are happily accepted along
    with patches to the [CMake][cmake] infrastructure to ensure future support.

(Can't we choose just 1 recommended option for Windows? Do we really need 10+?)

  • [CMake][cmake] version 3.14 or newer.

    • Windows:

      • Visual Studio IDE, Developer command or PowerShell console windows: No
        additional software installation needed. Microsoft provides cmake that
        can be invoked from the command prompt or from within the VS IDE.
        Microsoft has bundled various version of cmake into Visual Studio since
        VS 2015.

      • Otherwise, install cmake using your preferred Windows software package
        manager, such as [Chocolatey][chocolatey] or [Scoop][scoop]. You can also
        [download and install the cmake binary distribution][cmake_downloads]
        directly.

(How about one recommended way so the common person can build an executable from source? Also, do all the combinations of compiler toolchain options and CMake options work together?)

  • The [Git source control system][gitscm].

    • Windows:

      [Git][gitscm] is needed to apply patches to dependency feature
      libraries       when building those libraries.

      • Visual Studio IDE: [Git][gitscm] is available via the Visual Studio IDE.
        If you do all of your simulator development from within the VS IDE, no
        additional software needs to be installed.

      • Visual Studio Developer command or PowerShell console windows: Unlike
        cmake, git's location is not added to PATH or $env:PATH. Use the
        VS IDE for git-related tasks (add, commit, branch, push, pull, etc.)

      • Otherwise, install git using your preferred Windows software package
        manager, such as [Chocolatey][chocolatey] or [Scoop][scoop]. You can also
        [download and install the git client][gitscm_downloads] directly.

(Once again... multiple options under Windows. Which one would be the recommended way based on the unlimited combinations already listed above?)

  • GNU Make:

    • Required for Linux and macOS. Consult your appropriate package manager to
      install make if it is not already installed.

    • MinGW-w64 uses a version of GNU Make named mingw32-make. See the MinGW-W64
      notes below under "Feature Libraries."

  • Ninja:

    [Ninja][ninja] is an optional, but useful/faster parallel build alternative to
    Unix Makefiles and Visual Studio's msbuild.

(For Windows, is GNU Make required for all combinations of previous configurations, some of them, or none of them?)

Windows XP-compatible/Server 2003 binaries

Microsoft has deprecated XP tool support and it WILL eventually disappear in a
future Visual Studio release. Windows XP itself is well beyond End of Lifetime
support, with extended support having ended on April 8, 2014.

(I recommended a separate README-Windows-XP, based on the recommended build environment, for Windows XP as building for the most current version of Windows is complicated enough.)

Feature Libraries

All SIMH features are enabled by default. CMake only disables features
when the underlying support headers and libraries are not detected or the
feature is specifically disabled at cmake configuration time.

Available features are PCAP, TUN/TAP and VDE networking, SDL2 graphics, SDL2
TrueType font rendering, and simulator window snapshot support. Note that some
of these features are platform specific. For example, TUN/TAP and VDE networking
are not available on Windows platforms, whereas cmake tries to detect the
TUN/TAP header file on Linux and macOS, and VDE is an optionally installed
package on Linux and macOS.

Windows: "Legacy" superbuild or vcpkg

The SIMH CMake infrastructure has two distinct feature library dependency
strategies: the "legacy" superbuild and vcpkg. The principal differences
between the two strategies are:

  1. "legacy" can produce Windows XP-compatible executables.

  2. vcpkg has robust compiler support for MS Visual Studio compilers. Using
    GCC or Clang with vcpkg is a work-in-progress.

  3. vcpkg has a larger open source ecosystem and better long term support
    outlook1.

  4. "legacy" installs the minimal dependency features necessary to avoid
    becoming its own "ports" system (which is what vcpkg provides.) For
    example, "legacy" does not install bzip2 as a libpng subdependency,
    which limits the compression methods available to libpng when capturing
    screenshots.

  5. vcpkg installs more subdependencies, potentially increasing
    functionality. Continuing libpng as the example, vcpkg will install
    bzip2 as a subdependency, which adds compression methods to libpng when
    capturing simulator screenshots. vcpkg also installs the Harfbuzz text
    shaper as a Freetype subdependency.

  6. "legacy" compiles the dependency libraries as part of the overall
    compile/build process.

  7. vcpkg compiles and installs dependencies during the CMake configuration
    step, which makes the configuration process longer.

Setup and Usage:

  • "legacy" superbuild

    This is the default dependency feature library build strategy. It will
    download, compile and install the minimal feature libraries necessary to
    support the SIMH simulators: zlib, libpng, pcre (version 1, not
    PCRE2), freetype, SDL2 and SDL_ttf. The CMake configuration process
    generates a superbuild that installs the dependencies under the
    cmake/dependencies subdirectory tree. Once the dependency feature
    libraries finish building successfully, the superbuild invokes CMake to
    reconfigure SIMH to use these newly installed dependencies.

  • vcpkg

    Simply set the VCPKG_ROOT environment variable to use the vcpkg strategy.
    vcpkg operates in [Manifest mode][vcpkg_manifest]; refer to the vcpkg.json
    manifest file.

    The default platform triplets for the Visual Studio compilers are
    x86-windows-static and x64-windows-static, depending on the architecture
    flag passed to CMake.

    The x64-mingw-dynamic triplet is known to work from within a MinGW-w64
    console/terminal window using the GCC compiler.

    If you haven't git-cloned vcpkg, git clone vcpkg somewhere outside of the
    SIMH source tree. For example, you could choose to clone vcpkg in the
    directory above open-simh:

      ```powershell
  PS C:\...\open-simh> pwd
  C:\...\open-simh
  PS C:\...\open-simh> cd ..
  PS C:\...> git clone https://github.com/Microsoft/vcpkg.git
  PS C:\...> cd vcpkg
  PS C:\...\vcpkg> .\vcpkg\bootstrap-vcpkg.bat
  PS C:\...\vcpkg> cd ..\open-simh
  PS C:\...\open-simh>
  ```

    Then set the VCPKG_ROOT environment variable to the vcpkg installaton directory.

(WOW!!!! If that isn't enough to consider removing support for Windows from SIMH, like was done for OS/2, I don't know what is. When I finally got to the point of the opening CMakeList.txt in Visual Studio 2022, it immediately started complaining about vcpkg, which was confusing because as stated above, "legacy" superbuild is the default method. Well, I must have done something wrong with the infinite number of installation options provided. Delete (as much as that's possible on Windows) everything I installed and start over. Same problem. What would be the simplest, recommend option here?)

It goes on and on, mixing PowerShell, MigGW, and multiple other options, then dives into something about having to create a directory before doing anything you've been instructed to do above, and then another 20 pages of more options.

At the end of many, long and frustrating hours (I am having to use Windows), the result is here:

1> CMake generation started for default configuration: 'x64-Debug'.
1> Command line: "C:\WINDOWS\system32\cmd.exe" /c "%SYSTEMROOT%\System32\[chcp.com](http://chcp.com/) 65001 >NUL && "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"  -G "Ninja"  -DCMAKE_BUILD_TYPE:STRING="Debug" -DCMAKE_INSTALL_PREFIX:PATH="C:\Users\Patrick\source\repos\open-simh\out\install\x64-Debug" -DCMAKE_C_COMPILER:FILEPATH="C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe" -DCMAKE_CXX_COMPILER:FILEPATH="C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe"   -DCMAKE_MAKE_PROGRAM="C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.exe" "C:\Users\Patrick\source\repos\open-simh" 2>&1"
1> Working directory: C:\Users\Patrick\source\repos\open-simh\out\build\x64-Debug
1> [CMake] -- CMAKE_MODULE_PATH: C:/Users/Patrick/source/repos/open-simh/cmake;C:/Users/Patrick/source/repos/open-simh/cmake/installer-customizations
1> [CMake] -- SIMH_CMAKE_TOOLCHAIN_FILE is C:/Program Files/Microsoft Visual Studio/2022/Community/VC/vcpkg/scripts/buildsystems/vcpkg.cmake
1> [CMake] -- Executing deferred vcpkg toolchain initialization.
1> [CMake]     .. VCPKG target triplet is x64-windows-static
1> [CMake]     .. VCPKG_CRT_LINKAGE is static
1> [CMake] -- CMAKE_BUILD_TYPE is Debug
1> [CMake] -- Missing dependencies WILL NOT BE BUILT.
1> [CMake] -- macOS unversal binaries NOT WILL BE BUILT.
1> [CMake] -- Not changing CMAKE_C_FLAGS_RELEASE on MSVC
1> [CMake] CMake Error at C:/Program Files/Microsoft Visual Studio/2022/Community/VC/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package):
1> [CMake]   By not providing "FindPThreads4W.cmake" in CMAKE_MODULE_PATH this project
1> [CMake]   has asked CMake to find a package configuration file provided by
1> [CMake]   "PThreads4W", but CMake did not find one.
1> [CMake]
1> [CMake]   Could not find a package configuration file provided by "PThreads4W" with
1> [CMake]   any of the following names:
1> [CMake]
1> [CMake]     PThreads4WConfig.cmake
1> [CMake]     pthreads4w-config.cmake
1> [CMake]
1> [CMake]   Add the installation prefix of "PThreads4W" to CMAKE_PREFIX_PATH or set
1> [CMake]   "PThreads4W_DIR" to a directory containing one of the above files.  If
1> [CMake]   "PThreads4W" provides a separate development package or SDK, be sure it has
1> [CMake]   been installed.
1> [CMake] Call Stack (most recent call first):
1> [CMake]   cmake/pthreads-dep.cmake:18 (find_package)
1> [CMake]   cmake/os-features.cmake:9 (include)
1> [CMake]   CMakeLists.txt:324 (include)
1> [CMake] -- Configuring incomplete, errors occurred!
1> 'C:\WINDOWS\system32\cmd.exe' '/c "%SYSTEMROOT%\System32\[chcp.com](http://chcp.com/) 65001 >NUL && "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"  -G "Ninja"  -DCMAKE_BUILD_TYPE:STRING="Debug" -DCMAKE_INSTALL_PREFIX:PATH="C:\Users\Patrick\source\repos\open-simh\out\install\x64-Debug" -DCMAKE_C_COMPILER:FILEPATH="C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe" -DCMAKE_CXX_COMPILER:FILEPATH="C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe"   -DCMAKE_MAKE_PROGRAM="C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.exe" "C:\Users\Patrick\source\repos\open-simh" 2>&1"' execution failed with error: ''C:\WINDOWS\system32\cmd.exe' '/c "%SYSTEMROOT%\System32\[chcp.com](http://chcp.com/) 65001 >NUL && "C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"  -G "Ninja"  -DCMAKE_BUILD_TYPE:STRING="Debug" -DCMAKE_INSTALL_PREFIX:PATH="C:\Users\Patrick\source\repos\open-simh\out\install\x64-Debug" -DCMAKE_C_COMPILER:FILEPATH="C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe" -DCMAKE_CXX_COMPILER:FILEPATH="C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe"   -DCMAKE_MAKE_PROGRAM="C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.exe" "C:\Users\Patrick\source\repos\open-simh" 2>&1"' returned with exit code: 1'.

If someone would make a "README-Windows.md" file that contains ONE way, the recommended Open-SIMH way, of building executables for Windows for those that have been spoiled by operating systems such as Linux and MacOS, I would be forever grateful.

Footnotes

  1. vcpkg does not support the v141_xp toolkit required to compile Windows
    XP binaries. Windows XP is a target platform that SIMH can hopefully deprecate
    in the future. For the time being, Windows XP is a target platform that is part
    of the CI/CD pipelines and requires the "legacy" superbuild strategy. ↩

deltecent commented

Ah... a Windows build is quite simple:

  1. Install Visual Studio 2022
  2. Clone the repository in Visual Studio
  3. File->Open Project/Solution
  4. Open "Visual Studio Projects/.vcxproj"
  5. Build

Ignore all the CMake stuff if all you want to do is quickly and easily build a Windows executable.

I am not opposed to using CMake, but does it really need to try and support every version and combination of Windows software produced since Windows XP? Who is going to test all of those combinations every time there's a change to SIMH source or the long list of "supported" development tools and combinations?

It seems the current release of Visual Studio and building for Windows 10/11 would be sufficient and provide the best possibility of success.

Rhialto commented
deltecent commented

CMake seems to add a layer of complexity to solve a perceived problem while creating additional problems which requires more software to solve those problems which creates even more problems. The actual problem is Windows, which no amount of software can ever fix. I am a big fan of the KIS principle. Software that attempts to be all things for all people usually doesn't do anything well.

friesga commented
deltecent commented

Once again, you might be frustrated but please keep the discussion respectful.

@friesga You are correct and I apologize for my Windows rant.

I wanted to start testing my PRs under Windows before pushing. Considering contributors must now update a minimum of 3 different build systems when adding a device or simulator, it seemed like a good idea at the time.

Having a choice of compilers and building environments can hardly be seen as a negative thing, it provides freedom of choice and you can pick the one you like most.

We will simply have to disagree on that. In an open source project with many different contributors, there should be a "standard" method for building the project on the various supported platforms and that standard, regardless of what it is, should be documented as the standard. Not 3 (many more than that if you count all the possible Windows combinations) to accommodate every contributor's personal preferences. I happen to prefer indenting with tabs rather than spaces, but we don't have 3 sets of source code to accommodate everyone's formatting preferences.

Just as there is no such thing as a standard text editor.

A person's choice of editor doesn't impact anyone else or the build process. Whether someone uses vi, notepad, or edlin to edit the source code makes no difference, assuming the project's "standard" line termination is maintained. Supporting multiple ways to compile the same project impacts everyone that contributes because they all need to be updated and tested every time someone makes a change to the dependencies. This may also have something to do with GitHub's builds failing these days.

I have a long, not very pleasant, history with Windows, both as a user, developer and IT support. It simply amazes me how popular it still is considering how poorly it works, but you are absolutely correct that it shouldn't creep into my posts for assistance.

bscottm commented

@deltecent: There is the cmake-builder.ps1 script that I specifically made as the "Easy Button" (tm) for Windows binaries. If you really insist on directly invoking cmake from the command line, well, all you need is in the README-CMake.md. There are a bunch of moving parts.

With CMake, there's no real need for the Visual Studio projects subdirectory. Could be deleted because CMake will generate Visual Studio solution for manifest and sundry Visual Studio versions. Will we delete it? Probably not.

Why do I mention the various versions of Visual Studio? Because Pizz says that he can still compile with VS 2008. I erred on the side of caution and tested (successfully at one time) with older versions of VS. Happy to prune that down to VS 2022 and VS 2019, but I'm sure I'll get the usual whine about supporting older versions of VS.

Oh, and let's not forget that the CI/CD pipeline has to produce XP-compatible executables. If there's one thing I'd love to ditch, it's that mess. But I can't because, well, it's something that a VS 2008 compile produces and there are SIMH users still on XP, apparently. (XP is beyond obsolete and VS 2008 no longer installs on Windows 11, so both of those features should be discontinued...)

While the original motivation was to make building on Windows easier, CMake is designed to be cross-platform and can do more than building executables. It makes the CI/CD builds on Github much easier to maintain -- one meta-build system codebase, compiles for Linux, macOS and Windows. Oh, and it supports packaging -- apt, dmg, zip and msi.

I get your frustration with directly invoking the CMake command line. Even I use the cmake-builder.{ps1,sh} scripts when not working from within a CMake-cognizant IDE.

deltecent commented

Let's say I have brand new Windows 10 install. What needs to be installed in order to run cmake-builder.ps1 and end up with altairz80.exe?

bscottm commented

@deltecent: And the four build_*.bat scripts can be deleted. They serve no functional purpose. If we have to trim things down, keep the VS Projects directory for Pizz or use cmake/cmake-builder.ps1.

bscottm commented

Let's say I have brand new Windows 10 install. What needs to be installed in order to run cmake-builder.ps1 and end up with altairz80.exe?

Will answer when I get back from taking my wife to her Sunday hair appt.

markpizz commented

Why do I mention the various versions of Visual Studio? Because Pizz says that he can still compile with VS 2008. I erred on the side of caution and tested (successfully at one time) with older versions of VS. Happy to prune that down to VS 2022 and VS 2019, but I'm sure I'll get the usual whine about supporting older versions of VS.

I never attempted to support more than VS2008 plus whatever was current and stable of the newer VS versions. That still works just fine in simh/simh.

Oh, and let's not forget that the CI/CD pipeline has to produce XP-compatible executables. If there's one thing I'd love to ditch, it's that mess. But I can't because, well, it's something that a VS 2008 compile produces and there are SIMH users still on XP, apparently. (XP is beyond obsolete and VS 2008 no longer installs on Windows 11, so both of those features should be discontinued...)

It is nice when you make up facts. VS 2008 installs just fine on Windows 11. Anyone can follow the directions in "Visual Studio Projects\0ReadMe_Projects.txt".

The VS2008 produced binaries work just fine on every version of Windows since Windows XP.

bscottm commented

@markpizz: VS 2008 install failed miserably for me. YMMV.

bscottm commented

@deltecent: Here's what I've done in the past.

  • Install scoop. I prefer it over Chocolatey because it installs everything locally within your Windows home directory under a scoop subdirectory. Uninstalling scoop is as simple as deleting the scoop subdirectory.
  • Install cmake and git:
> scoop install cmake
> scoop install git
  • Install VS 2022 Community Edition if you haven't already done so.
  • Install vcpkg.
    • "Quick start" instructions are here for Windows. You can stop after running the bootstrap bootstrap-vcpkg.bat file.
    • Set the VCPKG_ROOT environment variable to the directory where you installed vcpkg.
  • You should be able to use cmake-builder.ps1 at this point:
> cmake\cmake-builder.ps1 vs2022 Release
deltecent commented

Set the VCPKG_ROOT environment variable to the directory where you installed vcpkg.

Is the environment variable set in Control Panel->System->Advanced System Settings->Environment Variables or is there a .PS1, .BAT, or other location where this should be set?

bscottm commented

Set the VCPKG_ROOT environment variable to the directory where you installed vcpkg.

Is the environment variable set in Control Panel->System->Advanced System Settings->Environment Variables or is there a .PS1, .BAT, or other location where this should be set?

If you primarily use cmd as your Windows command prompt (*) and want the setting to persist, you would set it in the Control Panel settings, as you noted.

For PowerShell, edit $profile and set it there to persist, e.g., add something similar to the following:

$env:VCPKG_ROOT="<Path to where vpkg was installed>"

(*) Calling cmd a "shell" is much too generous.

deltecent commented

I will give your procedure a go tonight. Thank you for your time and info.