hendriknielaender/zBench

Build command in pipeline on windows

Opened this issue · 0 comments

hendriknielaender commented

Issue with Build Command in CI Pipeline on Windows

Description

We've encountered an issue with the build command in the CI pipeline when running on windows-latest. The term windows-latest in GitHub Actions is a bit ambiguous, as it does not clearly specify the version and edition of Windows being used. After running the same commands on a physical machine with Windows 10 Enterprise, we've observed differences in behavior compared to the CI environment.

Upon further investigation, it appears that windows-latest in GitHub Actions refers to Windows Server editions. Historically, Windows Server editions have exhibited significant differences in behavior and compatibility compared to desktop editions of Windows, such as Windows 10 Enterprise.

Impact

This difference in operating system environments may lead to discrepancies in build results, unexpected failures, or issues that are not reproducible on a developer's local machine if they are using a desktop version of Windows. It's crucial for the reliability and consistency of our CI/CD pipeline to clearly understand and potentially control the Windows environment being used.

Suggested Actions

  1. Clarification of Windows Version: It would be helpful to have explicit documentation or clarification about the version and edition of Windows that windows-latest refers to in the GitHub Actions environment.

  2. Testing on Windows Server: Consider adding a CI pipeline step that explicitly tests on a Windows Server version, in addition to the current windows-latest runner, to cover the differences between server and desktop environments.

  3. Documentation Update: Update the project's documentation to note the potential differences when building on Windows Server vs. Windows desktop editions, so developers are aware of possible discrepancies.

Additional Context

This issue was identified while attempting to build zbench in our CI pipeline. The build command that works as expected on a local Windows 10 Enterprise machine does not produce the same results in the CI environment, leading to confusion and potential build issues.

Edit: Further research confirmed that windows-latest currently uses Windows Server. The differences between Windows Server and desktop Windows versions, such as Windows 10 Enterprise, can affect build processes and application behavior.