conda-forge/conda-forge.github.io

External runners for Windows

Opened this issue · 3 comments

jaimergp commented

Comes from Quansight/open-gpu-server#31

@wolfv and @baszalmstra asked in today's core call about the possibility of using the recently added Cirun + self-hosted GHA runner infra to add beefy Windows runners for the most demanding feedstocks. I think the idea is to sponsor this for PyTorch packages, in principle.

From the top of my head, this is what needs to happen, roughly:

  1. Choose a Cirun-compatible cloud provider for the Windows runners.
  2. Create VM images for Windows containing the necessary bits (compilers and other tooling not present by default). Make sure the licensing aspects are ok.
  3. Configure the cloud provider accordingly, and register the credentials in the Cirun app for conda-forge.
  4. Add support for registration and access control in the admin-requests + .cirun infra.
  5. Optional. Figure out the legal bits for liabilities and service abuse (e.g. restrict to approved users or maintainers).
carterbox commented
  1. Create VM images for Windows containing the necessary bits (compilers and other tooling not present by default). Make sure the licensing aspects are ok.

I worked on creating Docker containers a few years back (2022). AFAIR, the biggest two obstacles were that (1) Windows containers were large: around 11GB after you got all the stuff in there and (2) the "nano" Windows base image was 64-bit only and conda still required some 32-bit dependencies (that's probably changed?).

Maybe the Docker part isn't relevant to this issue, but there's some good links in that PR about how to install the MSVC tool chain without a GUI.

  • ❤️3
aktech commented

Thanks for creating the issue @jaimergp. I have just received an Azure account from @wolfv (thanks for that 🎉 ). I'll soon start the work and update here.

  • 🚀4
aktech commented

I have created a PR for the same in the access control repo here: conda-forge/.cirun#14

  • 🚀1