bioconda/bioconda-utils

Infrastructure update for Channel Cloning/CDN sync

Closed this issue · 2 comments

morremeyer commented

Hey everyone,

we (Anaconda) have been working on upgrades for the channel cloning/CDN sync infrastructure of anaconda.org, which is also used for bioconda.

Time planning

We are planning to switch to the new infrastructure tomorrow, 2024-08-27 at 12:00 UTC.
No downtime is expected for users.

We have already implemented this change for the r and anaconda channels to ensure that our maintenance process and the new infrastructure are working as desired

What's changing?

  • The codebase has been updated to make the cloning more fault-tolerant and reliable
  • The new infrastructure is more tightly integrated into our observability stack, enabling us to detect problems earlier and more granularly (e.g. a specific package that fails to clone instead of just a general cloning error)ö
  • We're adding a last-updated.json file at the channel root, which contains the last update time both as UNIX and ISO8601 timestamps. We are using this directly to monitor the updates end-to-end. Check it out for the anaconda channel if you're interested!

Other notes

This topic was already discussed in the conda-forge meeting on 2024-08-21 where @mbargull was present, and it was decided to bring it up here to raise awareness.

If there's any questions, please ask!

bgruening commented

Thanks @morremeyer for the updates. Good luck with the migration!

  • ❤️1
morremeyer commented

Hey again,

the migration was successful, the clone is now running happily on the new infrastructure.

If there's any questions coming up, please still go ahead and ask them here.
Since this work is done, I'll go and close the issue now.

  • ❤️1