[RFE] Specify a specific flarcar version with your own mirror and nebraska

[ajfriesen]

Current situation

The official and public stable flatcar release has brought some problems and downtime for our services.
Bugs in systemd, dnsmasq, ipv6 shenanigans, etc. pp.

Therefore we want to run our own mirror in order to test and control which stable release we want to deploy. That means a specific version.
I found the https://github.com/flatcar-linux/flatcar-release-mirror but then figured out that the actual update is not happening with the mirror.
It is nebraska.

I have deployed nebraska and I can configure my flatcar to use that as the update server.
I can disable updates for a channel, nice.

Problems I encountered:

I can not see all packages (releases) when I want to edit the package for the stable channel for example.

Is it possible to automatically pick up all packages from our own flatcar mirror?
Controlling the mirror is the easiest way.

1. We can grab a copy of stable with the script from https://github.com/flatcar-linux/flatcar-release-mirror
2. Create a directory for verified releases
3. Symlink only verified releases
4. Let a webserver host that verified releases folder

That way we can easily control which of the packages is actually the one we want to use.

Currently, I just see the option to enter the information via the web interface which is not good for lazy folks like me 😉

Impact

Without specifying the version we have a really lucky short time window in which we can do flatcar updates.
We are forced to remove the automatic updates to prevent damage and impact.
For that updates, we would have to remove the mask for update-engine.
Start update-engine and locksmiths.
Let things happen.
Reboot.
While we hope that in that time no new release is popping up.

New releases had brought downtime for a lot of customers.
We could be able to find in issues in our staging environment. But it is currently hard to do.

Ideal future situation

Possibility to choose our own stable version which we have tested.
I am fine with self-hosting a mirror or Nebraska.
The easiest way would be just using our own mirror, but I imagine that does not work since updates are going through a Nebraska server.

Implementation options

Additional information

I might be missing something. So if you have a better idea, please go ahead. That is just what I gathered from the nebraska repo and a little bit of experimenting.

[pothos]

Hi,
normally you don't even need your own mirror if this is about updates. You can start Nebraska with the syncer flag and optionally with the flag to have Nebraska also host the payloads.
Currently the syncing happens for the Stable channel and will pull in new (Nebraska) packages on it. So what you would do is create a new channel that you manually control by selecting the new version at your own pace.
You can point the Stable group to this new channel which makes it easier because then you don't have to populate the GROUP variable in each Flatcar node's update.conf but only the SERVER to be your Nebraska instance.

[pothos]

The flags are -enable-syncer -host-flatcar-packages. When trying it out you can reduce the syncing intervals with -sync-interval 8m and wait for a bit until the current packages are synced.

[ajfriesen]

My initial problem was updating to a specific flatcar version.

Instead of running Nebraska by myself I just use the flarcar-update script which is even been part of flatcar since release 3139.2.0.

The only issue I had to solve is getting the images/files from the mirror since I only had ipv6. But that is another issue.

I will close this then.