Consider building and shipping toolbx images using Quay.io

Question

Consider building and shipping toolbx images using Quay.io

travier opened this issue 3 years ago · 68 comments

Current status

We've created a set of community supported toolbox images for other distributions. The containerfiles are available in https://github.com/toolbx-images/images and the images themselves are hosted on Quay.io: https://quay.io/organization/toolbx-images

We now have Arch Linux, CentOS Stream 9, Debian 10, 11, testing, unstable, RHEL 8.2, 8.4, 8.6, 9.0 and Ubuntu 16.04, 18.04, 20.04, 22.04 images available.

Instructions on how to use those images are in https://github.com/toolbx-images/images.

Full official support in toolbx (i.e. via the --distro flag) is still tracked here.

Original issue

Is your feature request related to a problem? Please describe.

Toolbx currently only offer images for Fedora.

Describe the solution you'd like

Consider using Quay.io to ship toolbx images (excluding Fedora images which are currently available from the Fedora infra).

This would enable the community to more easily contribute images for other distributions.

The following PRs are stalled on that:

Community contributed toolbx images could live in a separated repository and automatic rebuilds for each commit could be set up on Quay. Multiple container images can be built on Quay from a single GitHub repo.

Describe alternatives you've considered

Having each distribution provide its own toolbx images does not scale. It's easier for the community to host that in a central place and build everything automatically on Quay.

Answer 1 · 2022-03-10T12:04:58.000Z

I can help with that if you're interested.

Answer 2 · 2022-05-02T12:12:07.000Z

I offered to help with maintain the Arch Linux image in December and haven't really heard back nor seen any movement. I really wish we could see some more movement on this as having support for other distro images is the main feature missing at the moment.

Answer 3 · 2022-05-09T17:05:23.000Z

Quay is currently having issue building Fedora 35+ based images so maybe we could directly build those from GitHub Actions from this or another repo in this org.

Answer 4 · 2022-05-20T20:34:25.000Z

I am totally on board (as author of #483).
We should look into having a community repository available in toolbox itself as well (integration for image pulling, etc).

Answer 5 · 2022-05-23T15:38:54.000Z

Building Fedora 35+ images is now fixed on Quay.io.

It would be great to move forward on this one. Creating another repo in this org would make things easier. I'd be available to do the Quay setup and reviews. @debarshiray @HarryMichal could you do help setup that up?

Answer 6 · 2022-05-24T12:34:58.000Z

@travier Could we make a toolbox-community organization maybe? It might seem a bit hostile(?) but we can move it to the containers project when there is more activity on this front.

Answer 7 · 2022-05-24T21:41:17.000Z

I assume we are talking about the default images for a distro that's picked to match the host or can be selected with --distro, right?

I have been meaning to wrap up the addition of Arch and Ubuntu for a long while now, but I got too stuck getting Toolbx enabled on RHEL, and taking care of the Flatpak stack in Fedora and RHEL. My apologies about that.

My opinion is that we are happy to support a distribution out-of-the-box as long as somebody from that community is happy to maintain the image, and volunteers to push through any fixes to the underlying distribution when necessary.

In concrete terms, supporting a distribution requires:

Adding an entry for it to supportedDistros in src/pkg/utils/utils.go.
It's up to the distro maintainer where the image is hosted. It can be on Docker Hub, Quay, something else. As the upstream Toolbx maintainer, I don't want to get into that.
Wherever the image resides, we carry a copy of the Containerfile in toolbox.git/images for reference. This makes it's easy to try out changes locally when hacking. Things can be quite bureaucratic in some cases. eg., RHEL. Or, if a contributor makes changes to the Toolbx code that requires changes to the images, then it's a lot easier to play with the changes if all the Containerfiles are easily accessible from one place.

Since I am also a Fedora contributor, and Toolbx was started for the needs of Fedora Silverblue, I and a few other Fedora contributors (@juhp and @HarryMichal) take care of it. It's hosted on Fedora infrastructure, because:

I want to push for a future where Toolbx becomes part of the Fedora release criteria so that Fedora is never released with broken Toolbx. That's not possible if it depends on non-Fedora infra'.
Fedora has it's own infra' for OCI images.

Similarly, the hosting of the Arch and Ubuntu images is entirely up to @Foxboron and @Jmennius :)

Anyway, I have created a toolbx organization on Quay, in case that's where you want to host them. Note that the toolbox organization is owned by somebody else. We had tried to track down the owners in the past, but failed.

@travier, I sent you an invite to join.

@Foxboron, @Jmennius do you already have some hosting in mind for the Arch and Ubuntu images? If you want to use our organization on Quay, then I might need your Quay usernames to give you write access.

Answer 8 · 2022-05-24T21:58:40.000Z

I don't mind hosting it on Quay. Arch also has a OCI infrastructure for building OCI images but we rely on docker to distribute them currently.

The username on quay is foxboron.

Answer 9 · 2022-05-24T22:07:18.000Z

The username on quay is foxboron.

Okay! I sent you an invitation.

Answer 10 · 2022-05-25T10:49:08.000Z

Thanks for the setup!

I suggested using Quay because we can use build triggers that will rebuild images each time we merge something. This however requires that the quay repo admin grants access to quay for those repos and setup those triggers.

Having a separated repo for Containerfiles would thus be better as it would only trigger rebuilds for commits that directly change those Containerfiles and not other changes in toolbx.

Answer 11 · 2022-05-25T10:51:08.000Z

Maybe we can start building those images without adding them as "distros" in toolbox to get them available first and then we can figure out how to make them official.

Answer 12 · 2022-05-25T10:56:41.000Z

Having a separated repo for Containerfiles would thus be better as it would only trigger rebuilds for commits that directly change those Containerfiles and not other changes in toolbx.

I am not familiar with quay, so pardon me :)
Is it not possible to limit build triggers to some scope (directory) of the repository?
I would also like to have a time based trigger in addition to changes in containerfiles (to have something like a be-weekly refresh of images).

Answer 13 · 2022-05-25T10:57:25.000Z

@debarshiray I'm in. username is jmennius.

Answer 14 · 2022-05-25T11:52:46.000Z

@debarshiray I'm in. username is jmennius.

Okay! I sent you an invitation.

Answer 15 · 2022-05-25T12:36:27.000Z

Having a separated repo for Containerfiles would thus be better as
it would only trigger rebuilds for commits that directly change those
Containerfiles and not other changes in toolbx.

I am not familiar with quay, so pardon me :)

I am not very familiar either. :)

Is it not possible to limit build triggers to some scope (directory) of the
repository? I would also like to have a time based trigger in addition to
changes in containerfiles (to have something like a be-weekly refresh
of images).

Umm... I suppose that should be possible. Otherwise we would need separate repositories for each distribution and version. eg., one repository for Ubuntu 22.04, another for 21.10, and so on.

@travier @Foxboron @Jmennius do you have the permissions to set up the build triggers for the images? Or do I need to tweak some knobs?

Answer 16 · 2022-05-25T12:44:02.000Z

Umm... I suppose that should be possible. Otherwise we would need separate repositories for each distribution and version. eg., one repository for Ubuntu 22.04, another for 21.10, and so on.

You can setup a build trigger for a directory on the same repository, so we just need to structure it in images/fedora/36/Containerfile or images/archlinux/rolling/Containerfile from what I can tell.

@travier @Foxboron @Jmennius do you have the permissions to set up the build triggers for the images? Or do I need to tweak some knobs?

I don't think it's possible for us to do anything with the Github integration unless we are invited to the org with some permissions to a repository.

Answer 17 · 2022-05-25T12:50:09.000Z

Regarding access to the repo I saw this.
I will work on this later this week, though..

Answer 18 · 2022-05-25T13:38:49.000Z

Unfortunately the current option with triggers on commit in Quay will always rebuild all images for each commits in a repo.

If we want something more curated, we would have to setup builds in a GitHub Action.

Answer 19 · 2022-05-25T13:39:56.000Z

Another way to have regular builds is to simply push an empty commit every two weeks.

Answer 20 · 2022-05-25T13:40:22.000Z

See https://github.com/coreos/actions-lib/tree/main/build-container for an example GitHub Action used in the CoreOS GitHub org.

Answer 21 · 2022-05-25T13:41:52.000Z

Another example, Quay only builds: https://github.com/travier/quay-containerfiles (my "personal" containers)

Answer 22 · 2022-05-25T13:43:41.000Z

https://quay.io/repository/toolbx/archlinux-toolbox > I don't have any "administrator" permission for this repo.

Answer 23 · 2022-05-25T13:46:52.000Z

Sorry, quay is new to me so I thought the permissions was inherited :) I added the contributors as admins now.

Answer 24 · 2022-05-25T13:48:35.000Z

Looks good now. Thanks

Answer 25 · 2022-05-25T15:11:42.000Z

If quay is that inflexible - can we use GH actions then (as far as i know it is flexible enough)? I assume we can still push to quay.

Answer 26 · 2022-05-25T15:40:48.000Z

For GitHub actions, we need regular commits as well to keep the repo active and thus the actions active. See https://github.com/coreos/mkpasswd-container/blob/main/.github/workflows/containers.yml.

Both solutions have trade offs.

Answer 27 · 2022-05-25T16:46:14.000Z

For GitHub actions, we need regular commits as well to keep the repo active and thus the actions active. See https://github.com/coreos/mkpasswd-container/blob/main/.github/workflows/containers.yml.

That is curious. I've had a remnant of #973 in my fork and it has been being triggered until I've disabled it manually. It'd be weird if GitHub Actions offered scheduled workflows when they're being disabled.

Answer 28 · 2022-05-26T01:00:25.000Z

@travier @Foxboron @Jmennius do you have the permissions to set up the build triggers for the images? Or do I need to tweak some knobs?

I don't think it's possible for us to do anything with the Github integration unless we are invited to the org with some permissions to a repository.

Do you need an invitation to the github.com/containers/toolbox repository?

Generally speaking, to streamline things, I think distribution image maintainers should get to own the images/<distro> hierarchy in this Git repository.

Note that I will be away from the keyboard for the next few days until Monday.

Answer 29 · 2022-06-02T12:23:02.000Z

I see that @Foxboron managed to put up a quay.io/repository/toolbx/archlinux-toolbox image pretty fast.

Do you need help with anything, @Jmennius ? If you are just busy with other things, then no stress -- take your time. :)

Answer 30 · 2022-06-02T12:27:00.000Z

I've only made the repository but I have yet to look into how we should build the image :) I'll start by touching up on the Arch image PR when I get some time next week.

Answer 31 · 2022-06-13T09:03:26.000Z

I'll setup two repos (one with GitHub Actions, one with Quay auto-build) soon (TM) so that we can have a basis for comparison and move forward. If we think other arches (aarch64?) support is important then we basically only have the GitHub Action route AFAIK.

Answer 32 · 2022-06-13T21:23:46.000Z

Here is what I did for Ubuntu on github actions + quay: Jmennius@eef9333.
Tested in my fork with Quay (in my namespace).
To me is looks like what we need, I can open a PR if there is a consensus 😄

Answer 33 · 2022-06-13T22:18:11.000Z

Here is what I did for Ubuntu on github actions + quay: Jmennius@eef9333. Tested in my fork with Quay (in my namespace). To me is looks like what we need, I can open a PR if there is a consensus smile

This does look sensible to me 👍. Considering that this would live in this repository, the concern about the schedule being disabled should not be problematic really.

Answer 34 · 2022-06-14T07:58:36.000Z

This does look sensible to me +1. Considering that this would live in this repository, the concern about the schedule being disabled should not be problematic really.

Ok, so I'll adjust it to push to toolbx org on Quay and update my Ubuntu-images PR with it. (a bit later)
In the meantime - can you create a robot user with write access to the repo and add it's token to GH secrets?

Answer 35 · 2022-06-14T20:19:26.000Z

Pushed an update in #483 - it now includes this workflow. (distro 'support' in pkg/utils now refers to quay as well).
Quay robot account secret is pending.

Answer 36 · 2022-06-15T14:20:01.000Z

Here is what I did for Ubuntu on github actions + quay: Jmennius@eef9333. Tested in my fork with Quay (in my namespace). To me is looks like what we need, I can open a PR if there is a consensus smile

Thanks, looks like a good option but this only support x86_64. That's why I linked earlier to https://github.com/coreos/actions-lib/tree/main/build-container which supports both x86_64 & aarch64.

Answer 37 · 2022-06-15T14:20:21.000Z

But of course we can start with that and improve later.

Answer 38 · 2022-06-15T14:24:52.000Z

Here is what I did for Ubuntu on github actions + quay: Jmennius@eef9333. Tested in my fork with Quay (in my namespace). To me is looks like what we need, I can open a PR if there is a consensus smile

Thanks, looks like a good option but this only support x86_64. That's why I linked earlier to https://github.com/coreos/actions-lib/tree/main/build-container which supports both x86_64 & aarch64.

Isn't this just a matter of using the platform parameter of docker build and push action (doc)?

Answer 39 · 2022-06-15T14:26:47.000Z

I did not include that in the PR because I prefer not to publish something I didn't test...
But if that's something we want/need - I can probably do it.

Answer 40 · 2022-06-16T09:46:27.000Z

It could be. I've never used the docker actions. The ones mentioned above use podman.

Answer 41 · 2022-06-19T13:03:44.000Z

I've implemented the above workflow and touched up a bit on the toolbox code to resolve Arch Linux containers images.

main...Foxboron:morten/archlinux

Everything seems to work fine :)

Answer 42 · 2022-06-23T10:07:32.000Z

I've tested multi-arch/multi-platform support with arm64/aarch64 - and it works flawlessly (with docker multi-platform action)!
See Jmennius/toolbox@ubuntu-images-github-ci~2...Jmennius:ubuntu-images-github-ci few changes I've done to make an E2E test, also images on quay.io.

The question is - which architectures do we enable?
Ubuntu images are available for those architectures: amd64, arm64, armhf, i386 (<= 18.04), ppc64el, s390x, riscv64 (>=20.04).
Judging from Fedora wiki 32-bit x86 is irrelevant, but everything else seems to be relevant.

Answer 43 · 2022-07-03T10:08:57.000Z

Any feedback, how do we proceed? @HarryMichal @debarshiray

Answer 44 · 2022-09-02T16:52:20.000Z

OK, to move forward on this one, I think we should separate the image builds from this repo and use another repo where we manage just that and trigger GitHub Actions to push builds to Quay.io.

@debarshiray Can you create a toolbox-images repo in this org and give us write access? Thanks

Answer 45 · 2022-09-02T16:52:51.000Z

The code from @Jmennius & @Foxboron looks good.

Answer 46 · 2022-09-02T17:15:27.000Z

I am assuming the official fedora-toolbox images will continue to be built within and distributed from Fedora infrastructure (koji & registry).

Answer 47 · 2022-09-02T17:19:03.000Z

Yes, this is very likely

Answer 48 · 2022-09-02T19:50:16.000Z

For some reason, I don't have the robot tab in the toolbx org in Quay so I can not use it right now.

So I've created a new one to get things moving and we'll see if we merge back everything later: https://quay.io/organization/toolbx-images. I've also created the ubuntu-toolbox & archlinux-toolbox repos there.

I've created https://github.com/toolbx-images/images, added the bot secret and pushed some initial changes from @Jmennius. I've added @Foxboron, @Jmennius, @debarshiray @HarryMichal as collaborators to the repo.

Answer 49 · 2022-09-02T19:50:49.000Z

Feel free to open an issue there if you want to get access and feel free to send PRs.

Answer 50 · 2022-09-03T01:38:42.000Z

Cool! I just tried the ubuntu image and it seems to be working:

$ toolbox create -i quay.io/toolbx-images/ubuntu-toolbox:22.04 
Image required to create toolbox container.
Download quay.io/toolbx-images/ubuntu-toolbox:22.04 (500MB)? [y/N]: y
Created container: ubuntu-toolbox-22.04
Enter with: toolbox enter ubuntu-toolbox-22.04
~
$ toolbox enter ubuntu-toolbox-22.04
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

~
⬢ ubuntu22.04$ apt -q list --installed 2>/dev/null | wc -l
348

(that's my own custom prompt of course)

Answer 51 · 2022-09-04T16:16:28.000Z

We now have images for Arch Linux, CentOS Stream 9, Debian unstable, testing, 11, 10, RHEL 9.0, 8.6, 8.4, 8.2, Ubuntu 22.04; 18.04, 16.04 so that should cover the basic use cases.

How should we go forward now?

Answer 52 · 2022-09-04T16:48:40.000Z

It would be nice to have create --distro support all the new images.

Answer 53 · 2022-09-17T14:12:51.000Z

I've created https://github.com/toolbx-images/images, added the bot secret and pushed some initial changes from @Jmennius. I've added @Foxboron, @Jmennius, @debarshiray @HarryMichal as collaborators to the repo.

I've missed the invite... can you resend it please?

Answer 54 · 2022-09-17T14:14:50.000Z

So, do we conclude that we are giving up all efforts for maintaining images in this repository, is there a decision? (we still need to upstream the integration part, which I saw no feedback on as well)

Answer 55 · 2022-09-21T12:39:58.000Z

So, do we conclude that we are giving up all efforts for maintaining images in this repository, is there a decision?

I think that is the case, yes.

(we still need to upstream the integration part, which I saw no feedback on as well)

Now that we have the images, repositories with the images and also automation and responsible people, we can start looking into the integration part. But first of all we need to do some maintenance work on Toolbx as it is currently not ready for a new release (e.g., testing is broken downstream - #959, or some images being hardcoded - #1065). We (Toolbx maintainers) need to fix that first before we can responsibly merge the added integration.

On a personal note, I'm very glad you're all working on this and trying to help Toolbx. The effort is not overlooked despite the lack of apparent progress. Thank you!

Answer 56 · 2022-09-21T13:34:45.000Z

Note that the Fedora images currently listed in images are not built from this repo but from the Fedora infra and the Containerfiles are "just" mirrored here.

If we truly want official images then they would have to be provided by the distributions themselves, otherwise it's just community images anyway, here or in https://github.com/toolbx-images/images.

What we can work on is to move the repo to the containers org if @HarryMichal @debarshiray are OK with it to make this more "official".

Answer 57 · 2022-09-21T13:39:54.000Z

If we truly want official images then they would have to be provided by the distributions themselves, otherwise it's just community images anyway, here or in https://github.com/toolbx-images/images.

Yes, that has been the intention and one cause of the slow process to support new images.

What we can work on is to move the repo to the containers org if @HarryMichal @debarshiray are OK with it to make this more "official".

I wouldn't be opposed to that. But let's wait for @debarshiray's opinion. He should be back from vacation in the coming days.

Answer 58 · 2022-11-14T20:26:16.000Z

He should be back from vacation in the coming days.

Looks like this fell through the cracks? 😂

Answer 59 · 2023-03-29T19:07:14.000Z

Phew, long thread, but I am very happy to see the progress that you folks have managed to make. :)

Let me answer the easiest questions first.

The question is - which architectures do we enable?

I think the only feasible answer is: up to you. :)

Like everything else in free software, we cannot offer things that nobody is willing to maintain. If you are a distribution maintainer for Toolbx, then problems around that distribution will end up on your lap. So, only enable what you can handle.

Of course, there are practical limitations of what the image registry and builders support, but in principle it's up to you.

Ubuntu images are available for those architectures:
amd64, arm64, armhf, i386 (<= 18.04), ppc64el, s390x,
riscv64 (>=20.04).

I see. Cool.

Judging from Fedora wiki
32-bit x86 is irrelevant, but everything else seems to
be relevant.

We only offer the registry.fedoraproject.org/fedora-toolbox images for aarch64 and x86_64. The Fedora infrastructure started supporting aarch64 somewhat recently, and we had already received some requests before that happened. I have also seen a request for ppc64le but the Fedora infra' doesn't cover that.

The registry.access.redhat.com/ubi8/toolbox and
registry.access.redhat.com/ubi9/toolbox images are available for aarch64, ppc64le, s390x and x86_64.

The thing with Toolbx is that a user on a RHEL host can use a Ubuntu container, and vice-versa. That's already very challenging in terms of the test matrix on a single architecture. I don't think we can offer images for all possible distributions covering every possible architecture that might be considered relevant for some distribution somewhere.

Answer 60 · 2023-03-29T21:56:24.000Z

By the way, we now have built-in support for the Ubuntu images from @Jmennius through --distro ubuntu.

Answer 61 · 2023-03-29T23:27:55.000Z

If we truly want official images then they would have
to be provided by the distributions themselves

My earlier comment described what I mean by supporting a distribution upstream. Then downstream, we need people to own the toolbox(1) command and the OCI image. Once we have those, it's official. :)

Ultimately, most distributions are also built and maintained by a/the community. Barring the very commercial ones like Red Hat Enterprise Linux, but even that one is increasingly trying to let the public into the sausage factory.

It seems that the various distribution communities have already packaged up Toolbx. eg., Arch Linux, Debian and Ubuntu.

@Foxboron owns the toolbox(1) command and the OCI image for Arch Linux. It's noteworthy, because it's not that different from the situation with me and Fedora, and once we wire it up to --distro, it would be the same. In the sense that Toolbx got increasingly adopted by Fedora because the community wanted it. It started off in a small corner of Silverblue as fedora-toolbox and spread organically to the point where folks like @juhp want to make it a release artifact and criterion.

I don't know if @Jmennius owns the toolbox(1) command in Ubuntu, but it would definitely be cool if he has the energy and time for it. :)

otherwise
it's just community images anyway, here or
in https://github.com/toolbx-images/images.

As I said before, it's up to the distribution and its maintainer where the built image is hosted.

I chose to host the built Fedora images on Fedora infrastructure, because I want to push for a future where Toolbx becomes part of the Fedora release artifacts and criteria so that Fedora is never released with broken Toolbx; and the RHEL images use Red Hat infra' for obvious reasons.

For the others it can be on Docker Hub, Quay, something else. As the upstream Toolbx maintainer, I don't want to get into that. It's official or supported (or some similarly grand adjective) as long as there's someone to maintain the image and to push fixes to the underlying distribution when necessary.

In fact, the Fedora and RHEL approach of keeping a copy of the images on separate Git repositories does come with the cost of keeping the copies synchronized. So, the idea of publishing the Arch Linux and Ubuntu images straight from the upstream toolbox.git is appealing to me.

What we can work on is to move the repo to the containers
org if @HarryMichal @debarshiray are OK with it to make
this more "official".

I think it's good to have a separate Git repository for distributions that don't have built-in support in Toolbx. However, as I said, for those that are built-in, the image definitions (ie., Container/Dockerfiles and associated files) should live in toolbox.git/images.

I want to get to a point where we podman build the images as part of our test suite, and run the tests both against the images that are currently on the registries and the ones that were built from source.

It will make it easy to try out changes locally when hacking. eg., it can be quite difficult to get hold of the sources for the RHEL images. If a contributor makes changes to the toolbox(1) code that requires changes to the images, or vice versa, then it's a lot easier to play with the changes and test them together, if all the Containerfiles are from a single Git repository. One single commit ID to denote a single logical change, on which the test suite was run as part of the pull request, etc..

Then there's the part where dependency chains in distributions and the base images are not set in stone. Sometimes they change in ways causing very visible breakage. So, regularly validating the sources for both the toolbox(1) command and the images can be reassuring.

All this is already difficult with only Fedora and RHEL to support. As we start advertising support for more and more distributions, it will be imperative for us to test all the combinations of containers and host distributions even more rigorously. Having the upstream copies of the image sources spread across multiple Git repositories will only cause more trouble.

Answer 62 · 2023-03-30T00:15:37.000Z

What we can work on is to move the repo to the
containers org if @HarryMichal @debarshiray are
OK with it to make this more "official".

Umm... I don't know. Is there any specific reason you want to move github.com/toolbx-images/images into the Containers organization?

I want to have clear expectations around which images and distributions are supported and to what extent. I am worried that if we move github.com/toolbx-images/images with the out-of-tree images into the Containers organization, then it may imply some higher level of support that isn't there.

If something is built-in through --distro with the image definitions in toolbox.git, then it should imply a decent degree of support. eg., users can expect not to hit regressions, to file issues against toolbox.git, etc.. Whereas for other images, we might close issues as WONTFIX because we don't care, and we can't possibly care about literally everything. :)

This is why I want to run our test suite on Ubuntu hosts, and want distribution maintainers to be willing to push fixes to the underlying distribution when necessary.

That's obviously a higher bar, and it doesn't always need to be that high.

If someone puts together a less than perfect image, which still works, and is unsure of how much energy and time they can commit on a continuous basis, then it will be good to offer a staging area for them. eg., there were many people who either expressed interest or submitted Ubuntu image definitions, but nobody other than @Jmennius wanted to stick around for the long haul. It seems like github.com/toolbx-images/images is working very well in that sense.

Answer 63 · 2023-05-10T22:29:24.000Z

Hey everyone and especially @debarshiray, in this already long discussion!

I assume we are talking about the default images for a distro that's picked to match the host or can be selected with --distro, right?

I have been meaning to wrap up the addition of Arch and Ubuntu for a long while now, but I got too stuck getting Toolbx enabled on RHEL, and taking care of the Flatpak stack in Fedora and RHEL. My apologies about that.

My opinion is that we are happy to support a distribution out-of-the-box as long as somebody from that community is happy to maintain the image, and volunteers to push through any fixes to the underlying distribution when necessary.

In concrete terms, supporting a distribution requires:
* Adding an entry for it to supportedDistros in src/pkg/utils/utils.go.

* It's up to the distro maintainer where the image is hosted.  It can be on Docker Hub, Quay, something else.  As the upstream Toolbx maintainer, I don't want to get into that.

* Wherever the image resides, we carry a copy of the Containerfile in toolbox.git/images for reference.  This makes it's easy to try out changes locally when hacking.  Things can be quite bureaucratic in some cases.  eg., RHEL.  Or, if a contributor makes changes to the Toolbx code that requires changes to the images, then it's a lot easier to play with the changes if all the Containerfiles are easily accessible from one place.
...

Does this still apply?
I am part of the Rocky Linux project and have been building the official toolbox images there since some time in the middle of last year, and thought it might finally be time to also fix the distro and release properties. 🙂
(have been using them since an even longer time, it's an amazing peace of software!)

So if it is, I can push a PR with the needed changes!

Answer 64 · 2023-05-11T12:50:15.000Z

Yes, that's still the case. Please feel free to hook up the Rocky Linux image to --distro and --release through a pull request.

I see that you are part of the Rocky Linux testing team. Are you also the person maintaining the package that ships the toolbox(1) binary on Rocky Linux? Do you know of any good CI systems that offer Rocky Linux hosts so that we can run our upstream CI on Rocky Linux as well?

Answer 65 · 2023-05-11T12:52:03.000Z

I've read this issue now and I'm still confused.

What is holding back publishing more container images right now?

Answer 66 · 2023-05-24T20:38:41.000Z

Yes, that's still the case. Please feel free to hook up the Rocky Linux image to --distro and --release through a pull request.

I see that you are part of the Rocky Linux testing team. Are you also the person maintaining the package that ships the toolbox(1) binary on Rocky Linux? Do you know of any good CI systems that offer Rocky Linux hosts so that we can run our upstream CI on Rocky Linux as well?

@debarshiray Sorry for the late reply, release weeks...
Okay I will give it a shot soon 👍🏻

I'm managing the toolbox container there (kind of part of both testing and the container and cloud SIGs), but the toolbox binary itself is managed by Releng themselves.
And unfortunately we ourselves don't know of any up to now, for our needs we are either running self-hosted Github, Gitlab or Gitea (Woodpecker CI right now) runners. But maybe that will change in the future 🙂 (for testing we are mostly settling on QEMU (OpenQA) right now)

Answer 67 · 2023-06-13T11:05:37.000Z

Yes, that's still the case. Please feel free to hook up the Rocky Linux image to --distro and --release through a pull request.
I see that you are part of the Rocky Linux testing team. Are you also the person maintaining the package that ships the toolbox(1) binary on Rocky Linux? Do you know of any good CI systems that offer Rocky Linux hosts so that we can run our upstream CI on Rocky Linux as well?

@debarshiray Sorry for the late reply, release weeks... Okay I will give it a shot soon 👍🏻

Sounds good. No need to apologize. :)

Just send a pull request once you have something ready.

I'm managing the toolbox container there (kind of part of both testing and the container and cloud SIGs), but the toolbox binary itself is managed by Releng themselves.

Okay!

And unfortunately we ourselves don't know of any up to now, for our needs we are either running self-hosted Github, Gitlab or Gitea (Woodpecker CI right now) runners. But maybe that will change in the future slightly_smiling_face (for testing we are mostly settling on QEMU (OpenQA) right now)

I see. Is there any chance of running the upstream Toolbx CI on those self-hosted runners? Our test matrix is growing very fast because we recently added built-in support for Arch Linux and Ubuntu. So, the current set-up of running our tests on only Fedora and CentOS Stream is getting more and more insufficient. :)

We can continue to discuss over email, if you want to.

Answer 68 · 2023-06-13T11:15:15.000Z

I've read this issue now and I'm still confused.

What is holding back publishing more container images right now?

I think it was a mix of various details. A big part of it was me trying to sort out my Quay.io credentials so that I could give access to @travier and others to set things up on quay.io/organization/toolbx, then setting things up on this GitHub repository, and then getting the GitHub workflows merged.

All that's done now.

The Arch Linux and Ubuntu images are already being built from this GitHub repository and published at quay.io/toolbx/.... Built-in support for Arch and Ubuntu are already merged.

So, I think we can close this now.