Consolidate dependabot configurations
roypat opened this issue · 8 comments
Right now, we use dependabot for updates to our rust-vmm-ci submodule (although on wildly varying schedules), and in some repositories also for rust dependency updates (either grouped, or ungrouped). What are peoples thoughts on consolidating our dependabot configuration? For example, enable grouped rust dependency updates and rust-vmm-ci submodule updates on the same schedule for all repositories.
Questions:
- Is it possible to have a centrally managed
dependabot.yml, or do we have to copy a dependabot configuration file to every repository? - What default dependabot configuration do we want?
Right now, we use dependabot for updates to our rust-vmm-ci submodule (although on wildly varying schedules), and in some repositories also for rust dependency updates (either grouped, or ungrouped). What are peoples thoughts on consolidating our dependabot configuration?
Yeah, I think it is a good idea.
For example, enable grouped rust dependency updates and rust-vmm-ci submodule updates on the same schedule for all repositories.
Questions:
1. Is it possible to have a centrally managed `dependabot.yml`, or do we have to copy a dependabot configuration file to every repository?
AFAIK we have to create for each repository manually the .github folder where to place dependabot.yml, but at this point we could create a git submodule to share the same file with all crates.
I was thinking of two alternatives
- a single git submodule where we would put both
rust-vmm-ci,dependabot.ymland any new files, and a script to run only the first time to set up symlinks - a new git submodule specifically for this case to place in the
.githubor otherwise link somehow there
I'm more in favor of the first option to have everything in one place and improve management a bit.
2. What default dependabot configuration do we want?
In vhost and vhost-device we've been experimenting with grouped updating for a while now, and it's not bad at all, especially since we have a lot less PR to handle.
The only problem I see is that if we can't update a dependency for some reason (e.g. necessary code changes), it's not easy to ignore (at least for what we saw a few months ago). We have to tell the bot to ignore the dependency and remember to un-ignore it (e.g., by creating an issue) when we have time to fix the issues.
We also haven't found a way to query the bot and get it to tell us what dependencies we've ignored, so keeping track of them is important.
Maybe we should define this process somewhere, hopefully there is a better way now or soon.
The schedule for the updates is not arbitrary (but also not documented :(). I will explain how we decided on the current schedule.
We defined 3 types of crates:
- high traffic (more than 3 contributions per week, for the last 2 months)
- medium traffic (more than 1 contribution per week, for the last 2 months)
- low traffic (1 contribution per month)
For high traffic we have weekly updates, for medium traffic every two weeks, and for low traffic every month. The reason was that myself and @lauralt were overloaded with dependabot updates as most of the reviews were done by us. This allowed for updates to happen gradually and in order of importance of the crate. This way we didn't have to fix clippy issues/other updates for 14 crates at a time, but rather in batches. If we have the same schedule for all we will get the PRs all at once. I notice that we are already lagging behind on dependabot reviews, so having even more PRs for low traffic crates doesn't seem like a good idea to me. Ideally we would have more crate maintainers that take care of these, but it seems like we are still just a few people taking care of a lot of crates. For this reason having them all weekly seems like an overkill.
As for the other things that you are proposing, I think it's a good idea to unify the configuration.
The schedule for the updates is not arbitrary (but also not documented :(). I will explain how we decided on the current schedule.
We defined 3 types of crates:
* high traffic (more than 3 contributions per week, for the last 2 months) * medium traffic (more than 1 contribution per week, for the last 2 months) * low traffic (1 contribution per month)For high traffic we have weekly updates, for medium traffic every two weeks, and for low traffic every month. The reason was that myself and @lauralt were overloaded with dependabot updates as most of the reviews were done by us. This allowed for updates to happen gradually and in order of importance of the crate. This way we didn't have to fix clippy issues/other updates for 14 crates at a time, but rather in batches. If we have the same schedule for all we will get the PRs all at once. I notice that we are already lagging behind on dependabot reviews, so having even more PRs for low traffic crates doesn't seem like a good idea to me. Ideally we would have more crate maintainers that take care of these, but it seems like we are still just a few people taking care of a lot of crates. For this reason having them all weekly seems like an overkill.
Ah, I see, that makes sense, thanks for the explanation!
I think we can combine this approach with @stefano-garzarella's solution of providing the dependabot.yml inside rust-vmm-ci and then having each crate symlink it to their github directory, by having two configurations in rust-vmm-ci (one monthly and one weekly), and then each crate can choose which one to symlink to.
I guess we can then also add the documentation about how the different schedules are chosen when we document this symlinking process :)
Alright, I just tried this in one of my repositories, and while dependabot will follow symlinks when it comes to reading dependabot.yml, it seemingly will not follow them into submodules :(
Alright, I just tried this in one of my repositories, and while dependabot will follow symlinks when it comes to reading
dependabot.yml, it seemingly will not follow them into submodules :(
:-( this is pretty bad, so force us to copy it. Maybe we should provide a script to copy it and setup everything, and in the CI we can check that the copy is aligned (if the crate configure it in some way. I mean, we should allow crates with their own dependabot.yml IMHO)
Yeah, I think for now we can provide the sample files in rust-vmm-ci and add instructions there about how to install them (and which one to choose), I can do that today. Definitely agree that people should be able to use their own dependabot files. Let's keep this issue to brainstorm ideas how to have a non-intrusive way to prompt people to update their dependabot files from rust-vmm-ci should the default ones ever change (which, hopefully, should be a very rate occurrence). Maybe an optional PR check that only triggers on dependabot PRs for rust-vmm-ci could work? E.g. "if PR is from dependabot, updates rust-vmm-ci AND one of the new commits touches a default dependabot yml AND the dependabot.yml in the repository matches neither of the default dependabot configurations -> fail optional check"?
Maybe an optional PR check that only triggers on dependabot PRs for rust-vmm-ci could work? E.g. "if PR is from dependabot, updates rust-vmm-ci AND one of the new commits touches a default dependabot yml AND the dependabot.yml in the repository matches neither of the default dependabot configurations -> fail optional check"?
Yes, this would be perfect, but how complicated is it to do? I don't know if it's worth it, in the end I don't think we'll touch it often, and in case we could handle it manually that time it happens.
Maybe an optional PR check that only triggers on dependabot PRs for rust-vmm-ci could work? E.g. "if PR is from dependabot, updates rust-vmm-ci AND one of the new commits touches a default dependabot yml AND the dependabot.yml in the repository matches neither of the default dependabot configurations -> fail optional check"?
Yes, this would be perfect, but how complicated is it to do? I don't know if it's worth it, in the end I don't think we'll touch it often, and in case we could handle it manually that time it happens.
Yeah, I agree that maybe that's overkill. I'm not entirely sure how we would realize an optional check with our current setup, so maybe best to leave it manual and look into this more only if we for some reason start updating the dependabot file regularly