cloudquery/plugin-sdk

Discussion: Dep update strategy

Closed this issue · 3 comments

yevgenypats commented

I think we should discuss how we want to do dep update. having a PR for every dep is quite heavy imo and create burden (at least emotionally for me :) ).

I think if we have one PR with all deps (I think we have in our main repo) can be a better approach and maybe run the dep checks once a month should be fine or once a week, and we can go through this and decide what we want to update and what not.

erezrokah commented

I think if we have one PR with all deps (I think we have in our main repo) can be a better approach and maybe run the dep checks once a month should be fine or once a week, and we can go through this and decide what we want to update and what not.

Happy to discuss, but I think dealing with 1 big PR with 10 unrelated changes will always be more difficult than 10 small PRs with a single change per PR. I Believe this is the case not only for dependencies updates, but in general.

Specifically let's consider the following use cases:

  1. We have 10 dependencies updates (either via a single PR or via 10 PRs). The CI passes so we merge them. I don't see a big difference between a single PR or 10 PRs in this case, especially since we auto merge most of them (We used to only auto merge patch updates, but cloudquery/.github#350 should handle both minor and digest updates)
  2. We have 10 dependencies updates. There's an unexpected breakage in one of the updates, and our CI is failing. With a single PR it's much more difficult to identify the breaking update, and also the breaking update blocks all other updates
  3. We have 10 dependencies updates. There's an unexpected breakage in one of the updates, and our CI is passing. We merge and release the breaking update. A users reports a bug and we track it to a dependency update. Now instead of doing a simple git revert on a commit, we need to either revert all updates in the commit, or craft a command to revert the specific update
  4. We have 10 dependencies updates. There's an unexpected breakage in more than one update, and our CI is failing. We fix what seems to be the breaking update, but CI is still broken. So now we're not sure if we really fixed the breaking update or there's something else that's wrong

You can see an example of 2. in cloudquery/cloudquery#2175. I grouped the Website updates to reduce the noise, but our Website build is failing on that PR. If we had multiple PRs I wouldn't even have to look at the logs to identify what broke and could merge the other updates (they would get merged automatically).

Also the benefits of multiple PRs are:

  1. If you want to skip an update you can just close the PR and the bot will ignore that update. With a single PR you'll ignore all updates so no granularity.
  2. We can do a release PR update, and if an update breaks something it will be much easier to detect what broke. I've experienced the value of this first hand were users would update to the latest version, see that it breaks, then do a kind of binary search of releases and would open an issue describing where the breakage started.

We do updates monthly so I think it's expected to have some noise (more than if we did weekly).
Also there's an interesting feature request here that can help reduce the noise while not blocking other updates if CI fails.

erezrokah commented

BTW we can use https://docs.renovatebot.com/configuration-options/#prconcurrentlimit to reduce the noise. I think the default is 10 or something like that in the base preset https://docs.renovatebot.com/presets-config/#configbase

yevgenypats commented

Alright. Closing for now and will follow-up as looks like there are bit less dep updates right now :)

  • 👍1