Optimize resource usage of our prowjobs
chrischdi opened this issue · 8 comments
Description
This is a tracking issue to go over all our prowjobs and optimise the set requests+limits for memory and CPU.
Reserving too much resources causes costs to the community which is why we should take a look and optimise our prowjob configurations.
The following dashboard from sig-test-infra helps to get a view on the current usage of the jobs:
Sheet I did use to get to the new values: here
The following prowjobs are to be adjusted:
- kubernetes/test-infra#32376
- presubmits
pull-cluster-api-apidiff-mainpull-cluster-api-build-mainpull-cluster-api-verify-main
- periodics:
periodic-cluster-api-e2e-conformance-ci-latest-mainperiodic-cluster-api-e2e-conformance-mainperiodic-cluster-api-e2e-dualstack-and-ipv6-mainperiodic-cluster-api-e2e-mainperiodic-cluster-api-e2e-mink8s-mainperiodic-cluster-api-e2e-upgrade-1-24-1-25-mainperiodic-cluster-api-e2e-upgrade-1-25-1-26-mainperiodic-cluster-api-e2e-upgrade-1-26-1-27-mainperiodic-cluster-api-e2e-upgrade-1-27-1-28-mainperiodic-cluster-api-e2e-upgrade-1-28-1-29-mainperiodic-cluster-api-e2e-upgrade-1-29-1-30-mainperiodic-cluster-api-test-mainperiodic-cluster-api-test-mink8s-main
- presubmits
In a second and/or third iteration we should address the following jobs too, by adapting values of the first round:
- periodics
periodic-cluster-api-e2e-conformance-ci-latest-release-1-7periodic-cluster-api-e2e-conformance-release-1-7periodic-cluster-api-e2e-dualstack-and-ipv6-release-1-5periodic-cluster-api-e2e-dualstack-and-ipv6-release-1-6periodic-cluster-api-e2e-dualstack-and-ipv6-release-1-7periodic-cluster-api-e2e-mink8s-release-1-5periodic-cluster-api-e2e-mink8s-release-1-6periodic-cluster-api-e2e-mink8s-release-1-7periodic-cluster-api-e2e-release-1-5periodic-cluster-api-e2e-release-1-6periodic-cluster-api-e2e-release-1-7periodic-cluster-api-e2e-upgrade-1-22-1-23-release-1-5periodic-cluster-api-e2e-upgrade-1-23-1-24-release-1-5periodic-cluster-api-e2e-upgrade-1-23-1-24-release-1-6periodic-cluster-api-e2e-upgrade-1-24-1-25-release-1-5periodic-cluster-api-e2e-upgrade-1-24-1-25-release-1-6periodic-cluster-api-e2e-upgrade-1-24-1-25-release-1-7periodic-cluster-api-e2e-upgrade-1-25-1-26-release-1-5periodic-cluster-api-e2e-upgrade-1-25-1-26-release-1-6periodic-cluster-api-e2e-upgrade-1-25-1-26-release-1-7periodic-cluster-api-e2e-upgrade-1-26-1-27-release-1-5periodic-cluster-api-e2e-upgrade-1-26-1-27-release-1-6periodic-cluster-api-e2e-upgrade-1-26-1-27-release-1-7periodic-cluster-api-e2e-upgrade-1-27-1-28-release-1-5periodic-cluster-api-e2e-upgrade-1-27-1-28-release-1-6periodic-cluster-api-e2e-upgrade-1-27-1-28-release-1-7periodic-cluster-api-e2e-upgrade-1-28-1-29-release-1-6periodic-cluster-api-e2e-upgrade-1-28-1-29-release-1-7periodic-cluster-api-e2e-upgrade-1-29-1-30-release-1-7periodic-cluster-api-test-mink8s-release-1-5periodic-cluster-api-test-mink8s-release-1-6periodic-cluster-api-test-mink8s-release-1-7periodic-cluster-api-test-release-1-5periodic-cluster-api-test-release-1-6periodic-cluster-api-test-release-1-7
- presubmits:
pull-cluster-api-e2e-blocking-mainpull-cluster-api-e2e-conformance-ci-latest-mainpull-cluster-api-e2e-conformance-mainpull-cluster-api-e2e-dualstack-and-ipv6-mainpull-cluster-api-e2e-mainpull-cluster-api-e2e-mink8s-mainpull-cluster-api-e2e-upgrade-1-29-1-30-mainpull-cluster-api-test-mainpull-cluster-api-test-mink8s-mainpull-cluster-api-apidiff-release-1-5pull-cluster-api-apidiff-release-1-6pull-cluster-api-apidiff-release-1-7pull-cluster-api-build-release-1-5pull-cluster-api-build-release-1-6pull-cluster-api-build-release-1-7pull-cluster-api-e2e-blocking-release-1-5pull-cluster-api-e2e-blocking-release-1-6pull-cluster-api-e2e-blocking-release-1-7pull-cluster-api-e2e-conformance-ci-latest-release-1-7pull-cluster-api-e2e-conformance-release-1-7pull-cluster-api-e2e-dualstack-and-ipv6-release-1-5pull-cluster-api-e2e-dualstack-and-ipv6-release-1-6pull-cluster-api-e2e-dualstack-and-ipv6-release-1-7pull-cluster-api-e2e-informing-release-1-5pull-cluster-api-e2e-mink8s-release-1-5pull-cluster-api-e2e-mink8s-release-1-6pull-cluster-api-e2e-mink8s-release-1-7pull-cluster-api-e2e-release-1-5pull-cluster-api-e2e-release-1-6pull-cluster-api-e2e-release-1-7pull-cluster-api-e2e-upgrade-1-27-1-28-release-1-5pull-cluster-api-e2e-upgrade-1-28-1-29-release-1-6pull-cluster-api-e2e-upgrade-1-29-1-30-release-1-7pull-cluster-api-test-mink8s-release-1-5pull-cluster-api-test-mink8s-release-1-6pull-cluster-api-test-mink8s-release-1-7pull-cluster-api-test-release-1-5pull-cluster-api-test-release-1-6pull-cluster-api-test-release-1-7pull-cluster-api-verify-release-1-5pull-cluster-api-verify-release-1-6pull-cluster-api-verify-release-1-7
/kind cleanup
/assign
/triage accepted
The following PR does the second round of optimisation by using the same values for all jobs:
I'll do another iteration after the next release is out.
/priority important-soon
I did take a look at the dashboards. It looks good I think for now. There's some buffer on the existing jobs but that one is not too much.
1 exception where we might want to adjust:
#10575 (comment)
/close
@chrischdi: Closing this issue.
In response to this:
/close
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.
Thx, all good. Have to think about in which job to put the linter. Maybe it gets its own job