webrecorder/browsertrix

[Feature]: Crawler cpu_limit

prestonvanloon opened this issue · 1 comments

What change would you like to see?

I would like to set resources.cpu_limit on the crawler pods so that the cluster is not overwhelmed.

Context

There is no way to set cpu_limit for the crawler pods.

This is generally not needed, since crawler pods all have a minimum CPU request - the remaining CPU is distributed amongst the available pods.
For more information why CPU requests are good, but CPU limits are bad, see: https://home.robusta.dev/blog/stop-using-cpu-limits