sourcegraph/src-cli

batches: allow resource limits to be set on spawned containers

LawnGnome opened this issue · 0 comments

From What happens when good Dockers go bad?:

Docker allows memory and CPU limits to be set on individual containers, and although these limits are optional, Docker Desktop ships with these limits enabled by default. We could trivially add limits when running containers — available memory divided by parallel processes, for example?

Unfortunately, we can’t apply limits to a group of containers, which is what would actually make more sense. If we had a way of saying “this process is going to spawn a bunch of containers; limit their total usage to 0.9*RAM”, that would be ideal, but that functionality does not exist within Docker.

Maybe an intermediate step here would be to expose a new flag or two on src batch that allows the user to opt into resource limits, but given the low awareness of -j noted above, I’m not sure that really helps.