tar fails extracting slug with resource limits set
drock opened this issue · 4 comments
I am seeing a strange problem when trying to deploy my app with resource limits set. With no resource limits set, everything works fine. However, when I set the resource limits of the app below a certain value, it fails to launch
deis limits:set web=512M
After doing so kubectl will show the new pod in a CrashLoopBackoff and the logs of the pod show
tar: ./.heroku/node/bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/regjsparser/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/regjsparser/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/regjsparser: Directory renamed before its status could be extracted
tar: ./client/node_modules/webpack/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/webpack/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/webpack: Directory renamed before its status could be extracted
tar: ./client/node_modules/acorn-jsx/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/acorn-jsx/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/acorn-jsx: Directory renamed before its status could be extracted
tar: ./client/node_modules/eslint/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/eslint/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/eslint: Directory renamed before its status could be extracted
tar: ./client/node_modules/gauge/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules/gauge/node_modules: Directory renamed before its status could be extracted
tar: ./client/node_modules/gauge: Directory renamed before its status could be extracted
tar: ./client/node_modules/.bin: Directory renamed before its status could be extracted
tar: ./client/node_modules: Directory renamed before its status could be extracted
tar: ./client: Directory renamed before its status could be extracted
tar: ./bin: Directory renamed before its status could be extracted
tar: Exiting with failure status due to previous errors
This is all strange as the error message seem to be unrelated to memory limits. The compressed slug size is ~150M and the uncompressed app folder tree is:
deis run 'du -sh'
Running 'du -sh'...
747M .
Bumping the memory limit up to only 600M solves the problem though.
Related, but probably separate is that if I drop the memory limit even lower to 400M then the failure happens when trying to merely download the slug with only this in the logs
/bin/get_object: line 13: 9 Killed objstorage --storage-type="$BUILDER_STORAGE" download "$SLUG_URL" "$GET_PATH"
My app only needs ~300M to run so I was hoping to not set the kubernetes limits so high.
I wonder if deis/controller#1179 might resolve this by proxy as it will be using a separate init container to pull down the slug.
That sounds plausible since only the memory limit seems to trigger this issue and that would download the slug and extract it before those limits are enforced?
This seems similar to docker/hub-feedback#727.
This issue was moved to teamhephy/slugrunner#4