Optimize Docker build

Question

Optimize Docker build

Closed this issue 2 years ago · 6 comments

Currently, .dockerignore does not exclude .git.
AFAIK, the .git folder is only required by ./scripts/build.sh to figure out what is the root folder.

Finding another way to identify the root folder (currently PROJECT_ROOT=git rev-parse --show-toplevel) allows to exclude the .git folder and avoid what is currently the copy of around 800+MB to the first stage image.

Answer 1 · 2019-06-04T15:37:06.000Z

@TriplEight sorry to add you to a bunch of things, but maybe you can take a look at this one too?

Answer 2 · 2019-06-04T18:42:58.000Z

@chevdor could you specify which docker build do you mean: parity/substrate or parity/rust-builder?
First one just copies substrate's binary, second anyways git fetches the whole repository every CI job.

Answer 3 · 2019-06-05T06:48:11.000Z

I was referring to https://github.com/paritytech/substrate/blob/master/Dockerfile
In the current version, the build.sh script needs the .git folder but only to find the root folder of the project magically. If we find a way to do it without needing the .git folder (which is not needed otherwise), we spare the copy (take time and space) of ~800MB and growing. The first stage image is already very fat so losing this weight would be very beneficial, especially for CI.

Answer 4 · 2019-06-06T10:09:17.000Z

@chevdor
The thing is our current CI (GitLab) makes use of this Dockerfile: https://github.com/paritytech/scripts/blob/master/docker-files-for-Gitlab-CI-rust/rust-builder/Dockerfile
This image is universal builder for Substrate based projects.

And this image https://github.com/paritytech/substrate/blob/master/scripts/docker/Dockerfile is used for publishing Substrate's binary.

I'm not aware if the Dockerfile you are talking about is being used anywhere, please tell me if it is.

Answer 5 · 2023-03-09T00:27:25.000Z

@chevdor @TriplEight could we close this issue now that #13472 has been merged (where we used BASH_SOURCE to find project root instead of using the .git folder, and and where we added a docker/substrate_builder.Dockerfile.dockerignore file to exclude .git from the docker build to reduce the docker image size to ~277MB).

Answer 6 · 2023-03-09T08:52:53.000Z

Nicely done @ltfschoen, I see no problem closing this issue now so I will just do that, @TriplEight feel free to reopen if you see something missing.