cloud66-oss/habitus

Hangup when build with large files

Opened this issue · 4 comments

adoyle-h commented

I am building a large project with many sub-projects (the file structures are complex) and the habitus hangup in long period.

2017/07/25 02:14:33 ▶ Using '/habitus/build.yml' as build file
2017/07/25 02:14:33 ▶ Collecting artifact information
2017/07/25 02:14:33 ▶ Building 1 steps
2017/07/25 02:14:33 ▶ Step 1 - build-db-migrator, image-name = 'test:test'
2017/07/25 02:14:33 ▶ Step 1 - Build for test:test
2017/07/25 02:14:33 ▶ Step 1 - Building test:test from context '/habitus'
2017/07/25 02:14:33 ▶ Step 1 - Parsing and converting 'db-migrator/Dockerfile'
2017/07/25 02:14:33 ▶ Step 1 - Writing the new Dockerfile into '/habitus/db-migrator/Dockerfile.generated'
2017/07/25 02:14:33 ▶ Step 1 - Building the test:test image from /habitus/db-migrator/Dockerfile.generated

# The progress never update

My build.yml is simple for test:

build:
  version: 2016-03-14
  steps:
    build-db-migrator:
      name: test:test
      dockerfile: db-migrator/Dockerfile

I run the habitus in my local image habitus:runner built based on the latest source code.
And run the script to start building:

readonly _ROOT_DIR="$(cd -P -- "$(dirname -- "$0")" && pwd -P)"
readonly WORKDIR="/habitus"

docker run -it --rm \
    --workdir $WORKDIR \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v "$_ROOT_DIR":$WORKDIR \
    habitus:runner \
    habitus -env build_version=test

I have tried to build a simple repo with the same docker image and shell scripts, it works.

I noticed that the building get slower when I generated a 1GB file with truncate -s 1G block in the work directory. It will get back to normal when set .dockerignore to ignore the block file.

Although I have ignored many things in the large project via .dockerignore, it still hang.
How can I debug the habitus server?

adoyle-h commented

I think it is due to some large files still omitted to ignore.
Is there any method to help detect the problem? Maybe Sending build context to Docker daemon XXX MB message is helpful.

foldingbeauty commented

Hi @adoyle-h. Hi need to investigate if I get the the size of the building context from the docker deamon API. Will put this on my todo list. Thanks for reaching out.

mumoshu commented

For anyone interested: The docker-build implementation seems to be here: https://github.com/docker/cli/blob/82a80858857aeeb37008bd7d2f4afcda3d39c966/cli/command/image/build.go#L333

mumoshu commented

Okay, so it turns out that habitus relies on go-dockerclient and passes build context as a directory path rather than io.Reader which is required in order to calculate the progress.

Habitus: https://github.com/cloud66/habitus/blob/5d45985f7af33d1a6bcbb5c782f80fcf39b829da/build/builder.go#L267
go-dockerclient: https://github.com/fsouza/go-dockerclient/blob/cb0ae363fa010610c248eb6a397aa72345b4b406/image.go#L489
Official docker client: https://github.com/moby/moby/blob/3a633a712c8bbb863fe7e57ec132dd87a9c4eff7/client/image_build.go#L21

Fortunately, we can switch habitus to pass an InputStream. The input stream could be created from a tar stream and can we wrapped with a progress reader to achieve what we want.

  • 👍1