Remote containers: Support target of multi-stage Dockerfile, buildArgs

Question

Remote containers: Support target of multi-stage Dockerfile, buildArgs

stffabi opened this issue 5 years ago · 13 comments

It would be awesome if one could specify the target of a multi-stage Dockerfile to be used for developing. This makes it possible to have one Dockerfile which first creates the build-image and afterwards the runtime container. This central Dockerfile can either be used from the buildserver to create the final container or by VSCode with a target to build only the dev container.

For example the following Dockerfile could be used to build the final container.

FROM golang:1.7.3 as builder
# Here one could install more needed developer tools

FROM builder as build
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html  
COPY app.go    .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .

FROM alpine:latest  
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=build /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]

On the other hand vscode could use the builder target to build the development container:

$ docker build --target builder .

Furthermore this would make the adaption of projects to Dev-Containers especially smooth and easy...

Answer 1 · 2019-05-18T17:20:46.000Z

hi,

as this ticket is referenced by another which requests buildArgs, i'd also vote for buildArgs. atm a remote container runs as root, so all files do belong to root and not the user i want. in your doc you mention a solution like this:

ARG USERNAME=user-name
ARG USERUID=1000
ARG USERGROUP=user-group
ARG USERGID=1000

RUN groupadd -g $USERGID $USERGROUP
RUN useradd -m $USERNAME -u $USERUID -g $USERGID
ENV HOME /home/$USERNAME
...
USER $USERNAME

this helps to create a remote container which uses my UID/GID and so all files have the correct rights on the mounted filesystem.

but as you see, the USERNAME,... are build-arg's which cannot be set with generic environment-values. if i set the USERNAME (and the others) in this dockerfile to my concrete username (and uid,gid,group), other members of the team cannot use this devcontainer, because they have other values.

so it would be great to specify build-args for the docker build and so one can extract the current user (or other environment-variables) and pass them to the build of the remote container. this will help to build containers which run as the same user and UID/GID as the one executing vscode.

at the moment i use the docker-compose solution, because in a compose file one can reference environment values (and use these values as build-args for a dockerfile). but here i have other problems ... i'd personally would like to use a pure container solution instead of a compose file.

Answer 2 · 2019-05-20T07:09:05.000Z

@ulrichSchreiner What are the issues you see when using a docker-compose file? I have so far considered that an alternative for when the devcontainer.json does not support all the required options.

Answer 3 · 2019-05-20T10:53:00.000Z

@chrmarti i want my directory name to be the mountpoint on the remote container. when using a compose file with the mount option

- "..:/workspace"

the thing on the remote side is called workspace. but i normally have multiple roots in my workspace and i want it to be named as my directory. but how should i mount it, within the compose file?

perhaps i have a different look as you to remote containers. i see them as a clean-room environment, for specific types of project. so i would like to attach the .devcontainer to a xx.code-workspace file and not to a single project. when opening the workspace, all projects will be opend in the container.

at the momemt this works, by opening one project and than adding folders to the workspace (on the remote side). as i mount my $HOME to the container (but not as the HOME on the remote), this works. but i always have to reopen the workspace when i disconnect, because the connection is always initiated by a Open folder in remote container

i think you see the .devcontainer as an addon to a single workspace/folder, so the recommended way for mounting the workspace with .. is ok. and this works with a compose file.

but this is my personal opinion which is a little off-topic here i think ...

Answer 4 · 2019-05-20T12:26:29.000Z

@ulrichSchreiner Thanks for clarifying. (Related: #278)

Answer 5 · 2019-07-03T01:38:04.000Z

@chrmarti I think pretty much all commands should be extensible and also support environment variable injection otherwise it's so easy to run into a rabbit hole over small details like this.
As with have runArgs, which should have buildArgs.

For my case docker compose might work and I'll give it a go but I would have preferred to stick to a simple container approached

Answer 6 · 2019-11-16T00:14:51.000Z

From @eedwards-sk #1857

I seem to be unable to specify build args during the dev container build process.

https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg

I want to perform an npm install during the build process, but this requires providing an environment variable for npm private registry authentication.

a build arg would be the way to accomplish this (an env var)

I would like a way to provide build args during the build process, from file (so e.g. an .env file), e.g. "build_arg_file": "npmjs.env", :

{
	"name": "foo",
	"context": "..",
	"dockerFile": "Dockerfile",
	"build_arg_file": "npmjs.env",    
	"settings": { 
		"terminal.integrated.shell.linux": "/bin/bash"
	},
	"extensions": [
		"dbaeumer.vscode-eslint"
	]
}

Answer 7 · 2020-01-27T10:58:55.000Z

We also need to inject user/pass at container build time and --build-args support would be great!

Answer 8 · 2020-01-27T11:02:29.000Z

What are the issues you see when using a docker-compose file? I have so far considered that an alternative for when the devcontainer.json does not support all the required options.

@chrmarti how would you use docker-compose.yml with vscode to solve the build time, developer dependent, username and password argument to docker?

Answer 9 · 2020-02-21T11:21:40.000Z

Adding "target" and "buildArgs" (a key/value-pairs map to be passed with --build-arg) properties to the devcontainer.json.

I wonder if we should place these together with "dockerFile" and "context" into a top-level "build" property. Similar to what the docker-compose.yml does.

Answer 10 · 2020-02-21T14:24:35.000Z

So with all the build related properties it would make sense to group them in a top-level "build" property similar to what the docker-compose.yml does:

{
    "build": {
        "dockerFile": "...",
        "context": "...",
        "target": "...",
        "args": {
            "FOO": "BAR"
        }
    },
    "preBuildCommand": "..."
}

This clarifies their use while avoiding the build prefix on each. (Also: the existing top-level "context" lacks that prefix and it can be unclear what it stands for.)

"preBuildCommand" (#1045) is a bit lengthy when in the "build" bag and "preCommand" doesn't seem to work well. We can keep this one top-level like "postCreateCommand".

Answer 11 · 2020-02-23T09:37:51.000Z

Renaming preBuildCommand to initializeCommand, see #1045 (comment).

dockerFile and context will remain supported as top-level properties.

Answer 12 · 2020-03-20T00:58:41.000Z

hi,

as this ticket is referenced by another which requests buildArgs, i'd also vote for buildArgs. atm a remote container runs as root, so all files do belong to root and not the user i want. in your doc you mention a solution like this:
ARG USERNAME=user-name
ARG USERUID=1000
ARG USERGROUP=user-group
ARG USERGID=1000

RUN groupadd -g $USERGID $USERGROUP
RUN useradd -m $USERNAME -u $USERUID -g $USERGID
ENV HOME /home/$USERNAME
...
USER $USERNAME
this helps to create a remote container which uses my UID/GID and so all files have the correct rights on the mounted filesystem.

but as you see, the USERNAME,... are build-arg's which cannot be set with generic environment-values. if i set the USERNAME (and the others) in this dockerfile to my concrete username (and uid,gid,group), other members of the team cannot use this devcontainer, because they have other values.

so it would be great to specify build-args for the docker build and so one can extract the current user (or other environment-variables) and pass them to the build of the remote container. this will help to build containers which run as the same user and UID/GID as the one executing vscode.

at the moment i use the docker-compose solution, because in a compose file one can reference environment values (and use these values as build-args for a dockerfile). but here i have other problems ... i'd personally would like to use a pure container solution instead of a compose file.

I'm not sure I fully understand everything in this thread, but I was looking for a solution to this problem as well. When I create new files inside a devcontainer they all belong to root which causes some problems. Has this been solved? If so, can someone provide an example of a solution? Thanks!

Answer 13 · 2020-03-20T07:07:38.000Z

@ajschmidt8 I assume you're on Linux. This works for single containers (not Docker Compose): Make sure your image has a regular user (non-root) and set "remoteUser" in the devcontainer.json to that user's name. If that doesn't work, check if your local user's UID and GID are already taken by a different user/group in the image.