dependency proxy not supported
Closed this issue · 11 comments
This is really two issues in one. They're submitted together because I believe the solution to one probably enables the solution to the other.
A) None of the Dependency Proxy predefined variables are defined. These variables, along with what I think are reasonable default values, are shown below.
| Variable | Default Value (Proposed) |
|---|---|
CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX |
${CI_SERVER_HOST}:${CI_SERVER_PORT}/${CI_PROJECT_ROOT_NAMESPACE}/dependency_proxy/containers |
CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX |
${CI_SERVER_HOST}:${CI_SERVER_PORT}/${CI_PROJECT_NAMESPACE}/dependency_proxy/containers |
CI_DEPENDENCY_PROXY_SERVER |
${CI_SERVER_HOST}:${CI_SERVER_PORT} |
CI_DEPENDENCY_PROXY_USER |
${GITLAB_USER_LOGIN} |
CI_DEPENDENCY_PROXY_PASSWORD |
${CI_JOB_TOKEN} |
(Note that CI_PROJECT_ROOT_NAMESPACE is also not currently defined, so the proposed default value for CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX would be invalid until that is resolved. See #1357.)
B) A related issue is that the if an image from the Dependency Proxy is listed as the image for a CI job, then that job fails if the user has not already logged into $CI_DEPENDENCY_PROXY_SERVER. It's debatable whether a user should really need to take that action, given that if all of the variables above are defined, then the login can happen automatically, as shown below and documented here.
echo "$CI_DEPENDENCY_PROXY_PASSWORD" | docker login $CI_DEPENDENCY_PROXY_SERVER -u $CI_DEPENDENCY_PROXY_USER --password-stdin
Minimal .gitlab-ci.yml illustrating the issue
---
build-and-run-fortune:
image: ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker
script:
# login to dependency proxy
- echo "$CI_DEPENDENCY_PROXY_PASSWORD" | docker login $CI_DEPENDENCY_PROXY_SERVER -u $CI_DEPENDENCY_PROXY_USER --password-stdin
# build image
- docker build --pull --build-arg BASE_IMAGE=${CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX}/ubuntu -t fortune .
# run it
- docker run --rm fortuneCorresponding Dockerfile:
ARG BASE_IMAGE
FROM $BASE_IMAGE
RUN apt-get update -y && apt-get install -y fortune
CMD /usr/games/fortune
Expected behavior
- Job begins to execute (issue B above)
- Job is able to reference variables in order to complete successfully (issue A above)
Expected output
parsing and downloads finished in 49 ms.
json schema validated in 225 ms
build-and-run-fortune starting MASKED_SERVER_NAME_AND_ROOT_NAMESPACE/dependency_proxy/containers/docker (test)
build-and-run-fortune copied to docker volumes in 876 ms
build-and-run-fortune pulled MASKED_SERVER_NAME_AND_ROOT_NAMESPACE/dependency_proxy/containers/docker in 8.23 s
build-and-run-fortune $ echo "$CI_DEPENDENCY_PROXY_PASSWORD" | docker login $CI_DEPENDENCY_PROXY_SERVER -u $CI_DEPENDENCY_PROXY_USER --password-stdin
build-and-run-fortune > WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
build-and-run-fortune > Configure a credential helper to remove this warning. See
build-and-run-fortune > https://docs.docker.com/engine/reference/commandline/login/#credential-stores
build-and-run-fortune >
build-and-run-fortune > Login Succeeded
build-and-run-fortune $ docker build --pull --build-arg BASE_IMAGE=${CI_DEPENDENCY_PROXY_DIRECT_GROUP_IMAGE_PREFIX}/ubuntu -t fortune .
build-and-run-fortune > #0 building with "default" instance using docker driver
build-and-run-fortune >
build-and-run-fortune > #1 [internal] load build definition from Dockerfile
build-and-run-fortune > #1 transferring dockerfile: 150B done
build-and-run-fortune > #1 DONE 0.1s
build-and-run-fortune >
build-and-run-fortune > #2 [auth] MASKED_ROOT_NAMESPACE/MASKED_NAMESPACE/dependency_proxy/containers/ubuntu:pull token for MASKED_SERVER_NAME
build-and-run-fortune > #2 DONE 0.0s
build-and-run-fortune >
build-and-run-fortune > #3 [internal] load metadata for MASKED_SERVER_NAME_AND_NAMESPACE/dependency_proxy/containers/ubuntu:latest
build-and-run-fortune > #3 DONE 1.2s
build-and-run-fortune >
build-and-run-fortune > #4 [internal] load .dockerignore
build-and-run-fortune > #4 transferring context: 2B done
build-and-run-fortune > #4 DONE 0.0s
build-and-run-fortune >
build-and-run-fortune > #5 [1/2] FROM MASKED_SERVER_NAME_AND_NAMESPACE/dependency_proxy/containers/ubuntu:latest@sha256:b359f1067efa76f37863778f7b6d0e8d911e3ee8efa807ad01fbf5dc1ef9006b
build-and-run-fortune > #5 resolve MASKED_SERVER_NAME_AND_NAMESPACE/dependency_proxy/containers/ubuntu:latest@sha256:b359f1067efa76f37863778f7b6d0e8d911e3ee8efa807ad01fbf5dc1ef9006b 0.0s done
build-and-run-fortune > #5 sha256:b359f1067efa76f37863778f7b6d0e8d911e3ee8efa807ad01fbf5dc1ef9006b 1.34kB / 1.34kB done
build-and-run-fortune > #5 sha256:74f92a6b3589aa5cac6028719aaac83de4037bad4371ae79ba362834389035aa 424B / 424B done
build-and-run-fortune > #5 sha256:61b2756d6fa9d6242fafd5b29f674404779be561db2d0bd932aa3640ae67b9e1 2.30kB / 2.30kB done
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 0B / 29.75MB 0.1s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 3.15MB / 29.75MB 0.3s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 6.29MB / 29.75MB 0.4s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 9.44MB / 29.75MB 0.5s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 16.78MB / 29.75MB 0.7s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 23.07MB / 29.75MB 0.9s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 26.21MB / 29.75MB 1.0s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 29.75MB / 29.75MB 1.1s
build-and-run-fortune > #5 sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 29.75MB / 29.75MB 1.1s done
build-and-run-fortune > #5 extracting sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0
build-and-run-fortune > #5 extracting sha256:eda6120e237e0bdd328bc3e0f610854590400d4f96d9678dfcf781edb2f541d0 2.0s done
build-and-run-fortune > #5 DONE 3.5s
build-and-run-fortune >
build-and-run-fortune > #6 [2/2] RUN apt-get update -y && apt-get install -y fortune
build-and-run-fortune > #6 0.594 Get:1 http://archive.ubuntu.com/ubuntu noble InRelease [256 kB]
build-and-run-fortune > #6 0.601 Get:2 http://security.ubuntu.com/ubuntu noble-security InRelease [126 kB]
build-and-run-fortune > #6 1.072 Get:3 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [477 kB]
build-and-run-fortune > #6 1.122 Get:4 http://archive.ubuntu.com/ubuntu noble-updates InRelease [126 kB]
build-and-run-fortune > #6 1.305 Get:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease [126 kB]
build-and-run-fortune > #6 1.388 Get:6 http://security.ubuntu.com/ubuntu noble-security/restricted amd64 Packages [446 kB]
build-and-run-fortune > #6 1.519 Get:7 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [19.3 MB]
build-and-run-fortune > #6 1.568 Get:8 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [367 kB]
build-and-run-fortune > #6 1.676 Get:9 http://security.ubuntu.com/ubuntu noble-security/multiverse amd64 Packages [13.7 kB]
build-and-run-fortune > #6 3.409 Get:10 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1808 kB]
build-and-run-fortune > #6 3.507 Get:11 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [331 kB]
build-and-run-fortune > #6 3.554 Get:12 http://archive.ubuntu.com/ubuntu noble/restricted amd64 Packages [117 kB]
build-and-run-fortune > #6 3.558 Get:13 http://archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Packages [17.8 kB]
build-and-run-fortune > #6 3.558 Get:14 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [678 kB]
build-and-run-fortune > #6 3.682 Get:15 http://archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [507 kB]
build-and-run-fortune > #6 3.698 Get:16 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Packages [446 kB]
build-and-run-fortune > #6 3.711 Get:17 http://archive.ubuntu.com/ubuntu noble-backports/universe amd64 Packages [11.8 kB]
build-and-run-fortune > #6 4.504 Fetched 25.2 MB in 4s (6078 kB/s)
build-and-run-fortune > #6 4.504 Reading package lists...
build-and-run-fortune > #6 5.312 Reading package lists...
build-and-run-fortune > #6 6.151 Building dependency tree...
build-and-run-fortune > #6 6.323 Reading state information...
build-and-run-fortune > #6 6.608 The following additional packages will be installed:
build-and-run-fortune > #6 6.610 fortunes-min librecode0
build-and-run-fortune > #6 6.611 Suggested packages:
build-and-run-fortune > #6 6.611 fortunes x11-utils bsdmainutils
build-and-run-fortune > #6 6.639 The following NEW packages will be installed:
build-and-run-fortune > #6 6.641 fortune-mod fortunes-min librecode0
build-and-run-fortune > #6 6.845 0 upgraded, 3 newly installed, 0 to remove and 2 not upgraded.
build-and-run-fortune > #6 6.845 Need to get 711 kB of archives.
build-and-run-fortune > #6 6.845 After this operation, 2129 kB of additional disk space will be used.
build-and-run-fortune > #6 6.845 Get:1 http://archive.ubuntu.com/ubuntu noble/main amd64 librecode0 amd64 3.6-26 [625 kB]
build-and-run-fortune > #6 7.473 Get:2 http://archive.ubuntu.com/ubuntu noble/universe amd64 fortune-mod amd64 1:1.99.1-7.3build1 [32.7 kB]
build-and-run-fortune > #6 7.475 Get:3 http://archive.ubuntu.com/ubuntu noble/universe amd64 fortunes-min all 1:1.99.1-7.3build1 [53.1 kB]
build-and-run-fortune > #6 7.617 debconf: delaying package configuration, since apt-utils is not installed
build-and-run-fortune > #6 7.650 Fetched 711 kB in 1s (858 kB/s)
build-and-run-fortune > #6 7.690 Selecting previously unselected package librecode0:amd64.
(Reading database ... 4378 files and directories currently installed.)
build-and-run-fortune > #6 7.694 Preparing to unpack .../librecode0_3.6-26_amd64.deb ...
build-and-run-fortune > #6 7.711 Unpacking librecode0:amd64 (3.6-26) ...
build-and-run-fortune > #6 7.785 Selecting previously unselected package fortune-mod.
build-and-run-fortune > #6 7.787 Preparing to unpack .../fortune-mod_1%3a1.99.1-7.3build1_amd64.deb ...
build-and-run-fortune > #6 7.796 Unpacking fortune-mod (1:1.99.1-7.3build1) ...
build-and-run-fortune > #6 7.852 Selecting previously unselected package fortunes-min.
build-and-run-fortune > #6 7.854 Preparing to unpack .../fortunes-min_1%3a1.99.1-7.3build1_all.deb ...
build-and-run-fortune > #6 7.862 Unpacking fortunes-min (1:1.99.1-7.3build1) ...
build-and-run-fortune > #6 7.922 Setting up librecode0:amd64 (3.6-26) ...
build-and-run-fortune > #6 7.947 Setting up fortunes-min (1:1.99.1-7.3build1) ...
build-and-run-fortune > #6 7.975 Setting up fortune-mod (1:1.99.1-7.3build1) ...
build-and-run-fortune > #6 8.002 Processing triggers for libc-bin (2.39-0ubuntu8.3) ...
build-and-run-fortune > #6 DONE 8.2s
build-and-run-fortune >
build-and-run-fortune > #7 exporting to image
build-and-run-fortune > #7 exporting layers
build-and-run-fortune > #7 exporting layers 0.3s done
build-and-run-fortune > #7 writing image sha256:7f04e643f2548730cd93d24cf1ef162d4ea0ab252d95a9d26c47f50980d7e8a4 done
build-and-run-fortune > #7 naming to docker.io/library/fortune done
build-and-run-fortune > #7 DONE 0.3s
build-and-run-fortune >
build-and-run-fortune > 1 warning found (use docker --debug to expand):
build-and-run-fortune > - JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals (line 5)
build-and-run-fortune $ docker run --rm fortune
build-and-run-fortune > There is an old time toast which is golden for its beauty.
build-and-run-fortune > "When you ascend the hill of prosperity may you not meet a friend."
build-and-run-fortune > -- Mark Twain
build-and-run-fortune finished in 25 s
PASS build-and-run-fortune
pipeline finished in 26 s
Actual output
Issue B
The job's image is ${CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX}/docker, which resolves to /docker:latest because CI_DEPENDENCY_PROXY_GROUP_IMAGE_PREFIX is undefined.
parsing and downloads finished in 59 ms.
json schema validated in 248 ms
build-and-run-fortune starting /docker:latest (test)
build-and-run-fortune copied to docker volumes in 7.76 s
Error: Command failed with exit code 1: docker pull /docker:latest
invalid reference format
at makeError (/snapshot/firecow-gitlab-ci-local/node_modules/execa/lib/error.js:60:11)
at handlePromise (/snapshot/firecow-gitlab-ci-local/node_modules/execa/index.js:118:26)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at actualPull (/snapshot/firecow-gitlab-ci-local/src/job.ts:915:13)
at Job.pullImage (/snapshot/firecow-gitlab-ci-local/src/job.ts:928:13)
at Job.execScripts (/snapshot/firecow-gitlab-ci-local/src/job.ts:692:13)
at Job.execPreScripts (/snapshot/firecow-gitlab-ci-local/src/job.ts:641:36)
at Job.start (/snapshot/firecow-gitlab-ci-local/src/job.ts:538:9)
at /snapshot/firecow-gitlab-ci-local/node_modules/p-map/index.js:57:22
Issue A
I define the *_DEPENDENCY_PROXY_* variables as shown in the table above (including providing a definition for CI_PROJECT_ROOT_NAMESPACE), such that the job's image properly resolves.
parsing and downloads finished in 55 ms.
json schema validated in 233 ms
build-and-run-fortune starting MASKED_SERVER_NAME_AND_ROOT_NAMESPACE/dependency_proxy/containers/docker (test)
build-and-run-fortune copied to docker volumes in 1.04 s
Error: Command failed with exit code 1: docker pull MASKED_SERVER_NAME_AND_ROOT_NAMESPACE/dependency_proxy/containers/docker
Error response from daemon: Head "MASKED_SERVER_NAME/v2/MASKED_ROOT_NAMESPACE/dependency_proxy/containers/docker/manifests/latest": error parsing HTTP 403 response body: no error details found in HTTP response body: "{\"message\":\"access forbidden\",\"status\":\"error\",\"http_status\":403}"
Using default tag: latest
at makeError (/snapshot/firecow-gitlab-ci-local/node_modules/execa/lib/error.js:60:11)
at handlePromise (/snapshot/firecow-gitlab-ci-local/node_modules/execa/index.js:118:26)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at actualPull (/snapshot/firecow-gitlab-ci-local/src/job.ts:915:13)
at Job.pullImage (/snapshot/firecow-gitlab-ci-local/src/job.ts:928:13)
at Job.execScripts (/snapshot/firecow-gitlab-ci-local/src/job.ts:692:13)
at Job.execPreScripts (/snapshot/firecow-gitlab-ci-local/src/job.ts:641:36)
at Job.start (/snapshot/firecow-gitlab-ci-local/src/job.ts:538:9)
at /snapshot/firecow-gitlab-ci-local/node_modules/p-map/index.js:57:22
(Note that I masked some values pertaining to my self-managed GitLab instance.)
Host information
Ubuntu
gitlab-ci-local 4.53.0
Containerd binary
docker
Additional context
I don't think it's actually relevant, but the project is hosted on a self-hosted GitLab instance.
i feel that for CI_DEPENDENCY_PROXY_USER and CI_DEPENDENCY_PROXY_PASSWORD, they should be configured via
home-file-variables instead.
It's debatable whether a user should really need to take that action, given that if all of the variables above are defined, then the login can happen automatically, as shown below and documented here.
yeah, i would prefer the authentication to be done manually by the user,
otherwise, give #1362 a try, and let me know if you have any improvement ideas
@ANGkeith, thanks for the quick action on making changes to support this. It's not quite working for me, and I think it's due to my self-hosted instance being on a port other than 443. Please see the comments I added to the pull request.
i feel that for
CI_DEPENDENCY_PROXY_USERandCI_DEPENDENCY_PROXY_PASSWORD, they should be configured via home-file-variables instead.It's debatable whether a user should really need to take that action, given that if all of the variables above are defined, then the login can happen automatically, as shown below and documented here.
yeah, i would prefer the authentication to be done manually by the user,
Yeah, that's the approach I'm taking. I certainly see how that makes sense for the password, but I'm having trouble thinking of a situation where CI_DEPENDENCY_PROXY_USER would be anything other than $GITLAB_USER_LOGIN, so that still seems like a safe default. Are there situations you've seen where a different value would be used?
@ANGkeith, thanks for the quick action on making changes to support this. It's not quite working for me, and I think it's due to my self-hosted instance being on a port other than
443. Please see the comments I added to the pull request.
Could you dump the output of your git remote -v (remember to redact any sensitive info)
eg.
git remote -v
origin https://foo:TOKEN@qwerty.xyz:8443/foo.git (fetch)
origin https://foo:TOKEN@qwerty.xyz:8443/foo.git (push)Well, that's a surprisingly complex answer. I'd originally cloned it using https, but when I tried using gitlab-ci-local and it failed, I added another remote in order to use SSH. (I did this thinking #605 was why things weren't working.) I just use git remote rename as needed so that origin points to what I want. Prior to submitting this issue the other day, I've always used the SSH origin when using gitlab-ci-local, so that's what I'm showing below.
origin ssh://git@masked.internal.com:8022/group/repo.git (fetch)
origin ssh://git@masked.internal.com:8022/group/repo.git (push)
origin-https https://masked.internal.com:5443/group/repo.git (fetch)
origin-https https://masked.internal.com:5443/group/repo.git (push)
origin-https-token https://foo:bar@masked.internal.com:5443/group/repo.git (fetch)
origin-https-token https://foo:bar@masked.internal.com:5443/group/repo.git (push)
The "token" version I just added now, after seeing your example. It made no difference relative to the non-token version of https.
As you can see, my local instance uses non-standard ports for both the HTTPS and SSH protocols. I'm happy to help you test out any changes related to supporting those use cases.
but I'm having trouble thinking of a situation where CI_DEPENDENCY_PROXY_USER would be anything other than $GITLAB_USER_LOGIN, so that still seems like a safe default. Are there situations you've seen where a different value would be used?
ahh yes, i think you're right.
However, currently this is how we're populating the following values https://github.com/firecow/gitlab-ci-local/blob/master/src/git-data.ts#L150-L160
GITLAB_USER_NAME is derieved from git config user.name
GITLAB_USER_LOGIN is derieved from git config user.email trimming away the email portion.
i believe only the origin-https-token would return the correct username reliabily, otherwise, we can't really tell how to determine the "GITLAB_USER_LOGIN"
Hmm, my experience is that it is working, but maybe I'm misunderstanding you. I've had this in my .gitlab-ci-local-variables.yml, using the ssh clone, and it's been picking it up correctly.
CI_REGISTRY_USER: ${GITLAB_USER_LOGIN}
it's just lucky that the git config user.email happens to be your username, which may not necessarily be the case
Ah, right. That makes sense.
There was a message (that I'm guessing you've since deleted) asking me to provide you job logs. I can still provide that if needed.