Failed to pull manifest by the resolved digest

Question

Failed to pull manifest by the resolved digest

Closed this issue 4 years ago · 32 comments

Thank you for this nice GitHub action! It worked great until recently when I use:

    - name: Install dependencies
      uses: php-actions/composer@v5
      with:
        php_version: 7.4
        version: 2

(See https://github.com/qownnotes/api/blob/4a313056a824ba928eaedf6f8e483a88d1ae363e/.github/workflows/test.yml#L17-L21)

I get:

Run php-actions/composer@v5
Running php-build composer
WARNING! Your password will be stored unencrypted in /home/runner/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
Pulling docker.pkg.github.com/qownnotes/api/php-actions_composer_api:php-7.4-build2
WARNING: ⚠️ Failed to pull manifest by the resolved digest. This registry does not
	appear to conform to the distribution registry specification; falling back to
	pull by tag.  This fallback is DEPRECATED, and will be removed in a future
	release.  Please contact admins of https://docker.pkg.github.com. ⚠️

php-7.4-build2: Pulling from qownnotes/api/php-actions_composer_api
4c0d98bf9879: Pulling fs layer
ed03eff2d636: Pulling fs layer
fa67667da1de: Pulling fs layer
6961b2fabe93: Pulling fs layer
939a5b1ba662: Pulling fs layer
f326b7215c3e: Pulling fs layer
8339edb3b7db: Pulling fs layer
771201734758: Pulling fs layer
5b747590f2e3: Pulling fs layer
87664f078628: Pulling fs layer
6961b2fabe93: Waiting
939a5b1ba662: Waiting
f326b7215c3e: Waiting
771201734758: Waiting
5b747590f2e3: Waiting
87664f078628: Waiting
8339edb3b7db: Waiting
fa67667da1de: Verifying Checksum
fa67667da1de: Download complete
ed03eff2d636: Verifying Checksum
ed03eff2d636: Download complete
6961b2fabe93: Verifying Checksum
6961b2fabe93: Download complete
4c0d98bf9879: Download complete
4c0d98bf9879: Pull complete
f326b7215c3e: Verifying Checksum
f326b7215c3e: Download complete
ed03eff2d636: Pull complete
fa67667da1de: Pull complete
939a5b1ba662: Verifying Checksum
939a5b1ba662: Download complete
6961b2fabe93: Pull complete
771201734758: Verifying Checksum
771201734758: Download complete
939a5b1ba662: Pull complete
f326b7215c3e: Pull complete
8339edb3b7db: Verifying Checksum
8339edb3b7db: Download complete
5b747590f2e3: Verifying Checksum
5b747590f2e3: Download complete
87664f078628: Verifying Checksum
87664f078628: Download complete
8339edb3b7db: Pull complete
771201734758: Pull complete
5b747590f2e3: Pull complete
87664f078628: Pull complete
Digest: sha256:3a57167d9218135c4438354da850b15c6c844a3f0ad9ea3ff1b4968325133725
Status: Downloaded newer image for docker.pkg.github.com/qownnotes/api/php-actions_composer_api:php-7.4-build2
docker.pkg.github.com/qownnotes/api/php-actions_composer_api:php-7.4-build2
Docker tag: docker.pkg.github.com/qownnotes/api/php-actions_composer_api:php-7.4-build2
Error: Process completed with exit code 1.

(See https://github.com/qownnotes/api/runs/2278527522?check_suite_focus=true)

Retrying the build process doesn't help.
Did anyone else see this?

Answer 1 · 2021-04-06T13:44:35.000Z

Hello, thank you for your message. I have just hit this myself and have found out what the issue is: recently Github bumped the default OS from Ubuntu 18.04 to Ubuntu 20.04. I'm not sure why this has caused an issue, but it's certainly too much of a coincidence to ignore.

My plan is to hardcode the reference to the previously working Ubuntu 18.04, make a patch to the v5 release, then move on to investigate what incompatibility there is caused by the newer OS.

Answer 2 · 2021-04-06T13:52:02.000Z

Ah, great. Thank you very much! So I can stay at ubuntu-latest when you update php-actions/composer@v5?

Answer 3 · 2021-04-06T13:53:29.000Z

I definitely want this to continue working on any ubuntu (and I'm also planning how to make it work on Windows, but that's another story).

Currently not sure exactly what's failing, but I hope I'll find it soon as it's going to start affecting a lot of people, including my own repositories.

Any help identifying the issue is appreciated.

Answer 4 · 2021-04-06T13:57:12.000Z

I am getting the same here on my repos 😬

I tried downgrading from ubuntu-20.04 (which I was on before, which worked) to 18.04. Did not help.

Answer 5 · 2021-04-06T13:59:09.000Z

I definitely want this to continue working on any ubuntu (and I'm also planning how to make it work on Windows, but that's another story).

Currently not sure exactly what's failing, but I hope I'll find it soon as it's going to start affecting a lot of people, including my own repositories.

Any help identifying the issue is appreciated.

Seems to be to do with the docker version as opposed to Ubuntu (obviously newer ubuntu shipped with newer Docker)

https://docs.docker.com/engine/deprecated/#pulling-images-from-non-compliant-image-registries

Attempting to help look at what might be the exact cause and will report back if i find anything.

Anyone seeing this on any other GH registry hosted images or just this one?

Answer 6 · 2021-04-06T13:59:34.000Z

Could be a docker thing. Lately I'm getting errors like Could not do a head request, falling back to regular pull. when Watchtower is pulling my repos...

Answer 7 · 2021-04-06T14:00:56.000Z

I can also confirm what @polesen states: downgrading Ubuntu does not fix this. I'm testing on https://github.com/php-actions/example-composer - investigating Docker repository now.

Answer 8 · 2021-04-06T14:02:48.000Z

I hope it not that the newish Docker API rate limiting is affecting the GitHub Action servers.

Answer 9 · 2021-04-06T14:15:17.000Z

Maybe it is time to switch away from docker.pkg.github.com in the php-actions build script that is sourced, as it seems like that registry does not conform to the (docker)spec and will be a problem also going forward:
moby/moby#41687

GCHR seems to be their(github's) solution for docker images, going forward:
https://docs.github.com/en/packages/guides/migrating-to-github-container-registry-for-docker-images

Which should be a compliant registry.

Answer 10 · 2021-04-06T14:21:44.000Z

This repo originally used GHCR but it required the user to register their own Personal Access Token to use, and plenty of other limitations that made it run incredibly slowly and tediously. It might be improved now, but it was almost unusable before. I'll look into it again once I've resolved this.

Answer 11 · 2021-04-06T15:04:41.000Z

Hey everyone, please can you all test to see if a change I've made fixes this for your repositories?

If you could change uses: php-actions/composer@v5 to the following: uses: php-actions/composer@april-2021-issue, it will start to use the ghcr.io domain.

I've checked and there have been loads of updates to GHCR. Mainly, the GITHUB_TOKEN environment variable now works, which is great - let's see if this simple fix will resolve our issues.

Answer 12 · 2021-04-06T15:08:08.000Z

@g105b thx for investigating this issue!!

On my project with uses: php-actions/composer@april-2021-issue on ubuntu-latest the action is running again! :)

Answer 13 · 2021-04-06T15:14:05.000Z

Phew! I'll wait for confirmation from others, then I'll make a patch release to composer@v5 so others can continue without having to change anything.

Answer 14 · 2021-04-06T15:17:14.000Z

It works, and it does not work. It seems to be getting the image now, but the previous bug from this issue #47 re-appears, not being able to clone private repos, which I guess is due to this fix branch being branched from master.

So, a backport of the ghcr fix to v5 would probably work for me 👍

Answer 15 · 2021-04-06T15:19:13.000Z

Initially I got the following error

denied: failed_precondition: Improved container support has not been enabled for 'ORGNAME'. Learn more: docs.github.com/packages/getting-started-with-github-container-registry/enabling-improved-container-support

Having followed those instructions to enable improved container support, it does indeed work as expected using

    runs-on: ubuntu-20.04
    uses: php-actions/composer@april-2021-issue

Answer 16 · 2021-04-06T16:01:27.000Z

I've taken into account @polesen's latest tip, and have updated the april-2021-issue branch to reflect this. Please test again. I am having success here: https://github.com/php-actions/example-composer/runs/2279874830?check_suite_focus=true (it even requires a private repo).

Answer 17 · 2021-04-06T16:18:48.000Z

Update: I've tested all the various setups that I've come across while maintaining this repo here: https://github.com/php-actions/example-composer/actions

The latest run "Update all workflows to test April 2021 issue fix" changed all workflow config files to use april-2021-issue branch, and they all ran successfully.

If there's any other case that anybody can report that doesn't work, I'll try and replicate in the example repository before making a backport patch into v5. I'll make the patch at 09:00AM BST tomorrow (Wednesday 7th April) if there aren't any more issues reported on this.

Answer 18 · 2021-04-06T17:22:47.000Z

My block

    - name: Install dependencies
      uses: php-actions/composer@april-2021-issue
      with:
        php_version: 7.4
        version: 2

...runs through when using php-actions/composer@april-2021-issue after I enabled improved container support for the organization of the repository.

But my block:

    - name: Run tests
      uses: php-actions/composer@april-2021-issue
      with:
        command: run test
        php_version: 7.4
        version: 2

... fails in https://github.com/qownnotes/api/runs/2280392933?check_suite_focus=true#step:4:253.
I guess I need to more look into the GH registry...

Answer 19 · 2021-04-06T18:06:07.000Z

@g105b Excellent, my build runs without errors now on april-2021-issue 👍

Answer 20 · 2021-04-06T18:07:41.000Z

On my project, it started to work with v5 (without any changes).

      - name: Install composer dependencies
        uses: php-actions/composer@v5
        with:
          php_version: 7.4
          version: 2

Answer 21 · 2021-04-06T19:12:02.000Z

Initially, I got the same error as @benswinburne:

denied: failed_precondition: Improved container support has not been enabled for 'ORGNAME'. Learn more: docs.github.com/packages/getting-started-with-github-container-registry/enabling-improved-container-support

After enabling improved container support, it started to work with v5 on ubuntu-18.04:

    - name: Install dependencies
      uses: php-actions/composer@v5
      with:
        php_version: 7.2
        version: 1

Answer 22 · 2021-04-06T19:48:17.000Z

It looks like the issue is due to something changing with the GitHub container repository. I'll contact their support to learn more before making any changes to the v5 patch, as I want to ensure any changes are actually going to improve things.

Answer 23 · 2021-04-07T12:40:45.000Z

I'm waiting to hear back from Github Support, but in the meantime I have successfully managed to run this action without having to change anything, so it seems the issue was only temporary.

Please can people provide an update here if they're still having any issues?

Answer 24 · 2021-04-07T13:33:14.000Z

I've been running other workflows today, which run with v5 composer install and they have worked, so yes, it seems to not be failing at the moment.

Answer 25 · 2021-04-08T06:55:00.000Z

I also can confirm php-actions/composer@v5 is working for me again! Thank you very much for all the investigation!

Answer 26 · 2021-04-08T11:06:28.000Z

Please feel free to re-open this issue if there are any other problems using the container repository.

Answer 27 · 2021-04-13T08:30:07.000Z

Hi everyone.

I'm just following up with the response from GitHub support.

We did have an incident last week that may have affected workflow runs around that time period.

The Packages team does suggest users migrate to the Container Registry and the GITHUB_TOKEN now supports private package auth:

https://github.blog/changelog/2021-03-24-packages-container-registry-now-supports-github_token/

But I don't think users can delete container images themselves at this time:

https://docs.github.com/en/packages/learn-github-packages/deleting-and-restoring-a-package

Answer 28 · 2021-05-03T14:30:00.000Z

Apparently i am still getting the same error, i already enabled the improved container support.
Tried it with ubuntu-latest and ubuntu-18.04

Run php-actions/composer@v5
  with:
    ssh_key: ***
    ssh_key_pub: ***
    php_version: 7.4
    version: 2
    args: --profile --ignore-platform-reqs
    command: install
    interaction: no
    dev: yes
    progress: no
    quiet: no
# github.com:22 SSH-2.0-babeld-10d0a39d
# gitlab.com:22 SSH-2.0-OpenSSH_7.9p1 Debian-10+deb10u2
# bitbucket.org:22 SSH-2.0-conker_e3385e86ad 3150b49e095e
/usr/local/bin/composer: line 2: syntax error: unexpected newline
Running php-build composer
WARNING! Your password will be stored unencrypted in /home/runner/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
Pulling docker.pkg.github.com/bergwerk/ddm-typo3/php-actions_composer_ddm-typo3:php-7.4-build2
WARNING: ⚠️ Failed to pull manifest by the resolved digest. This registry does not
	appear to conform to the distribution registry specification; falling back to
	pull by tag.  This fallback is DEPRECATED, and will be removed in a future
	release.  Please contact admins of https://docker.pkg.github.com. ⚠️

php-7.4-build2: Pulling from bergwerk/ddm-typo3/php-actions_composer_ddm-typo3
540db60ca938: Already exists
933cf2f4a68f: Pulling fs layer
93c5cc202a60: Pulling fs layer
74403c16157d: Pulling fs layer
f39378a44d29: Pulling fs layer
9417049c4f35: Pulling fs layer
21742283d02f: Pulling fs layer
50815441106c: Pulling fs layer
13a20d3bc047: Pulling fs layer
f2d717243aa2: Pulling fs layer
f39378a44d29: Waiting
9417049c4f35: Waiting
21742283d02f: Waiting
50815441106c: Waiting
13a20d3bc047: Waiting
f2d717243aa2: Waiting
93c5cc202a60: Verifying Checksum
93c5cc202a60: Download complete
74403c16157d: Verifying Checksum
74403c16157d: Download complete
933cf2f4a68f: Verifying Checksum
933cf2f4a68f: Download complete
9417049c4f35: Verifying Checksum
9417049c4f35: Download complete
933cf2f4a68f: Pull complete
93c5cc202a60: Pull complete
50815441106c: Verifying Checksum
50815441106c: Download complete
74403c16157d: Pull complete
f39378a44d29: Verifying Checksum
f39378a44d29: Download complete
13a20d3bc047: Verifying Checksum
13a20d3bc047: Download complete
21742283d02f: Verifying Checksum
21742283d02f: Download complete
f39378a44d29: Pull complete
9417049c4f35: Pull complete
f2d717243aa2: Verifying Checksum
f2d717243aa2: Download complete
21742283d02f: Pull complete
50815441106c: Pull complete
13a20d3bc047: Pull complete
f2d717243aa2: Pull complete
Digest: sha256:ad29f0c234107732e345a88c17b80c84d217cecbaee2b9b3ca3abc5bfe47ac36
Status: Downloaded newer image for docker.pkg.github.com/bergwerk/ddm-typo3/php-actions_composer_ddm-typo3:php-7.4-build2
docker.pkg.github.com/bergwerk/ddm-typo3/php-actions_composer_ddm-typo3:php-7.4-build2
Docker tag: docker.pkg.github.com/bergwerk/ddm-typo3/php-actions_composer_ddm-typo3:php-7.4-build2
Storing private key file for root
PRIVATE KEY:
0e375b3535500c915ee91236cee9f079  /home/runner/.ssh/action_rsa
PUBLIC KEY:
8b459f4e91e04fe0fec1577e1e1f5d79  /home/runner/.ssh/action_rsa.pub
Command: composer install --no-progress --no-interaction --profile --ignore-platform-reqs
Error: Process completed with exit code 1.

Answer 29 · 2021-05-03T15:49:23.000Z

This issue popped up for me just now as well.

Answer 30 · 2021-05-03T16:49:54.000Z

Also experiencing this "/usr/local/bin/composer: line 2: syntax error: unexpected newline" as of this morning.

Answer 31 · 2021-05-04T13:40:54.000Z

I'll jump back onto the ticket for Github support. I don't know where we can find out any more information because https://www.githubstatus.com/ tells us "All systems are operational"...

Answer 32 · 2022-10-19T16:55:25.000Z

Is there any update to this issue? We're experiencing it as well in v6