Artifact download URL only work for registered users (404 for guests)

Question

Artifact download URL only work for registered users (404 for guests)

JayFoxRox opened this issue 5 years ago · 32 comments

We used to copy the URL from https://github.com/actions/upload-artifact#where-does-the-upload-go to our website for nightly builds (which have short retention and frequent updates, and don't warrant a GitHub Releases push).

We intended to replace this with programmable download URLs that have been discussed in many issues on the repositories (either a latest/release.zip or a third party service which asks the Actions API for the latest URL and redirects the user).

This stopped working very recently.

Until recently, the actions tab was only viewable for logged in users (confusing 404 HTTP error for guests), but the artifact download URLs were still public (working for logged in users and also guests).

Likewise, when the Actions API was released, API requests worked without any authentication. One could simply query the API for an artifact download URL and redirect the guests to it. - All of that worked without GitHub account and could have worked from JavaScript or a small lightweight webservice which redirects the end-user to the latest artifact download through HTTP redirects (I wrote https://github.com/JayFoxRox/GitHub-artifact-URL for this purpose).

However, within the last days these artifact download URLs were suddenly made private - they only work for registered GitHub users now. Everyone clicking the download button on our website (who isn't logged into GitHub) gets a confusing 404 error for direct artifact download URLs now.

Even my tool to redirect users doesn't work anymore because the Actions API also requires the API client to be authenticated now (also getting a confusing "not found" error otherwise). See JayFoxRox/GitHub-artifact-URL#4 ; even if I implemented authentication now, the download URL (we redirect to) would likely not work for guest users (such as end-users of our software, who don't have a GitHub account).

It really starts to feel like we are working against how GitHub Actions is intended to work (now, and in the future - none of which seems to be documented very well). It is clearly different from any other CI I have ever worked with; because Travis and AppVeyor had public artifact URLs with simple URLs, which could be easily linked from our website (Compare AppVeyor).

The GitHub documentation for artifacts says

Artifacts allow you to share data between jobs in a workflow and store data once that workflow has completed.

The first part is obvious; but the second part is really vague. Who shall access this stored data? Why? - My thinking has been (from experience with other CI systems): to share temporary builds, with users for early-testing ("continuous integration"), without pushing a release.

Please make GitHub Actions artifact URLs work for guest users. If this is a non-goal, it should be documented.
Please document how things are supposed to work for real world applications (How to best handle nightly builds - currently neither GitHub Releases or GitHub Actions artifacts seem suitable).

c3rb3ru5d3d53c commented 2 years ago

bump?

barakugav commented a year ago

bump?

👍12
😕5

Answer 1 · 2020-02-04T07:58:56.000Z

This broke our workflow for us as well, used to use this feature to test automated builds in virtual machines where we don't want to login.

Answer 2 · 2020-05-02T12:59:38.000Z

Fun fact: for the API even authenticating is not enough! So, through the API you can't do what you can do through the browser (i.e. with cookies).

This works (assuming logged in):
https://github.com/oprypin/crystal/suites/626035583/artifacts/5010746

This doesn't (using a personal access token for authentication):

$ curl -u "$USER:$PERSONAL_TOKEN" https://api.github.com/repos/oprypin/crystal/actions/artifacts/5010746/zip
{
  "message": "Must have admin rights to Repository.",
  "documentation_url": "https://developer.github.com/v3/actions/artifacts/#download-an-artifact"
}

Which goes directly against the documentation (linked in the response itself!!!):

Anyone with read access to the repository can use this endpoint.

But yes, it does work when I use the API to download from my own repository and only if the token has repo access. And no, my repository isn't private.

Answer 3 · 2020-05-02T13:44:08.000Z

@oprypin i've had success with
Authorization: token $PERSONAL_TOKEN

Edit: that works with a personal token with no permissions, for any public repo.

Answer 4 · 2020-05-02T14:06:51.000Z

@MCOfficer I am not able to reproduce your success there. I get the exact same results as with my example (and this doesn't seem to apply to the browser-based URL either).

Answer 5 · 2020-05-02T14:10:19.000Z

@oprypin try this:

$ curl -v -H "Authorization: token $SOME_TOKEN_WITHOUT_PERMISSIONS" https://api.github.com/repos/endless-sky/endless-sky/actions/artifacts/5036752/zip

you should get a Location header, among others. I don't have any special permissions to endless-sky/endless-sky, and this works for me.

Answer 6 · 2020-05-02T14:18:41.000Z

OK so it works but the token must have public_repo access.

Answer 7 · 2020-06-22T01:41:49.000Z

It's worthwhile noting other continuous-integration platforms such as AppVeyor allow guests to download artifacts.

Quassel IRC, an open source IRC client, made use of this to allow for non-developers in the community to test new builds and new features, and since moving to GitHub Actions guests can no longer download artifacts.

If GitHub does not want to universally allow guests to download artifacts, an option on a per-repository or per-organization level would sufficiently address this issue.

Note: I am a community contributor to Quassel IRC. I am not speaking officially on behalf of the Quassel project.

Answer 8 · 2020-11-30T19:21:59.000Z

I have developed a service that lets anyone download an artifact without logging in.
It can also automatically find the currently-latest artifact of any branch.
This is exposed as permanent, shareable URLs, including direct ZIP download URLs.

https://nightly.link/

https://github.com/oprypin/nightly.link

This is implemented a GitHub App, and communication with GitHub is authenticated, but users that only download don't need to deal with that at all.

Answer 9 · 2020-12-09T14:30:33.000Z

I just implemented CI/Actions on all my main repositories but I hadn't realised there's a blanket policy to deny public access to artifacts. This seriously cripples the artifact mechanism for some obvious use cases. At least the policy should be modifiable at some granularity via the API??

Answer 10 · 2021-03-05T15:50:44.000Z

I just got hit by this as well, I had scripts that download some artifacts from github and they're all broken now. Adding authentication to these scripts is not ideal.

Answer 11 · 2021-04-21T10:25:08.000Z

Wow, https://nightly.link/ provides exactly what we also need! Thank you so much for this service, @oprypin <3

Please GitHub, it would be amazing if you could provide the same user/repo/workflows/build-latest/master/foo.zip

Answer 12 · 2021-06-07T16:19:55.000Z

That was already clarified:

#51 (comment)

@oprypin try this:
$ curl -v -H "Authorization: token $SOME_TOKEN_WITHOUT_PERMISSIONS" https://api.github.com/repos/endless-sky/endless-sky/actions/artifacts/5036752/zip
you should get a Location header, among others. I don't have any special permissions to endless-sky/endless-sky, and this works for me.

OK so it works but the token must have public_repo access.

The rest still stands

Answer 13 · 2022-01-20T13:53:50.000Z

(edited for emphasis, re @oprypin:)
Hello! As of January 2022 trying to download an artifact via a URL of the format:
https://api.github.com/repos/{user}/{repo}/actions/artifacts/{run_id}/zip

still, two years since this issue's opening, results in the error: "You must have the actions scope to download artifacts."

Having this link be publicly accessible would be the simplest way to distribute nightly builds to my team.

Having read through this and all the related open issues, github has never commented on or officially documented this feature disparity with other CI/CD solutions. I would transition to a different solution, but am on a deadline.

Answer 14 · 2022-01-20T13:57:19.000Z

You're saying "As of January 2022" like it's something new. But it's always been that way. Check for alternatives above in the thread.

Answer 15 · 2022-01-20T14:04:05.000Z

You're saying "As of January 2022" like it's something new. But it's always been that way. Check for alternatives above in the thread.

Have updated my original comment; my point is it seems a lot of people feel it shouldn't be this way. nightly.link is a wonderful service but it's still ultimately a middleman adding complexity.

Answer 16 · 2022-04-11T19:56:27.000Z

Any idea when this gets implemented?

Answer 17 · 2022-08-25T23:23:18.000Z

Bump? It's been 2 3/4 years since this issue was opened, and I still don't understand what GitHub expects us to do for nightly builds. https://nightly.link/ works great (thanks @oprypin), but it's an additional service I have to configure, and adds another middleman. Can someone at GitHub at least give us a statement, and what we are supposed to do?

Answer 18 · 2022-08-25T23:30:02.000Z

It's effort on the project's part, but one workaround is to publish artifacts to a nightly tag and update it with every CI run. See for example Endless Sky, AppImageKit

Answer 19 · 2022-08-25T23:33:24.000Z

See, the problem I have (for my project) is that I don't just have one nightly build- I want to have builds of every commit.
I'm trying use Swift packages for a Rust project, so I have an action that builds it, uploads it with nightly.link, and then generates a manifest with the URL and hash, which is then committed to swift/originalbranchname. So you can put swift/originalbranchname into Xcode, and it works.

Answer 20 · 2022-11-20T01:01:33.000Z

Any progress regarding this? nightly.link is an absolute hack and we shouldn't need to use it so that users can get the latest software.

Answer 21 · 2023-02-09T18:54:03.000Z

This is still a problem in 2023.

Answer 22 · 2023-02-09T19:23:29.000Z

Here's an alternative that works:

Use Releases, create a latest release in your repository (tag + description)
Add an upload/release/nightly job to your workflow
Use pyTooling/Actions/releaser@main to upload your artifact to the latest tag

Example here: https://github.com/kernelkit/infix/blob/main/.github/workflows/build.yml

For every build your latest release is updated with artifacts from the latest build. The tag will remain where you created it, so slightly confusing, but works and will always be at the top of your Releases listing. Good luck!

Answer 23 · 2024-01-16T15:02:58.000Z

Solution similar to proposed by @troglobit

- name: Release nightly
  uses: softprops/action-gh-release@v1
  with:
    prerelease: true
    name: nightly
    tag_name: nightly
    files: file.txt
    fail_on_unmatched_files: true

And instruct your users to download from https://github.com/OWNER/REPO/releases/download/nightly/file.txt
Example here.

That file.txt will be updated on every commit, keeping single pre-release release without cluttering with other releases and latest stable release.

As a bonus it won't disappear if you don't commit for 90 days (as with artifacts).

Answer 24 · 2024-02-22T23:22:52.000Z

Gosh this really kinda sucks. Ran into this today and can't believe this hasn't been fixed... Github team... why do we do this and make it hard for users?

@konradpabjan , is this bandwidth limiting or other abuse issue ? Letting users turn it on per-repo at least would be maybe ok ?

Answer 25 · 2024-03-28T17:29:16.000Z

@psiberfunk I don't think that GH will do anything about this and services like nightly.link will be the only alternative sadly.

Answer 26 · 2024-04-23T07:34:49.000Z

I... don't get it. You can maintain a CD pre-release, update it continuously, and the download link works not just for logged-of users, but with plain wget. For example #51 (comment) or: https://github.com/AppImage/type2-runtime/releases/download/continuous/runtime-x86_64

Does this problem actually exist?

Answer 27 · 2024-04-23T07:48:46.000Z

@mralusw this is for downloading artifacts from the Action Runners. Sure you can attach them to a pre-release.. but that's not wanted in every usecase.

The problem that the artifacts are only downloadable when logged into Github is still an issue.

Try to get the artifact when logged in vs when in Incognito mode: https://github.com/performous/performous/actions/runs/8774492740/artifacts/1433575543

When in incognito mode you'll be presented a 404.

Answer 28 · 2024-04-23T08:53:15.000Z

If you want a legit usecase, i'm maintaining a launcher for endless-sky, which allows users to test new features (i.e. download builds of PRs before they are merged). We do maintain a pre-release for the latest master build, but there are also 3 builds (Win/Linux/MacOS) for each of the 156 current open PRs - shoving all those into the release section would be madness.

Sure, we could create an extra repo for the sole purpose of holding hundreds of releases, but at that point we are just monkeypatching together a service that github themselves could - should - provide themselves. So yes, this is an issue.

Answer 29 · 2024-04-23T11:24:17.000Z

I don't think this will ever change, even if a thousand users complain, GitHub is just too big. You could either use a hacky solution like nightly.link which might break at any point in time or use another CI platform (for example a selfhosted one like Jenkins, or you could mirror your repo to GitLab and use their CI maybe)

Answer 30 · 2024-07-16T16:33:48.000Z

For anyone struggling with this issue, I think the approach implemented in this tool solves this pretty well, might be worth giving it a try:

https://github.com/TheAssassin/pyuploadtool

I found about about this after quite extensive research and wanted to leave it here as bread crumb for others to find the short cut :)