visual studio prompts a login window while using s3

Question

visual studio prompts a login window while using s3

danielwu223 opened this issue 2 years ago · 7 comments

Hello, I pushed some packages to our s3 bucket. in the NuGet.config file I used s3_path_to_index.json as the value of the package source, and then every time I opened up the Nuget management it asked me to log in, with the username and password required. I have the credential file in place and the env parameters with access key id and secret key.
Am I doing anything wrong here with the configuration of fetching the package via visual studio?

Thanks

Answer 1 · 2023-06-07T00:30:57.000Z

The s3 bucket needs to be public in order for NuGet to read it. NuGet doesn't support authorizing to an s3 bucket.

Answer 2 · 2023-06-07T03:05:40.000Z

ah, ok, thanks, saved me a lot of time.

Answer 3 · 2023-06-07T03:10:17.000Z

oh, one thing you might already know, your link at the bottom of the S3 document, about a private s3 bucket, hosted on LinkedIn, is broken.

Answer 4 · 2023-07-05T01:41:45.000Z

Article can be accessed via internet archive.

https://web.archive.org/web/20210122003915/http://kynatro.com/blog/2018/01/03/a-step-by-step-guide-to-creating-a-password-protected-s3-bucket/

@emgarten , I am still getting this same problem when using the CloudFront distribution method. I have not connected any of the private password stuff that is gone over in the above article. Just plain old CloudFront pointing to S3.

I can access https://cloudfronturl.com/index.json in my browser, so I assume CloudFront is working. I used the path property in my sleet.json to be the correct path and have re-created + invalidate CDN of my custom feed. I have also deleted nuget cache (as well as nuget http cache) from my machine and rebooted.

My nuget.config relative to my project has <add key="mycustom-nuget" value="https://cloudfronturl.com/index.json" /> in it. I am unsure if protocolVersion="3" is required but I seem to get the same result with and without it.

Opening nuget browser in VS4Mac it lists my custom packages fine, but when I go to install that is when I get the password prompt. The package console window has this output,

Getting restore information for project /Users/bradmoore/Projects/ConsoleTest1/ConsoleTest1/ConsoleTest1.csproj
Running restore with 12 concurrent jobs.
Reading project file /Users/bradmoore/Projects/ConsoleTest1/ConsoleTest1/ConsoleTest1.csproj.
Restoring packages for /Users/bradmoore/Projects/ConsoleTest1/ConsoleTest1/ConsoleTest1.csproj...
Restoring packages for .NETCoreApp,Version=v7.0...
Resolving conflicts for net7.0...
Checking compatibility of packages on net7.0.
Checking compatibility for ConsoleTest1 1.0.0 with net7.0.
Checking compatibility for MyCustomPackage 1.0.4 with net7.0.
Checking compatibility for System.Text.Json 5.0.1 with net7.0.
All packages and projects are compatible with net7.0.
Response status code does not indicate success: 403 (Forbidden).

I am unable to see here, or in macOS console what URL is being hit that is forbidden. I can't see a way to increase the verbosity of the package console in Visual Studio for Mac.

Is it possible there are some new endpoints during install that Visual Studio for Mac is trying to hit but sleet does not provide?

I'll try cli and I'll try on Windows and see if I get the same result.

EDIT 1:
Visual Studio 2022 on Windows also prompts and has the following output,


Restoring packages for C:\Users\brads\source\repos\ConsoleApp1\ConsoleApp1\ConsoleApp1.csproj...
  GET https://cloudfronturl.com/flatcontainer/mycustompackage/index.json
  GET https://api.nuget.org/v3-flatcontainer/mycustompackage/index.json
  OK https://cloudfronturl.com/flatcontainer/mycustompackage/index.json 132ms
  GET https://cloudfronturl.com/flatcontainer/mycustompackage/1.0.4/mycustompackage.1.0.4.nupkg
  OK https://cloudfronturl.com/flatcontainer/mycustompackage/1.0.4/mycustompackage.1.0.4.nupkg 134ms
  GET https://cloudfronturl.com/flatcontainer/system.text.json/index.json
  GET https://api.nuget.org/v3-flatcontainer/system.text.json/index.json
  NotFound https://api.nuget.org/v3-flatcontainer/mycustompackage/index.json 799ms
  OK https://api.nuget.org/v3-flatcontainer/system.text.json/index.json 838ms
  GET https://api.nuget.org/v3-flatcontainer/system.text.json/5.0.1/system.text.json.5.0.1.nupkg
  GET https://api.nuget.org/v3-flatcontainer/system.text.json/5.0.1/system.text.json.5.0.1.nupkg
  GET https://api.nuget.org/v3-flatcontainer/system.text.json/5.0.1/system.text.json.5.0.1.nupkg
  GET https://api.nuget.org/v3-flatcontainer/system.text.json/5.0.1/system.text.json.5.0.1.nupkg
  OK https://api.nuget.org/v3-flatcontainer/system.text.json/5.0.1/system.text.json.5.0.1.nupkg 13ms
Installed MyCustomPackage 1.0.4 from https://cloudfronturl.com/index.json with content hash XXXXXXXXXXX==.
Installed System.Text.Json 5.0.1 from https://api.nuget.org/v3/index.json with content hash XXXXXXXXXXX==.
  Forbidden https://cloudfronturl.com/flatcontainer/system.text.json/index.json 97ms
  GET https://cloudfronturl.com/flatcontainer/system.text.json/index.json
  Forbidden https://cloudfronturl.com/flatcontainer/system.text.json/index.json 111ms
Installing NuGet package MyCustomPackage 1.0.4.
  GET https://cloudfronturl.com/flatcontainer/system.text.json/index.json
  Forbidden https://cloudfronturl.com/flatcontainer/system.text.json/index.json 117ms
Writing assets file to disk. Path: C:\Users\brads\source\repos\ConsoleApp1\ConsoleApp1\obj\project.assets.json
Successfully installed 'System.Text.Json 5.0.1' to ConsoleApp1
Successfully installed 'MyCustomPackage 1.0.4' to ConsoleApp1
Executing nuget actions took 279 ms
Time Elapsed: 00:00:05.4411349
========== Finished ==========

Time Elapsed: 00:00:00.0149951
========== Finished ==========


  GET https://cloudfronturl.com/flatcontainer/system.text.json/index.json
  Forbidden https://cloudfronturl.com/flatcontainer/system.text.json/index.json 113ms
  GET https://cloudfronturl.com/flatcontainer/system.text.json/index.json
  Forbidden https://cloudfronturl.com/flatcontainer/system.text.json/index.json 59ms
  GET https://cloudfronturl.com/flatcontainer/system.text.json/index.json
  Forbidden https://cloudfronturl.com/flatcontainer/system.text.json/index.json 100ms

After the permissions prompt it appears to have actually installed my package unlike VS4Mac which straight up errored.
I tried to add the normal nuget feed (<add key="nuget.org" value="https://api.nuget.org/v3/index.json" protocolVersion="3" />) into nuget.config and have it above or below my custom feed hoping it will take priority for some things but it still did not want to work.

I wonder if I need to change response from CloudFront to 404 instead of 403 for when files are missing so that way VS on both platforms does not prompt for a password.

I'll also check by removing all dependencies in my nuget package if that then lets it install fine. If so its more of a CloudFront problem. Maybe a VS4Mac problem for not falling back to other feeds.

EDIT 2:

Adding a Lambda@Edge function in my CloudFront for origin response that looks like will now return 404 instead of 403, and the system works fine in Visual Studio on both macOS and Windows.

'use strict';

export const handler = async(event, context, callback) => {
    const response = event.Records[0].cf.response;
    
    if (response.status == 403)
    {
        response.status = 404;
        response.statusDescription = 'NOT FOUND';
        response.body = '';
    }

    callback(null, response);
};

Testing out the viewer request as per the blog post

'use strict';

export const handler = async(event, context, callback) => {

    // Get request and request headers
    const request = event.Records[0].cf.request;
    const headers = request.headers;
 
    // Configure authentication
    const authUser = 'YOUR USERNAME';
    const authPass = 'YOUR PASWORD';
 
    // Construct the Basic Auth string
    const authString = 'Basic ' + new Buffer(authUser + ':' + authPass).toString('base64');
 
    // Require Basic authentication
    if (typeof headers.authorization == 'undefined' || headers.authorization[0].value != authString) {
        const body = 'Unauthorized';
        const response = {
            status: '401',
            statusDescription: 'Unauthorized',
            body: body,
            headers: {
                'www-authenticate': [{key: 'WWW-Authenticate', value:'Basic'}]
            },
        };
        callback(null, response);
    }
 
    // Continue request processing if authentication passed
    callback(null, request);
};

And for good measure this is what my nuget.config looks like to make use of passwords (which I'll be moving out to be environment variable)

<?xml version="1.0" encoding="utf-8"?>
<configuration>
    <packageSources>
        <add key="MyCustomFeed" value="https://cloudfronturl.com/index.json" />
    </packageSources>
    <packageSourceCredentials>
        <MyCustomFeed>
            <add key="Username" value="YOUR USERNAME" />
            <add key="ClearTextPassword" value="YOUR PASWORD" />
        </MyCustomFeed>
    </packageSourceCredentials>
</configuration>

I feel this issue can be closed now. @emgarten, is there any place you'd like this info updated for anyone else coming across it?

Answer 5 · 2023-07-28T17:19:29.000Z

Great write up @beeradmoore, I think this would be useful as a new doc under /doc/ for AWS + Cloudfront you can add it there.

Answer 6 · 2023-07-30T22:24:37.000Z

Good idea. I am currently writing it all up as a blog post, but I'll adapt the above for /doc/ in this repo as well.