HTTP Status code of JWKS response is not checked

If the JWKS URL responds with an HTTP error status code (e.g. 4xx, 5xx), Get() does not return an error. The list of keys will be empty, leading to a silent failure.

keyfunc/get.go

Lines 147 to 157 in 57ba545

    
           resp, err := j.client.Do(req) 
        
           if err != nil { 
        
           	return err 
        
           } 
        
           //goland:noinspection GoUnhandledErrorResult 
        
           defer resp.Body.Close() 
        
           jwksBytes, err := io.ReadAll(resp.Body) 
        
           if err != nil { 
        
           	return err 
        
           }

resp.StatusCode should be checked.

I'm happy to create a pull request, if this is desired.

Thank you for opening this issue. I don't know if I considered checking resp.StatusCode when I wrote this code, perhaps it's a mistake or perhaps I intentionally omitted that because looking over the main JWK Set related RFC, RFC 7517, I can't seem to find a mention of hosting a JWK Set via HTTP and appropriate response codes. There is a mention of a "server" and requiring the use of TLS for a certain JWK member in 4.6, however I don't see this as an explicit direction applying to hosting a JWK Set as a whole. On the other hand, it does seem reasonable, if not obvious, that the expected HTTP response code should be http.StatusOK.

It seems to me there's a second part of this issue, which is below:

The list of keys will be empty, leading to a silent failure.

So I'll refer to the "first part" as the HTTP response code issue and the "second part" as the potential silent failure or unexpected emptying of the keys stored in *keyfunc.JWKS.

For the first part of this issue, I'd like to do a few things. The first thing I'd like to do is read the relevant RFCs more carefully and determine if serving a JWK Set via HTTP is mentioned and if so, see what is said about response codes. If an RFC mentions that a JWK Set served via HTTP should respond with a 200, then I would like to mark this issue as a bug, include a note in the README.md about it, then make a release that fixes the bug and does not upgrade to /v2. If we cannot find a mention in a relevant RFC about this, then the first part of this issue will not be labeled as a bug. I know that's not a satisfying answer, but I can't justify breaking backward compatibility for a use case that deals with custom HTTP codes unless the RFC has defined the behavior. The next thing I'll do about the first part of the issue is complete this new issue: #49. This new issue is something I thought about doing when I first created #35, but didn't justify the effort for in my head at the time. Additionally, it's possible we should move reprocessing check after the NewJSON function call because subsequent calls where a response body errors would not produce an error. Only the first one would.

keyfunc/get.go

Lines 159 to 168 in 57ba545

    
           // Only reprocess if the JWKS has changed. 
        
           if len(jwksBytes) != 0 && bytes.Equal(jwksBytes, j.raw) { 
        
           	return nil 
        
           } 
        
           j.raw = jwksBytes 
        
           updated, err := NewJSON(jwksBytes) 
        
           if err != nil { 
        
           	return err 
        
           }

For the second part of this issue, I haven't yet reproduced the error you are describing. I hope that any errors wouldn't go away silently, is it possible the errors are happening in the "background goroutine" and the RefreshErrorHandler field in keyfunc.Options is not populated? Below I've made a naive proof-of-concept for how keyfunc.Get will error when it receives a request with an empty body. It could be used as a template to make a proof-of-concept or two for the below items:

Make a proof-of-concept that causes the keys stored in *keyfunc.JWKS to empty when the JSON in the response body is valid and contains a non-empty JWK Set.
Make a proof-of-concept that causes an error to go unhandled when a RefreshErrorHandler is given.

If there's something I'm missing about the second part of this issue, please feel free to elaborate. I'll be attempting to make a proof-of-concept that covers one or more of the above items sometime today.

package main

import (
	"log"
	"net/http"
	"net/http/httptest"

	"github.com/MicahParks/keyfunc"
)

func main() {
	server := httptest.NewServer(http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
		writer.WriteHeader(http.StatusInternalServerError)
	}))

	var errorHandler keyfunc.ErrorHandler = func(err error) {
		log.Printf("Error from error handler: %s", err)
	}

	options := keyfunc.Options{
		RefreshErrorHandler: errorHandler,
	}
	_, err := keyfunc.Get(server.URL, options)
	if err != nil {
		log.Fatalf("Failed to get JWK Set from server.\nError: %s", err)
	}
}

@lwimmer I've made this PR: #50. I'd like to invite you to review it. I hope it can resolve the first part of this issue. I haven't been able to identify anything in a relevant RFC that says JWK Sets hosted via HTTP must return a 200 status code.

I've have not created a POC to cover the second part of this issue. Which may have been attributed due to lack of free time. If anyone can make a working POC to cover the second part of this issue, please post it here.

@MicahParks thank you for your detailed response and sorry for the delay in answering.

To be more concrete, we have run into an actual issue, where the server serving the JWKS responded with an HTTP status code of 500 and returned an error response as JSON (something like {"message":"error"}).

This leads to a silent failure, as keyfunc.Get does return a nil error, but KID() will be empty.

I have not read the JWKS RFC in detail, but it seems to focus on the data format and does not mention anything regarding fetching the JWKS. Section 4.6 only refers to the X.509 URL, which is not relevant here.
The JWKS could be supplied by different means besides HTTP(S).

But if HTTP(S) is used to fetch the JWKS, I would assume that the standard semantics of HTTP apply.

For example a 500 status code is defined by RFC 9110 as:

The 500 (Internal Server Error) status code indicates that the server encountered an unexpected condition that prevented it from fulfilling the request.

Or 400 as:

The 400 (Bad Request) status code indicates that the server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).

These do not read to me like the content of the response should be interpreted as meaningful content.

In contrast 200 is defined as:

The 200 (OK) status code indicates that the request has succeeded. The content sent in a 200 response depends on the request method. […]

This explicitly refers to the content, while the other status codes do not.

As a quick way of simulating these error responses I'm talking about, you can use something like:

server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	j := make(map[string]interface{})
	j["error"] = "an error occured"
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusInternalServerError)
	json.NewEncoder(w).Encode(j)
}))
jwks, err := keyfunc.Get(server.URL, keyfunc.Options{})

But if HTTP(S) is used to fetch the JWKS, I would assume that the standard semantics of HTTP apply.

This is an argument I've been thinking about and becoming more receptive to. Currently, I'm leaning towards treating this as a bug and making the default behavior in the next release only accepting HTTP response codes between 200 and 299 inclusive. (Perhaps only 200...) I wouldn't mark this as /v2, but maybe I should. I'm quite busy this week, so I can't promise a decision this week. I think my goal is to finish #49 sometime today, then merge that, so you have a solution in the short term, then make the breaking change/bug fix decision later.

It would be convenient for me if the upstream JWT package introduced a breaking change around this time, then I would be forced to upgrade to v2 and could include this change/others. See this PR: #46

A lesson I was hoping to avoid that it's important to get every detail perfect before releasing v1 🙂 because upgrading to v2 and beyond is difficult.

I'd also like to include behavior where keyfunc.Get will return a typed error that can be unwrapped with errors.Is if the JWK Set contains no keys. Or maybe update the documentation to show how to check the if keys are present.

I've released v1.3.0. While it doesn't change the default behavior, it will allow for the checking of the response code via a field on keyfunc.Options.

Release notes: https://github.com/MicahParks/keyfunc/releases/tag/v1.3.0

As for checking if the JWKS is empty, that could be done after the call to keyfunc.Get by calling the .KIDs method, then checking the length of the resulting slice.

My current plan is to treat this as a bug and change the default behavior to check response codes during the next release, v1.3.1.

I'd also like to include behavior where keyfunc.Get will return a typed error that can be unwrapped with errors.Is if the JWK Set contains no keys.

I'm on the fence about including this in the next release. Perhaps I'll make a .Len method that will return the length of the underlying map instead.

@MicahParks thanks, this allows me at least to implement my own check here and handle it appropriately.

Just a small feedback: With v1.3.0 and ResponseExtractorStatusOK, the behavior is as we'd expect it.

Thank you for your work!

I'm changing the default behavior to use ResponseExtractorStatusOK.

Also, @lwimmer, please upgrade to v1.4.0 if you've included v1.3.0. The ResponseExtractorStatusOK function didn't close the HTTP response body, which is a resource leak.

See the new release of v1.4.0: https://github.com/MicahParks/keyfunc/releases/tag/v1.4.0

Please feel free to reopen the issue if you feel the new release doesn't satisfy.

	resp, err := j.client.Do(req)
	if err != nil {
	return err
	}
	//goland:noinspection GoUnhandledErrorResult
	defer resp.Body.Close()

	jwksBytes, err := io.ReadAll(resp.Body)
	if err != nil {
	return err
	}

	// Only reprocess if the JWKS has changed.
	if len(jwksBytes) != 0 && bytes.Equal(jwksBytes, j.raw) {
	return nil
	}
	j.raw = jwksBytes

	updated, err := NewJSON(jwksBytes)
	if err != nil {
	return err
	}