Ignoring parsing errors of parseHeaders in client.go

[prepaser]

I have to deal with the wrong response header (Referrer Policy, not Referrer-Policy maybe a typo?) sent by the server, but fasthttp causes an error, the moment I receive the wrong header field, and everything goes down. To deal with this, there is nothing I can do but change the fasthttp code. Simply deleting the validHeaderFieldByte in parseHeaders works fine. Browsers also generally seem to just ignore those fields.

Could you please provide a way to ignore the wrong header field? I think it would be enough to optionally disable header field validation, or provide a way to modify validHeaderFieldByteTable.

[erikdubbelboer]

A pull request would be welcome. Preferably matching net/http in behavior.

[ksw2000]

In net/http, they cut the header by : and convert the header key by calling canonicalMIMEHeaderKey. When canonicalizing the header key, if there is space in the key, they just skip it. See https://go.dev/issue/34540.

That is to say, when processing headers, there is no error if the header key contains "spaces" in net/http. However, if we change the example, it will still result in an error.

See the example below:

header with the key include space

func main() {
	rawRequest := "HTTP/1.1 200 OK\r\nRefer Policy: origin\r\n\r\n"

	fmt.Println("net/http:")
	reader := strings.NewReader(rawRequest)
	res, err := http.ReadResponse(bufio.NewReader(reader), nil)
	if err != nil {
		log.Println(err)
	} else {
		fmt.Printf("%#v\n", res.Header)
	}

	fmt.Println("fasthttp:")
	req2 := fasthttp.AcquireResponse()
	defer fasthttp.ReleaseResponse(req2)

	reader.Seek(0, io.SeekStart)
	err = req2.Read(bufio.NewReader(reader))
	if err != nil {
		log.Println(err)
	} else {
		req2.Header.VisitAll(func(key []byte, value []byte) {
			fmt.Println(string(key), string(value))
		})
	}
}

net/http:
http.Header{"Refer Policy":[]string{"origin"}}
fasthttp:
2024/12/13 08:57:12 error when reading response headers: invalid header key "Refer policy". Buffer size=41, contents: "HTTP/1.1 200 OK\r\nRefer policy: origin\r\n\r\n"

header with empty key

func main() {
	rawRequest := "HTTP/1.1 200 OK\r\n: empty key\r\n\r\n"

	fmt.Println("net/http:")
	reader := strings.NewReader(rawRequest)
	res, err := http.ReadResponse(bufio.NewReader(reader), nil)
	if err != nil {
		log.Println(err)
	} else {
		fmt.Printf("%#v\n", res.Header)
	}

	fmt.Println("fasthttp:")
	req2 := fasthttp.AcquireResponse()
	defer fasthttp.ReleaseResponse(req2)

	reader.Seek(0, io.SeekStart)
	err = req2.Read(bufio.NewReader(reader))
	if err != nil {
		log.Println(err)
	} else {
		req2.Header.VisitAll(func(key []byte, value []byte) {
			fmt.Println(string(key), string(value))
		})
	}
}

net/http:
2024/12/13 08:58:36 malformed MIME header line: : empty key
fasthttp:
2024/12/13 08:58:36 error when reading response headers: invalid header key "". Buffer size=32, contents: "HTTP/1.1 200 OK\r\n: empty key\r\n\r\n"