HEAD fails on compressed response
blyxxyz opened this issue · 4 comments
$ xh head httpbin.org/gzip
[...]
xh: error: error decoding gzip response body: failed to fill whole buffer
This didn't happen before #241, so reqwest deals with it correctly. I don't know how.
This comment from seanmonstar/reqwest#83 might be relevant
// libflate does a read_exact([0; 2]), so its impossible to tell
// if the stream was empty, or truly had an UnexpectedEof.
// Therefore, we need to peek a byte to make check for EOF first.
It took me a while to understand, but I think the relevant piece in the current reqwest code is here: https://github.com/seanmonstar/reqwest/blob/2a6e012009fb79065767cb49a8a000d354c47ba6/src/async_impl/decoder.rs#L285
reqwest doesn't wrap a decoder around the stream until it did a poll_peek
to make sure that there's actual data to receive. If there's no data then it creates a "fake" empty stream.
So we have the following situations:
- When xh 0.15 receives an empty gzip response of any kind it turns it into an empty stream, never even constructing a decoder.
- When xh 0.16 receives an empty gzip response to a GET request, marked with
Content-Length: 0
, we handle that explicitly by not trying to decode. - When xh 0.16 receives an empty gzip HEAD response it doesn't trigger the explicit handling so we try to decode it and fail.
So we need to skip the gzip decoding or ignore the error if the body is totally empty.
Maybe we can fold this into the existing InnerReader
trick? Set a flag if it succeeds in reading any data, and then if the decoder errors we can check that and return Ok(0)
if it's still false
.
I will cut a patch release that contains this fix
Edit: Actually, let's wait for #256 to get merged first.
Fixed in xh v0.16.1