ducaale/xh

HEAD fails on compressed response

blyxxyz opened this issue · 4 comments

$ xh head httpbin.org/gzip
[...]
xh: error: error decoding gzip response body: failed to fill whole buffer

This didn't happen before #241, so reqwest deals with it correctly. I don't know how.

This comment from seanmonstar/reqwest#83 might be relevant

// libflate does a read_exact([0; 2]), so its impossible to tell
// if the stream was empty, or truly had an UnexpectedEof.
// Therefore, we need to peek a byte to make check for EOF first.

It took me a while to understand, but I think the relevant piece in the current reqwest code is here: https://github.com/seanmonstar/reqwest/blob/2a6e012009fb79065767cb49a8a000d354c47ba6/src/async_impl/decoder.rs#L285

reqwest doesn't wrap a decoder around the stream until it did a poll_peek to make sure that there's actual data to receive. If there's no data then it creates a "fake" empty stream.

So we have the following situations:

  • When xh 0.15 receives an empty gzip response of any kind it turns it into an empty stream, never even constructing a decoder.
  • When xh 0.16 receives an empty gzip response to a GET request, marked with Content-Length: 0, we handle that explicitly by not trying to decode.
  • When xh 0.16 receives an empty gzip HEAD response it doesn't trigger the explicit handling so we try to decode it and fail.

So we need to skip the gzip decoding or ignore the error if the body is totally empty.

Maybe we can fold this into the existing InnerReader trick? Set a flag if it succeeds in reading any data, and then if the decoder errors we can check that and return Ok(0) if it's still false.

I will cut a patch release that contains this fix

Edit: Actually, let's wait for #256 to get merged first.

Fixed in xh v0.16.1