Sometimes an invalid content length error is produced

Question

Sometimes an invalid content length error is produced

blaubaer opened this issue 8 years ago · 14 comments

blaubaer commented 8 years ago

If...

caddy returns a content-length header
The modified content-length is different from the original one
gzip plugin is not used

...browsers will reject the usage of the responded content because of the mismatching header value.

Answer 1 · 2017-01-06T09:21:24.000Z

Caddyfile:

:2015 {
  tls off
  filter rule {
    content_type .*
    search_pattern a
    replacement bb
  }
  proxy / :2016
}

:2016 {
  tls off
}

text.txt:

Results:

$ curl -i http://localhost:2015/test.txt
HTTP/1.1 200 OK
Content-Length: 2
Content-Type: text/plain; charset=utf-8
Date: Fri, 06 Jan 2017 09:20:04 GMT
Last-Modified: Fri, 06 Jan 2017 09:18:23 GMT
Server: Caddy

curl: (18) transfer closed with 2 bytes remaining to read

Adding { header_downstream Content-Length 3 } to the proxy directive allows the curl to succeed and output the correct results (i.e. bb).

Answer 2 · 2017-01-25T08:37:36.000Z

You are right. I can reproduce this with the current version of caddy again. I will try to find a fix for this.

A workaround is to activate gzip for the same host configuration like this that uses filterdirective because this also fixes the content-length size.

Answer 3 · 2017-01-25T09:23:10.000Z

Mhh... This only does not work with the packaged version of caddy if I download it from the page. If I build this fully on my disk there is no problem and all works. I created a thread in caddy forum to acquire more information about this.

Answer 4 · 2017-01-25T09:36:15.000Z

gzip isn't an option if you're sending content downstream to another reverse proxy. I'm currently working around this by unsetting the Content-Length header and letting Varnish re-count the bytes downstream.

Answer 5 · 2017-01-25T09:37:19.000Z

And I was able to reproduce it under Caddy 0.9.5 built from source with Go 1.8rc2.

Answer 6 · 2017-01-25T14:54:59.000Z

Here is the way how to reproduce my way, could you please recheck it?

Start and enter a new and clean docker container:

docker run --rm -ti -p 2020:2020 golang

Prepare and start Caddy with caddy-filter:

# Download caddy sources with all dependencies
go get -u github.com/mholt/caddy/caddy
# Download caddy-filter with all dependencies
go get -u github.com/echocat/caddy-filter

# Configure caddy to use caddy-filter
sed -i 's/\t\/\/ This is where other plugins get plugged in (imported)/\t_ "github.com\/echocat\/caddy-filter"/' $GOPATH/src/github.com/mholt/caddy/caddy/caddymain/run.go

# Add Caddyfile
cat << EOF > /tmp/Caddyfile
localhost:2020
root /tmp
markdown
filter rule {
  content_type text/.*
  search_pattern "Replace me!"
  replacement "OK!"
}
EOF

# Add example index file
cat << EOF > /tmp/index.md
<html>
<head><title>Replace me!</title></head>
<body><p>Replace me!</p></body>
</html>
EOF

# Build caddy
go build -o $GOPATH/bin/caddy github.com/mholt/caddy/caddy

# Run caddy
caddy -conf /tmp/Caddyfile

Check result:
Open http://localhost:2020 and you should see the text OK! in your browser. If you see Replace me! or an error the latest fixes was not applied to this build.

Answer 7 · 2017-01-25T16:35:39.000Z

You're missing a critical piece: you need a proxy directive. The issue only occurs when you are trying to modify an upstream response. This modified Caddyfile reproduces the error:

localhost:2020 {
  filter rule {
    content_type text/.*
    search_pattern "Replace me!"
    replacement "OK!"
  }

  proxy / localhost:2021
}

localhost:2021 {
  root /tmp
  markdown
}

$ curl -i localhost:2020
HTTP/1.1 200 OK
Content-Length: 212
Content-Type: text/html; charset=utf-8
Date: Wed, 25 Jan 2017 16:32:43 GMT
Last-Modified: Wed, 25 Jan 2017 16:32:39 GMT
Server: Caddy

<!DOCTYPE html>
<html>
	<head>
		<title>index</title>
		<meta charset="utf-8">
		
	</head>
	<body>
		<p><html>
<head><title>OK!</title></head>
<body><p>OK!</p></body>
</html></p>

	</body>
curl: (18) transfer closed with 16 bytes remaining to read
</html>

Answer 8 · 2017-01-25T16:44:20.000Z

Ok understand but we are speaking here about to different problems. Also markdown does not work before and if I download the current version from the site it still does not work with the caddyfile mentioned above. The problem with the proxy is a new problem which I have to look at it.

The handling pipeline of Caddy is quite special 😉

Answer 9 · 2017-01-25T16:46:14.000Z

When I reported the issue here, you told me to move it here. 😄 Would you like me to open a new issue?

Answer 10 · 2017-01-25T16:47:55.000Z

Sorry for that. But I'am a littlebit confused that my test scenario also does not work with the latest caddy version (but it should be) - but again: Yes I will check this proxy problem 😄

Answer 11 · 2017-01-25T16:50:02.000Z

No worries! Thank you for all your help. This plugin is solving a huge problem for my Caddy use case so let me know if I can help troubleshoot the proxy issue any further.

Answer 12 · 2017-01-27T15:46:37.000Z

So, now I can reproduce the problem reported by you and: It is really strange - I need some time to debug the whole procedure... 😕

Answer 13 · 2017-01-27T19:07:30.000Z

So, now I fixed it also for proxy and added integration tests for static, markdown and proxy. I will close this issue if I can guarantee that the download page of caddy serves this version.

Answer 14 · 2017-01-27T19:19:33.000Z

Can confirm that this fixes the issue in my environment. Thank you!