flynn-archive/strowger

server read err: unexpected EOF

Closed this issue · 3 comments

jayd3e commented

Randomly, I'm having requests return a "Bad Gateway" error. It's due to the following error getting thrown by strowger:

server read err: unexpected EOF

Error is thrown here.

jayd3e commented

After a lengthy discussion today, we found that this was happening due to the fact that strowger tries to keep 1:1 mapping of connections to backends. This is idealistic, b/c other servers can potentially close the connections without telling strowger it's doing so. The best way to solve this seems to use Peek(1) to determine if the connection is active or not. Look here for possible implementation. Going to take a crack at this now.

miracle2k commented

Just ran into this as well, with the rethinkdb ui - it simply doesn't do keep-alive, and always closes the connection after a request.

Supporting HTTP pipelining with such backends in existence is also interesting:

  • Distributing pipelined requests to multiple backend connections would be the easiest solution.
  • Otherwise we have to count the number of responses from a backend connection, and then resend the remaining requests to a different backend.
  • If the backend is single-threaded (I remember the Heroku intelligent routing incident - does Rails still have this problem?), or does not support pipelining itself, then it would actually be preferable not to send pipelined requests over the same backend connection.