jetstack/kube-oidc-proxy

partially full response buffers never flush

mkinsley opened this issue · 3 comments

Expect

Non-full buffers should flush in a reasonable amount of time

Observed

  • Partial buffer fills on the response are not flushed to oidc-proxy client.
  • From kubectl client perspective this will often show up as partial json obj... which will fail to decode.
  • Pod delete will hang until something terminates the streaming connection (ie LB timeout)
  • setting proxyHandler.FlushInterval = -1 resolves the issue

Symptoms

  • kubectl delete pod hangs until api server connection is terminated

Fix
I set the proxy to flush immediately and the issue resolves. I'm happy to submit a PR if you would like. We could flush immediately, pick a sane value, or make it configurable.

// proxy.go: 128
// set up proxy handler using proxy
proxyHandler := httputil.NewSingleHostReverseProxy(url)
proxyHandler.Transport = p
proxyHandler.ErrorHandler = p.Error

// ----------------------------------------------------
// pod delete (anything that calls w/ watch=true ) can hang 
// setting FlushInterval < 0 immediately flushes 
// and solves issue
// ----------------------------------------------------
proxyHandler.FlushInterval = -1

Hi @mkinsley,

It is annoying you are running into this. As you can see from the golang docs, running with a 0 value means no periodic flushing which is the current behavior.

I would like to see this changed to some sane value that can be configured. I am nervous at having a negative value as the default, perhaps 200ms would be better?

I would like more advise here on what the implications of having immediate vs periodic flushing means. In the mean time I'll put a PR together to put things in place before we agree on what the default should be.

/cc @simonswine

/assign @mkinsley

Whoops!

/unassign @mkinsley
/assign

I have opened up this PR to fix this
#137