"context canceled" error when using with http.ReverseProxy

[fengye87]

Hi, I'm having this weird problem when using yamux with http.ReverseProxy. The thing I want to achieve is:

1. A server listening on :8888 for client to connect
2. When a client is connected, the connection is handed over to yamux, and
   • the client becomes the real server, which is just a simple proxy to example.org
   • the server starts listening on :8080, which would proxy to the client

The weird part is that the first curl localhost:8080 would success (with 404 from example.org), but then all the following curl localhost:8080 would return 502 bad gateway, with the client prints out http: proxy error: context canceled with each request sent.

Here's some clue:

1. The error message is from the http roundtripper, where it would error out if ctx is done
2. If I avoid yamux, and do server -> client -> example.org proxy directly with http.ReverseProxy, the problem would just go away
3. If I set DisableKeepAlives: true to server's transport, the problem would go away too because each request would Dial a new connection then

Below is the detailed code and steps to reproduce the problem:

server code:

package main

import (
	"log"
	"net"
	"net/http"
	"net/http/httputil"
	"net/url"
	"time"

	"github.com/hashicorp/yamux"
)

func main() {
	lis, err := net.Listen("tcp", ":8888")
	if err != nil {
		panic(err)
	}

	conn, err := lis.Accept()
	if err != nil {
		panic(err)
	}

	sess, err := yamux.Server(conn, nil)
	if err != nil {
		panic(err)
	}

	go func() {
		u, err := url.Parse("http://whatever")
		if err != nil {
			panic(err)
		}

		rp := httputil.NewSingleHostReverseProxy(u)
		rp.Transport = &http.Transport{
			Dial: func(network string, addr string) (net.Conn, error) {
				return sess.Open()
			},
		}

		log.Println("serving at :8080")
		if err := http.ListenAndServe(":8080", rp); err != nil {
			panic(err)
		}
	}()

	for {
		time.Sleep(time.Second)
	}
}

client code:

package main

import (
	"log"
	"net"
	"net/http"
	"net/http/httputil"
	"net/url"

	"github.com/hashicorp/yamux"
)

func main() {
	conn, err := net.Dial("tcp", "localhost:8888")
	if err != nil {
		panic(err)
	}

	sess, err := yamux.Client(conn, nil)
	if err != nil {
		panic(err)
	}

	u, err := url.Parse("http://example.org")
	if err != nil {
		panic(err)
	}

	log.Println("serving")
	if err := http.Serve(sess, httputil.NewSingleHostReverseProxy(u)); err != nil {
		panic(err)
	}
}

And do below in terminal to see the problem:

curl localhost:8080 // success
curl localhost:8080 // 502 bad gateway