valyala/ybc

My reverse proxy implementation using ybc

sajal opened this issue · 4 comments

sajal commented

Just wanted you to know as a side project im trying to build another reverse proxy inspired by cdn-booster

https://github.com/sajal/gohttpcache/blob/master/proxy.go

Im doing it just for fun... Its a very very rough draft at the moment and probably does not perform too well.

Some things I do differently.

  1. Cache all response headers and pass them to client, not just content-type
  2. Obey Vary directives. This has performance implications since now each request must do 2 lookups. 1 to check for Vary directives registered by a particular url, and another for the actual object.
  3. Somewhat process cache control headers, enforcing a min ttl.
  4. Allow multiple hostname/origin pairs.
valyala commented

On Sun, Aug 3, 2014 at 8:45 PM, Sajal Kayan notifications@github.com
wrote:

Just wanted you to know as a side project im trying to build another
reverse proxy inspired by cdn-booster

https://github.com/sajal/gohttpcache/blob/master/proxy.go

Nice project. It looks like generic caching http proxy. I also tried
creating such a proxy, but stuck with http 1.1 weirdness
http://www.w3.org/Protocols/rfc2616/rfc2616.html. See below my comments.

Im doing it just for fun... Its a very very rough draft at the moment and
probably does not perform too well.

Some things I do differently.

  1. Cache all response headers and pass them to client, not just
    content-type

Note that according to http, certain headers shouldn't be proxied by
conforming http proxies. I didn't remember details, so just read http/1.1
rfc for details :)

  1. Obey Vary directives. This has performance implications since now each
    request must do 2 lookups. 1 to check for Vary directives registered by a
    particular url, and another for the actual object.

This shouldn't be a problem, since the proxy performance will be limited by
network IO. Ybc can handle up to a million lookups per second, so you can
freely issue multiple ybc.Get() requests per each incoming http request
without worrying about its' performance.

  1. Somewhat process cache control headers, enforcing a min ttl.

Again, look into the corresponding http specification
http://www.w3.org/Protocols/rfc2616/rfc2616-sec13.html#sec13.

  1. Allow multiple hostname/origin pairs.

Nice.

I'd recommend also implementing Content-Range support
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.16, which
may be useful when dealing with big objects such as video files. Big
objects also require modified caching algorithm - they must be cached in
chunks with 100Kb-1Mb sizes. Each cached chunk should have a key containing
original_object_url and offset. This should reduce latency when querying
for multi-GB files.

Good luck implementing such a project - it have all chances to become Squid
killer :)

—
Reply to this email directly or view it on GitHub
#7.

Best Regards,

Aliaksandr

sajal commented

Hi,
Thanks a lot for your comments. Im basically trying to write a reverse proxy, ones that a CDN might use, and trying to add in all the features that I think a cdn must have.. and exploring the complexities involved.
Im using http://tools.ietf.org/html/rfc7234 as guideline for the caching logic.
Content-Range is high up on my list. some other things on my list are

  1. Customizable caching logic
  2. Runtime reconfiguration - i.e. reconfigure the settings without downtime.
  3. Purging

Big objects also require modified caching algorithm - they must be cached in chunks with 100Kb-1Mb sizes. Each cached chunk should have a key containing original_object_url and offset. This should reduce latency when querying for multi-GB files.

Is that for optimizing cache(ybc) latency? Or are you reffering to http latency? I am worried about such chinking because assume i store a 1 GB file in 1000 1MB chunks, and chunk 999 gets evicted. Then when I want to serve the full 1 GB file, and notice 1 chunk is missing, I must make a request to the origin, now if the origin does not support range-request, or if the object has changed, i would need to replace all the chunks. Perhaps ill worry about multi-GB objects later.

By the way since i last wrote to you, ive changed the proxy to be modularized.

example server: https://github.com/sajal/gohttpcache/blob/master/proxyserver.go
The actual proxy : https://github.com/sajal/gohttpcache/blob/master/proxy/proxy.go

Currently the user can supply their own request to cachekey logic.

valyala commented

Is that for optimizing cache(ybc) latency? Or are you reffering to http latency?

I'm referring to http latency. Suppose 1Gb file is requested via the proxy. Without chunk caching you have two options:

  1. Download the whole file from upstream server, store it to the cache and then return to the user. This introduces big latency from the user PoV, since he should wait until the whole file is cached before receiving the first byte in the response. This option also is vulnerable to dogpile effect when multiple users request the same file at the sime time. The proxy should either download the same file multiple times or wait until the file is cached from the first request before serving the file to other users. YBC supports 'dogpile-effect-aware' get() functions (see the docs at https://godoc.org/github.com/valyala/ybc/bindings/go/ybc), so the second approach can be easily implemented, but again, it may introduce high latencies.
  2. Stream the file to the user while downloading it from upstream source. This solves latency problem, but introduces another problem when improperly implemented - since bandwidth between the user and the proxy is usually lower comparing to the bandwidth between the proxy and upstream servers, straightforward blocking loop (download a chunk from upstream, then send it to the user) will be limited by user's bandwidth. This may significantly increase the duration for big files' caching. This also enables the possibility for malicious cache sabotage similar to slowloris - http://en.wikipedia.org/wiki/Slowloris_(software) .
sajal commented

What i had in mind regarding large files and dogpile is...
(Assuming it mostly affects cache misses)

  1. If the request was for a range, 1a : send range request to origin and serve user while launching background task to fetch the full file. 1b: Fetch full file from origin and serve user the requested range.
  2. Somehow keep track of inflight origin requests where headers have been received but the body download is pending. Hold subsequent requests for same object (not too strict check, basically only pass vary, etc) and then somehow copy the body to all requesters. So 100 people making request for 10 GB file at the same time would only cause a single request to origin.
  3. Stream to user(s) while downloading from origin : This is also something I want to do. https://github.com/sajal/gohttpcache/blob/master/proxy/proxy.go#L306 SpyReader is defined in https://github.com/sajal/gohttpcache/blob/master/proxy/utils.go . Its very rough draft at the moment and ive not worked on it for a while, but the goal is as bytes are coming in from the origin, send it to the client as well as cache at the same time while trying my best to not keep anyone waiting. I dont want slow client delaying the object from going into cache ... and also I dont want slow origin delaying starting to send object to client.

Most CDNs are either optimized for large file or for small object.. (or have separate configs/servers for each). The ones that do both in the same config typically don't perform that well. I'm trying to write a server that does small objects(smaller than ~10 MB) well while supporting decent performance/features for large files(tens of MB to couple of GB). Very large files(1GB+) is not high on my list.