Is it possible to use tollbooth to limit memory usage?

[msaron]

Is it possible to use tollbooth to limit total memory usage by my Go program? I would like to specify the maximum total memory that my go web application can use, such as, say 8GB, and if the memory has reach this limit, to not process the request. Thanks.

[didip]

I propose that you are using a different tool for this. Memory growth can be a symptom of more serious things (beyond request handling). Example of these kind of tools:

• If you are using systemd to daemonize your Go program, you can actually configure maximum RAM consumed. When max RAM is exceeded, systemd will restart your app.

• The same effect can be achieved with Supervisord+Superlance plugin.

[msaron]

I was thinking more in terms of when there is a spike in the number of requests coming in. For example, if the memory specified is 4GB and the app sees that the 4GB limit has been reached, it should stop receiving any more requests and return a Server Not Available or something until the memory usage goes below 4GB. If we use systemd or something similar, they will kill all the current running requests also.

So maybe we can stop processing requests when the memory limit of 4GB has been reached and configure systemd to kill the app when, say, 4.5GB of memory has been reached.

[didip]

This is kind of intriguing...

From where should tollbooth get the memory information? My first hunch is to pull it from runtime package, like this:

import (
        "log"
        "runtime"
)

func main() {
        var mem runtime.MemStats
        runtime.ReadMemStats(&mem)  // Pulling this information is not exactly free of cost.

        log.Println(mem.Alloc)      // Maybe use this one? Limit by allocated bytes?
        log.Println(mem.TotalAlloc)
        log.Println(mem.HeapAlloc)
        log.Println(mem.HeapSys)
}

References

• https://golang.org/pkg/runtime/#MemStats

[msaron]

There is one library out there called oom that solves this issue in a limited way. But it does not allow to restrict memory usage by RSS size to the server as a whole and to each request. In addition they do not have an implementation for Windows. They list the following reasons for limiting memory size:

• leaky GCO libs
• spiky load or memory use
• thundering herd issues
• not enough time to track memory leak
• all of the above

I think this feature can be far more elegantly designed in tollbooth because that is what tollbooth is designed for. The oom library pulls the memory information from the operating system and not from the runtime library.

I am coming from a NodeJS background where I used the hapijs framework developed and used by Walmart to drive their website. One feature it has is that you can set the RSS size; see their api description on the settings here. This is the functionality that I would like tollbooth to have. Here is what the hapijs api documenation says about setting the server or connection load:

• load - process load monitoring where:
  • sampleInterval - the frequency of sampling in milliseconds. Defaults to 0 (no sampling).
• load configuration - load limits configuration where:
  • maxHeapUsedBytes - maximum V8 heap size over which incoming requests are rejected with an HTTP Server Timeout (503) response. Defaults to 0 (no limit).
  • maxRssBytes - maximum process RSS size over which incoming requests are rejected with an HTTP Server Timeout (503) response. Defaults to 0 (no limit).

The maxHeapUsedBytes and maxRssBytes settings can be applied to the server as a whole OR per request OR both. This is incredibly useful and powerful! It can probably be incorporated by using golang context (not sure about this as I have just come into golang from NodeJS)?

If all this can be incorporated into Tollbooth, it would make it incredibly powerful.

[didip]

I see...

Let me think + play around a bit. Reading from /proc/meminfo makes it OS dependent, I prefer to have a pure Go implementation for cross-platform support.

[didip]

I decided to not implement this, I still feel this is out-of-scope of the library.