Consider running scripts as services rather than CGI

[jamesaoverton]

PR #122 improved CGI performance significantly, but I still see an overhead of about one second on every request. The main use case I'm thinking about is running Python Flask apps. When comparing DROID CGI (a bit sluggish) to long-running Flask (very snappy), the contrast is frustrating. So I want to at least think through a couple of alternatives.

The key advantages of CGI are that it's relatively simple, very flexible, and we do not pay ongoing costs for memory usage. The key downside is this overhead. These are big wins, so I expect we'll stick with CGI in the end.

An alternative would be to run the Flask app, keep it running for some amount of time (say 10 minutes), listening on some port. There would be an initial delay similar to what we have to CGI scripts while Flask starts, but we could just forward subsequent requests to that port/process. So we would be using memory for longer but still limited periods.

There would still be overhead from the DROID code, but this would eliminate the Python/Flask startup time for subsequent requests. I'm not sure this is a big enough win. We should probably be optimizing startup time anyway.

This pushes on our design principle of a single process per branch. Other tasks (such as Make and git tasks) would have to interrupt these processes, probably without a confirmation dialogue. I'm not clear on what would need to be done to the Docker configuration. Overall, I expect this would make the DROID code more complicated.

(There would also have to be some way to configure this in the Makefile, which I haven't really considered yet.)

Another alternative that we should at least think about is for DROID to run a FastCGI server, but I don't really know what that would mean.