openanalytics/shinyproxy

Container restart policy

Opened this issue · 1 comments

I have been implementing the pre-initialized containers. I feel like the application.yml needs a container restart policy field. It should be something like interval (hourly) or time once per night at 1:00 AM. Ideally, ShinyProxy would spin up new containers in the background and swap out the seats gracefully.

ShinyProxy 3.1.1 has a delete method in the restAPI that will delete containers. I was unable to use this with my authentication method. It appears to be geared towards, manual restarts. Maybe, it isn't understood, why this is necessary. In general R/Shiny are not stateful programs that gracefully handle long running tasks. R is a functional language and by definition a memory leak. Functional languages don't handle threading, so there is no good way to do background updates for example. Even Shiny only mimics threading by using CPU parallelization and interleaving. These programs need to be killed and restarted frequently.

I have been able to rig something up with a docker compose stack, that deletes the app services, stops the app containers, then stops the ShinyProxy container which gets immediately restarted, and it then relaunches the preinitialized containers.

I feel like there should be a better way to handle this preferably setting the restart policy for a container. While this works for my situation, I could imagine a situation where I may need to restart an app container more or less frequently, in addition to the other containers. In scraping the help requests, there are many users with this need.

I hope this clears things up a little. I do want to say that the container restarts/dying are much better and more robust with 3.1.1 though.

Hi, first of all thank you for your extensive testing and feedback on the pre-initialization feature. Although we can't cover every use-case with ShinyProxy, it's good to know how it's being used and what issues users are facing.

I understand that R apps should not run for an infinite amount, and indeed the pre-initilization feature must make a trade-off between performance and the inherent limitations of R. I created an internal ticket for a feature to restart the containers on a schedule, we will probably include this in the next release.

Ideally, ShinyProxy would spin up new containers in the background and swap out the seats gracefully.

I would also prefer to implement this in this way, but it's not trivial. The current behavior for restarting all containers is:

  1. mark all containers as to be removed -> no new users can use seats of the old containers
  2. launch new containers such that there are again enough seats
  3. the old containers are removed when users stop using the container

There are two main issues with this:

A. there is a moment where there are less than the minimum number of seats
B. the machine or cluster running ShinyProxy needs to support the additional containers spin-up during step 2

however, the main advantages is:

C. as soon as the new version is deployed, users cannot claim seats on the old containers. This is important when deploying bug fixes.

It's impossible implementing a restart where all A, B and C are respected, therefore up to now we prioritized property C.
For a scheduled restart property C is less important, so we could implement a restart mechanism where ShinyProxy restarts one (or more) container at a time, achieving properties A and B.