Question about Drone Horizontal Scaling in Plane
akihiro-suzuk1 opened this issue · 5 comments
Hello, thank you for your response to my previous question.
We are currently considering running Plane on cloud platforms such as GCP.
I would like to ask about scaling.
Does Plane support horizontal scaling for drones?
From the current documentation, I understand that multiple containers can be launched within a drone. Therefore, to scale, is the only way to upgrade the drone machine's specifications rather than increasing the number of drones?
Hi @akihiro-suzuk1, yes, Plane is built to support multiple drones. Scaling up is simply a matter of starting a new drone and providing it with the same controller. To scale down, you send a "drain" command to the drone you want to spin down, which tells it to not accept any more backends -- then you can wait for the backends already on it to finish, and safely terminate the (virtual) machine.
Thanks for pointing out this deficiency in our documentation, I've opened #726 to add some docs here.
Hi @paulgb, Thank you very much for your prompt response.
You mentioned a deficiency in the documentation, and I have a few additional questions to confirm my understanding:
On the page here, it's stated that:
Unlike the controller, the Plane drone is not a simple stateless web server; it expects to run on a machine (or virtual machine) that exists for the dedicated purpose of being a drone. As such, it’s not compatible with being run in Kubernetes, which abstracts away the underlying machine.
Based on this, I understand that when deploying drones in the cloud, they should be placed on a dedicated cloud resource, such as GAE (GCP) or EC2 (AWS). In this case, vertical scaling is possible, but horizontal scaling can only be achieved by dynamically adding new GAE or EC2 instances.
You mentioned starting a new drone. Does this imply that a drone can be dynamically launched on GCP via the gcloud CLI to start a new GAE instance, or can it also be deployed on services like Cloud Run or Kubernetes?
I would greatly appreciate any clarification you can provide. Thank you!
Based on this, I understand that when deploying drones in the cloud, they should be placed on a dedicated cloud resource, such as GAE (GCP) or EC2 (AWS).
On GCP, you would most likely use Compute Engine Instances. I'm not sure if GAE would work as I haven't used it. On AWS, you are correct that it would be EC2 instances.
In this case, vertical scaling is possible, but horizontal scaling can only be achieved by dynamically adding new GAE or EC2 instances.
Yes, that's also correct. Plane is cloud-agnostic and has no facility for requesting more compute from your cloud provider; it would either be done manually or be auto-scaling code that you would write to handle your needs.
You mentioned starting a new drone. Does this imply that a drone can be dynamically launched on GCP via the gcloud CLI to start a new GAE instance, or can it also be deployed on services like Cloud Run or Kubernetes?
Yes, the former. To scale programmatically, you would start a new GCP Compute Instance and run the Plane Docker image on it in the "drone" configuration.
You can't deploy the drone itself it on Cloud Run or Kubernetes because it needs to be paired with a Docker instance, and neither Cloud Run nor Kubernetes are able to provide that (at least as far as I know). You could run the Plane controller process on one of those, though, just not the drone.
My apologies, I meant GCE instead of GAE.
So, if I understand correctly, using GCE, we can achieve horizontal scaling by dynamically starting up and shutting down GCE instances.
I will continue to explore the use of Plane. Thank you very much for your assistance.
So, if I understand correctly, using GCE, we can achieve horizontal scaling by dynamically starting up and shutting down GCE instances.
Yes, exactly.
Happy to help, I'll close this since it's resolved but feel free to re-open or open another if you have more questions!