👩💻 Documentation Request: Details on Serverless Containers cold start process and image versionning
Closed this issue · 3 comments
Summary
Both questions are related:
- Does the container have to pull the image at each cold-start?
- If so, what happens for the image tagged
latest?
Why is it needed?
Question 1
Regarding this question, I found some information on the product page through the following image:
https://www-uploads.scaleway.com/Serverless_Caa_S_Schema_1040px_Dark_c3821db246.webp
I believe this information should be in the documention as well.
Question 2
What happens when the selected tag is latest. Does it pulls the latest image at the time of the cold start? Or does it pull the image that was tagged latest when the container was deployed (which may or may not be the latest image at the time of the cold start) ? In other words, do I need to repeploy containes each time I push a new latest image?
Note
I think some pointers on optimizing cold start duration would be welcomed.
Want to write this documentation yourself?
No
Related PR(s)
No response
(Optional) Scaleway Organization ID
No response
Email address
No response
Hello @durierem, Sorry for the late reply! Thank you for your feedback, this issue is under technical review. We will keep you updated, meanwhile, you can ask your questions directly in our Community Slack if you haven't already.
Best,
Hi @durierem, @SamyOubouaziz is right, community is a better place to ask product questions, as this repo is purely about the docs themselves.
Does the container have to pull the image at each cold-start?
It depends. The system will schedule the container to run on a node in one of our clusters. If the container has run on that node recently, and the image is still cached, it will not need to be pulled. However, if the scheduler picks a different node, or the image is no longer cached, it will need to pull the image. We use Kubernetes under the hood, so if you are interested in the details, you can look up how Kubernetes schedules pods on nodes.
If so, what happens for the image tagged
latest?
The system will always check whether there is a more recent version of the tag, so even if you are pushing to latest, and the node has a cached version of the image, it will still check. In general I would advise against reusing tags, and instead version your images on each build.
I believe this information should be in the documention as well.
This is a sketch of how our internal architecture works, and is something we reserve the right to change. As a result, I don't personally think it should be included in the docs. Can you specify what information you find useful from this diagram? It's interesting to know about the architecture, but I'm focusing more on how users will change their behaviour or use of the service as a result.
I think some pointers on optimizing cold start duration would be welcomed.
I understand this, but it's essentially the same advice as a container running in K8s anywhere:
- Reduce image size as much as possible
- Make any container instantiation asynchronous and not on the critical path
From your questions, can I assume that you're experiencing issues with cold starts? If so, a question in the Community Slack is the way to go.
Thanks for the replies 👍
In general I would advise against reusing tags, and instead version your images on each build.
I generally agree with that but my use case here is precisely "run the latest version". I don't want to manually select a version and redeploy the container each time hence my question about latest.
Can you specify what information you find useful from this diagram?
The "incoming request"to "Knative" to "pull and deploy" made me assume the image is pulled each time there's a request that triggers a cold start. I had been looking for this exact information on the documentation but didn't find anything useful in that regard. The only way to check this is actually conducting a test by pushing a new latest and playing around (which is not practical).
If the container has run on that node recently, and the image is still cached, it will not need to be pulled. However, if the scheduler picks a different node, or the image is no longer cached, it will need to pull the image.
The system will always check whether there is a more recent version of the tag, so even if you are pushing to latest, and the node has a cached version of the image, it will still check.
Thanks for the technical insights 👍 I believe knowing when the system checks for a newer version (at every request? at cold-start only?) is an information worth displaying on the doc. I understand that it may be seen as an internal detail but it's the kind that can lead to questions like mine. Maybe a section "What happens when I push a new image version to the registry" or something similar?
From your questions, can I assume that you're experiencing issues with cold starts? If so, a question in the Community Slack is the way to go.
The real issue for me was knowing what can I optimize on my side of things and what are the intrisic limitations of the Serveless Containers platform that all my image optimization techniques won't address.