Kubernetes preset
sh0rez opened this issue · 6 comments
It would be incredibly useful to have a Kubernetes preset, that spins up an empty cluster (maybe https://k3s.io/) and gives code access to it in some sort (https://github.com/kubernetes/client-go)?
Hi @sh0rez, great idea!
What would be your use case if there was such preset?
Would a single node cluster be enough with API access only?
Which options would be useful?
At this point I'm not even sure it would be possible to build such preset, but we definitely should gather requirement before getting to work 😼
Hey @orlangure!
As a use-case, I am primarily looking at automatically testing tools like Grafana Tanka, which does application definition for Kubernetes. It uses a special language (Jsonnet) to define k8s objects and calls kubectl to apply afterwards.
I'd like to do some basic testing in CI, but many operations require a Kubernetes cluster, which is rather hard to get. It would be incredibly helpful if my tests would just spin one up using docker, run my tests against it and tear it all down again.
A single-node cluster with basic api-access would be all I need.
Does this sound like something worth trying?
@sh0rez,
Thanks for clarification, that's exactly what I thought your use case was.
I already made some progress in that direction, and now there is a docker image that I'll use in this new preset (https://github.com/orlangure/k3s-dind available at https://hub.docker.com/repository/docker/orlangure/k3s). it features a single node k3s cluster running in a container.
There will be a Config function on the new preset that returns *rest.Config of kubernetes go client, or ConfigBytes that returns a raw kubeconfig file content to use by the clients that don't want to use *rest.Config.
I am now testing the implementation and will soon start working on all the wrappers to make this preset available over HTTP as well and not only from Go, so it'll take a bit more time.
In the mean time, would you like to check out #67? It would be nice to get some early feedback before I finish working on it and publish a new release.
I prefer charts + Dockerfile to deploy any a preset as a pod in k8s. A k8s cluster is out of the scope of this test framework.
@dreamofboy,
We are not talking about deploying stuff into Kubernetes clusters, we want to have a lightweight cluster for tests, mostly for people that develop automation tools around Kubernetes. Say if you have a custom tool that triggers alerts based on pod statuses, you can test this tool using k3s preset.