A kind data platform on your local machine. 🤗
The objective of this project is to be able to quickly setup a data platform on your local machine via kind. It should also help you get familiar with production-ready setup and tools such as terraform, kubernetes and helm.
The following diagram shows the services currently being used, all running in a kind cluster:
Install the following tools:
My docker setup i.e. docker info
...
CPUs: 10
Total Memory: 15.6GiB
...
If you give the docker daemon less resources, make sure to adjust the timeout parameter in the helm releases as creating them can take up more time.
First create the main infrastructure by running the following command:
task -t infra/Taskfile.yml setupThen create the localstack services, for this you first need to expose localstack first by running the following:
task -t infra/Taskfile.yml expose:localstackand then create the services by running the following:
task -t infra/Taskfile.yml setup:localstack-servicesSee infra for more information about the projects infrastructure.
You can find what is being worked on and what is to do in the project.
You're welcome to join this project and I would be more than happy to have you on board. Feel free to raise any issue you might encounter or open a PR to address it. Check out CI to learn more about how we develop and test code.
If you want to jump right into it, here are..
In case you like the project, just follow me..
In case you don't like the project or have any feedback for me drop me a mail at feluelle@pm.me.