This is a cookiecutter template made for data engineering projects using python and spark.
TODO
Install the latest Cookiecutter::
pip install cookiecutter
Then generate your project::
cookiecutter https://github.com/victor-lucio/pyspark-etl-cookiecutter.git
- Python - Programming language
- Flake8 - Python linter which checks the source code and also acts as a bug and quality checker.
- Black - Python code auto-formatter. Black reformats entire files in place and also formats the strings to have double-qoutes.
- Pyenv - Version and environment management for developing and testing you project without any headache.
- GitHub Actions - The built-in CI/CD workflow in GitHub, This will run your tests and deploy your application.
- Sphinx - Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Georg Brandl and licensed under the BSD license.