/pyspark-etl-cookiecutter

This is a cookiecutter template made for data engineering projects using python and spark.

Primary LanguageMakefile

pyspark-etl-cookiecutter

This is a cookiecutter template made for data engineering projects using python and spark.

Features

TODO

Quickstart

Install the latest Cookiecutter::

pip install cookiecutter

Then generate your project::

cookiecutter https://github.com/victor-lucio/pyspark-etl-cookiecutter.git

Built With

  • Python - Programming language
  • Flake8 - Python linter which checks the source code and also acts as a bug and quality checker.
  • Black - Python code auto-formatter. Black reformats entire files in place and also formats the strings to have double-qoutes.
  • Pyenv - Version and environment management for developing and testing you project without any headache.
  • GitHub Actions - The built-in CI/CD workflow in GitHub, This will run your tests and deploy your application.
  • Sphinx - Sphinx is a tool that makes it easy to create intelligent and beautiful documentation, written by Georg Brandl and licensed under the BSD license.