View on Medium

Prefect + DVC Demo

A demo on how to use Prefect with DVC

Tools Used in This Project

  • DVC: Data version control
  • Prefect: Orchestrate and observe dataflow

Project Structure

  • src: consists of Python scripts
  • config: consists of configuration files
  • data: consists of data
  • notebook: consists of Jupyter Notebooks
  • tests: consists of test files

Set Up the Project

  • Install requirements:
pip install -r requirements.txt

Run the Project

To run all flows, type:

python src/main.py