Prefect (https://docs.prefect.io/) Task Flow paradigm base code for data science research
pip install -r requirements.txt
pip install -r requirements-dev.txt
Using the caching
export PREFECT__FLOWS__CHECKPOINTING=true
Example test:
ENV_FOR_DYNACONF=test nosetests tests/tasks/sample_task/sample_task_test.py
Running the example flow:
python -m sample.flows.sample_flow