Purpose of this repo is to test airflow with papermill and docker integration (and maybe kubernetes in the future)
Some files:
Dockerfile
describes simple image with jupter server and papermilldocker-compose.yml
contains build instructions, run$ docker-compose build
to build the imagerun_pm_from_docker.sh
then runs the papermill built imagerun_pm_local.sh
run papermill locally
Directories:
out/
- dir for papermill to output notebooks; contains some exampleshome/
- airflow home, contains dagsnotebooks/
- dir with input notebooks for papermill