airflow_tutorial
airflow python tutorial & example
해당 repo는 블로그에 기술해 놓은 내용에 사용된 airflow example 코드입니다.
blog
Dags dir
- example_basic_dag.py
- Basic example DAG
- Same as airflow doc : https://airflow.apache.org/docs/apache-airflow/stable/tutorial.html
- Blog description : https://lsjsj92.tistory.com/631
- example_sequence_dag.py
- Sequence DAG example ( dag branch )
- Blog description : https://lsjsj92.tistory.com/632
- example_python_dag.py
- Python basic DAG example
- Blog description : https://lsjsj92.tistory.com/632
- example_ml_dag.py
- Machine Learning basic DAG example
- Run with MLproject dir
- Titanic data & scikit-learn
- Blog description : https://lsjsj92.tistory.com/633
- example_slack_dag.py
- Airflow slack alert example
- Basically the same as example_ml_dag.py code and added slack api
- Blog description : https://lsjsj92.tistory.com/634
Repository file structure
airflow_tutorial
│ README.md
│ .gitignore
│
└───dags
│ │ example_basic_dag.py
│ │ example_sequence_dag.py
│ │ example_python_dag.py
│ │ example_ml_dag.py
│ │ example_slack_dag.py
└───MLproject
│ │ config.py
│ │ dataio.py
│ │ model.py
│ │ preprocess.py
│ │ titanic.py
│ └───data
│ └─────titanic
│ │ │ train.csv
│ │ │ prepro_titanic.csv
└───utils
│ │ slack_alert.py