Windmill

Drag'n'drop web app to manage and create Airflow DAGs. DAGs are described usinga JSON "wml" file, which can be transpiled into a Python DAG file and pushed to a configured git repository.

Front end is built using React on Typescript
Back end is built using Flask on Python 3.6+

Getting Started

python3 -V # 3.6 and 3.7 should both be fine
If on *nix, sudo apt-get install python3-venv
python3 -m venv venv
source venv/bin/activate
Install with pip install 'airflow-windmill'
1. Airflow is expected to be installed on the system. This allows Windmill to run with arbitrary versions of Airflow
2. Otherwise it can be packaged with windmill using pip install 'airflow-windmill[airflow]'. The version is defined in pyproject.toml
Run windmill init to create a local Windmill project
cd windmill-project
Run windmill run from this folder to run the app locally
Navigate to 127.0.0.1:8000

MVP

For progress on MVP see https://github.com/mayansalama/windmill/projects/1

Dev User Guide

To run as a dev:

Clone from git
Run curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
Run source $HOME/.poetry/env
Activate your python virtualenv of choice
Run poetry install -E airflow
Run npm --prefix ./windmill/http/app install ./windmill/http/app
Run windmill-dev start-frontend
Run windmill-dev start-backend
Open 127.0.0.1:1234

Run Python Tests

Expects Tox and Poetry to be available on path

pyenv install 3.6.5 3.7.7
pyenv local 3.6.5 3.7.7
tox

Future Usage Patterns

Auto-sync for windmill project to git

Deployment

Deployment to PyPi is managed using Travis and should be done in the following steps:

Run poetry version {patch|minor|major}
Increment the version number in windmill/__init__.py
Commit and merge code into the master branch
Ensure that the travis build is green
Create a git tag for the new build
Push the tag to origin