Runflow is a tool to define and run workflows. To find out more about Runflow, check out runflow.org.
Runflow supports Python (3.7, 3.8, 3.9). The main audiences are devops, data scientists and hackers.
Runflow is in alpha release stage.
Runflow is not job scheduler. It does not schedule the job runs for you. However, you can combine Runflow with some existing scheduling solutions to run your job periodically, such as APScheduler, crontab, etc.
Runflow is not a job worker. It does not watch workloads from somewhere such as job queue. When the job run is complete, the program exits. Nonetheless, it's quite easy to integrate Runflow with some existing worker solutions, such as Celery, Python-RQ, etc.
- Simple: We want you feel simple when developing and running workflows. No hazzle.
- Flexible: We want it integrate to many existing solutions to broaden its use cases.
Runflow provides building blocks for defining tasks and workflows, failure recovering, command-line interfaces, etc.
Typically, you will
- Define your workflows in HCL2 syntax.
- Run your workflows using a simple command
runflow run
.
You can choose whatever technology stack you’re familiar with and glue them using Runflow.
Alternatives: Airflow, Prefect, Oozie, Azkaban.
First, prepare a Python environment:
$ python3 -mvenv env
$ source venv/bin/activate
Run pip install runflow
to install the latest version from PyPI.
$ pip install runflow
To install the HEAD, run pip install git+https://github.com/soasme/runflow.git
.
Next, write a flow spec. Let’s create a file "example.hcl":
$ mkdir myrunflow
$ cd myrunflow
$ vi example.hcl
# File: example.hcl
flow "example" {
variable "content" {
default = "Hello World!"
}
task "bash_run" "echo" {
command = "echo ${var.content}"
}
}
At last, let’s run it.
$ runflow run example.hcl
Hello World!
$ runflow run example.hcl --var="content=Hello Runflow!"
Hello Runflow!
From this point, you have run a minimal example using Runflow.
For more information, please head to the full documentation.
To run all of the test cases, please run: make lint test
.
Please report an issue at: https://github.com/soasme/runflow/issues.