We've rebuilt data engineering for the data science era.
Prefect is a new workflow management system, designed for modern infrastructure and powered by the open-source Prefect Core workflow engine. Users organize Tasks into Flows, and Prefect takes care of the rest.
Read the docs; get the code; ask us anything!
from prefect import task, Flow
@task
def say_hello():
print("Hello, world!")
with Flow("My First Flow") as flow:
say_hello()
flow.run() # "Hello, world!"Pefect's documentation -- including concepts, tutorials, and a full API reference -- is always available at docs.prefect.io.
Documentation for developers is also available, covering code style, documentation, and testing.
From the Latin praefectus, meaning "one who is in charge", a prefect is an official who oversees a domain and makes sure that the rules are followed. Similarly, Prefect is responsible for making sure that workflows execute properly.
It also happens to be the name of a roving researcher for that wholly remarkable book, The Hitchhiker's Guide to the Galaxy.
Prefect requires Python 3.5+.
pip install prefectgit clone https://github.com/PrefectHQ/prefect.git
pip install ./prefectPrefect is licensed under the Apache Software License version 2.0.