flowpipe: A Go repository from Turbot

Workflow for DevOps

Flowpipe enables automation and workflow to connect your clouds to the people, systems and data that matter.

Pipelines. A pipeline is a sequence of steps to do work.

Steps. A step can make an HTTP call, gather human input, send a message, run a query, or run a pipeline.

Triggers. A trigger runs a pipeline when an event occurs, via a webhook, query, or schedule.

Code, not clicks. Our pipelines are code: version-controlled, composable, shareable, easy to edit — designed for the way you work.

Demo time!

Watch on YouTube →

Documentation

See the documentation for:

Install Flowpipe

Install Flowpipe from the downloads page:

# MacOS
brew install turbot/tap/flowpipe

# Linux or Windows (WSL)
sudo /bin/sh -c "$(curl -fsSL https://flowpipe.io/install/flowpipe.sh)"

Now, create and run your first pipeline →.

Flowpipe mods: libraries and samples

Flowpipe library mods are available for services including AWS, Azure, GCP, GitHub, Jira, Okta, PagerDuty, SendGrid, Slack, Teams, Zendesk ... and many more!

Check out Flowpipe samples for ready-to-run samples that use various library mods.

Developing

If you want to help develop the Flowpipe binary, these are the steps to build it.

Clone

Clone github.com/flowpipe and github.com/turbot/pipe-fittings.

git clone git@github.com:turbot/flowpipe
git clone git@github.com:turbot/pipe-fittings

Build

cd flowpipe
make

The Flowpipe binary lands in the current directory.

Check the install

./flowpipe --version

./flowpipe --help

Try it!

./flowpipe pipeline list --mod-location ./internal/es/estest/test_suite_mod/

MOD                   NAME                                                                                                        DESCRIPTION
mod.mod_depend_a      mod_depend_a.pipeline.echo_one_depend_a
mod.test_suite_mod    test_suite_mod.pipeline.any_param
mod.test_suite_mod    test_suite_mod.pipeline.bad_email_with_expr
mod.test_suite_mod    test_suite_mod.pipeline.bad_http_ignored                                                                    Ignored bad HTTP step.
mod.test_suite_mod    test_suite_mod.pipeline.bad_http_not_ignored                                                                Pipeline with a HTTP step that will fail. Error is not ignored.
</snip>

Now run a simple pipeline:

./flowpipe pipeline run --mod-location ./internal/es/estest/test_suite_mod/ simple

[flowpipe] Execution ID: exec_clsm62ko47mjp5f74730
[simple] Starting pipeline
[simple.echo] Starting transform
[simple.echo] Output echo_1 = echo 1
[simple.echo] Output echo_2 = echo 2
[simple.echo] Complete 2ms
[simple] Output val = Hello World
[simple] Complete 12ms exec_clsm62ko47mjp5f74730

DevContainer

There are other third party tools that are required for the full suite that are not required for initial development tasks. We have built a DevContainer that has all the required tools installed.

Install Docker
Install VS Code
Pull the Dev Container: docker pull ghcr.io/turbot/flowpipe-devcontainer:latest
In VS Code install devcontainer extension.
Open flowpipe in Dev Containers: Open Folder in Container... option.
Run make to build the Flowpipe binary.

Flowpipe DevContainer bundles the following:

If you're interested in developing Flowpipe mods, see our documentation for mod developers.

Turbot Pipes

Coming soon: bring your team to Turbot Pipes to use Flowpipe together in the cloud.

Open source and contributing

This repository is published under the AGPL 3.0 license. Please see our code of conduct. Contributors must sign our Contributor License Agreement as part of their first pull request. We look forward to collaborating with you!

Flowpipe is a product produced from this open source software, exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get involved

Join #flowpipe on Slack →