vaerohq/vaero

Complete log pipeline solution in Python. Runs fast in Go.

View the documentation and quickstart guide online for usage information.

What is Vaero?

Vaero is a programmable high-performance data pipeline for log data. Collect, transform, and route your log data from any log source to any destination. Vaero helps you save on log storage costs, send data to different systems of analysis, normalize data to different formats, enrich data with more information, and control logs from all your vendors in one place.

Principles

• Easy to program - Code your log pipelines in Python. No need to learn a config file format or new language.
• Fast - Log pipelines are executed entirely in Go, with performance in mind.
• Building blocks - Built-in transforms and source and destination integrations, so you don't have to reinvent the wheel.

Use Cases

• Reduce log storage costs by filtering and routing logs
• Centralize log management instead of configuring multiple sources
• Adjust one system when you change vendors, not all your sources
• Normalize log data between systems
• Mask PII before storage
• Improve overall log ETL performance

Code Log Pipelines in Python

from vaero.stream import Vaero

result = Vaero().source("http_server", port = 8080, endpoint = "/log") \
        .rename("hostname", "host") \
        .add("newfield", "Hello, world!") \
        .sink("s3")

Vaero.start()

Vaero's Python syntax is modern and easy to use. Log pipelines are executed with high-performance Go code.

Documentation

Getting Started

• Installation
• Quickstart

Pipelines

• Creating Pipelines
• More Examples

Deployment

• Deploying Vaero

Adminstration

• CLI

Reference

• Sources - HTTP endpoint, Okta, and more.
• Transforms - add, delete, rename, and more.
• Routes
• Sinks - S3, Stdout, and more.
• Modifiers - Options, Option File, Secrets Manager, and more.
• Path Syntax

Configuration

• Global Settings

---

Built and supported by Vaero