stream-processing
There are 1049 repositories under stream-processing topic.
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
google-ai-edge/mediapipe
Cross-platform, customizable ML solutions for live and streaming media.
vectordotdev/vector
A high-performance observability data pipeline.
zhisheng17/flink-learning
flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
oxnr/awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
redpanda-data/redpanda
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
madd86/awesome-system-design
A curated list of awesome System Design (A.K.A. Distributed Systems) resources.
ThreeDotsLabs/watermill
Building event-driven applications the easy way in Go.
redpanda-data/connect
Fancy stream processing made operationally mundane
risingwavelabs/risingwave
Real-time event streaming platform. Streaming CDC, stream processing, low-latency serving, and Iceberg management.
fluent/fluent-bit
Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows
robinhood/faust
Python Stream Processing
hazelcast/hazelcast
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
MaterializeInc/materialize
Real-time Data Integration and Transformation: use SQL to transform, deliver, and act on fast-changing data.
apache/hudi
Upserts, Deletes And Incremental Processing on Big Data.
online-ml/river
🌊 Online machine learning in Python
infinyon/fluvio
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
javascriptdata/danfojs
Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.
ag2ai/faststream
FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.
ArroyoSystems/arroyo
Distributed stream processing engine in Rust
memgraph/memgraph
Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.
manuzhang/awesome-streaming
a curated list of awesome streaming frameworks, applications, etc
douban/dpark
Python clone of Spark, a MapReduce alike framework in Python
PeerDB-io/peerdb
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
pipelinedb/pipelinedb
High-performance time-series aggregation for PostgreSQL
numaproj/numaflow
Kubernetes-native platform to run massively parallel data/streaming jobs
francoispqt/gojay
high performance JSON encoder/decoder with stream API for Golang
reugn/go-streams
A lightweight stream processing library for Go
timeplus-io/proton
Fastest SQL pipeline engine in a single C++ binary, for stream processing, analytics, observability and AI.
yomorun/yomo
🦖 Serverless AI Agent Framework with Geo-distributed Edge AI Infra.
bytewax/bytewax
Python Stream Processing
nerevu/riko
A Python stream processing engine modeled after Yahoo! Pipes
siddhi-io/siddhi
Stream Processing and Complex Event Processing Engine
WallarooLabs/wally
Distributed Stream Processing
quixio/quix-streams
Python Streaming DataFrames for Kafka
halaxa/json-machine
Efficient, easy-to-use, and fast PHP JSON stream parser