stream-processing
There are 999 repositories under stream-processing topic.
google-ai-edge/mediapipe
Cross-platform, customizable ML solutions for live and streaming media.
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
vectordotdev/vector
A high-performance observability data pipeline.
zhisheng17/flink-learning
flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》
oxnr/awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
redpanda-data/redpanda
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
madd86/awesome-system-design
A curated list of awesome System Design (A.K.A. Distributed Systems) resources.
redpanda-data/connect
Fancy stream processing made operationally mundane
ThreeDotsLabs/watermill
Building event-driven applications the easy way in Go.
risingwavelabs/risingwave
Stream processing and management platform.
robinhood/faust
Python Stream Processing
fluent/fluent-bit
Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows
hazelcast/hazelcast
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
MaterializeInc/materialize
Real-time Data Integration and Transformation: use SQL to transform, deliver, and act on fast-changing data.
apache/hudi
Upserts, Deletes And Incremental Processing on Big Data.
online-ml/river
🌊 Online machine learning in Python
javascriptdata/danfojs
Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.
infinyon/fluvio
Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.
ArroyoSystems/arroyo
Distributed stream processing engine in Rust
airtai/faststream
FastStream is a powerful and easy-to-use Python framework for building asynchronous services interacting with event streams such as Apache Kafka, RabbitMQ, NATS and Redis.
manuzhang/awesome-streaming
a curated list of awesome streaming frameworks, applications, etc
memgraph/memgraph
Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.
douban/dpark
Python clone of Spark, a MapReduce alike framework in Python
pipelinedb/pipelinedb
High-performance time-series aggregation for PostgreSQL
PeerDB-io/peerdb
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
francoispqt/gojay
high performance JSON encoder/decoder with stream API for Golang
reugn/go-streams
A lightweight stream processing library for Go
numaproj/numaflow
Kubernetes-native platform to run massively parallel data/streaming jobs
yomorun/yomo
🦖 Stateful Serverless Framework for Geo-distributed Edge AI Infra. with function calling support, write once, run on any model.
timeplus-io/proton
High-performance, low-footprint SQL database written in C++. Process millions of rows per second from Kafka/Pulsar, Iceberg, or ClickHouse, and seamlessly write results back. Supports powerful features like JOIN, CDC, UPSERT, and LOOKUP, enabling real-time analytics and ETL at scale.
bytewax/bytewax
Python Stream Processing
nerevu/riko
A Python stream processing engine modeled after Yahoo! Pipes
siddhi-io/siddhi
Stream Processing and Complex Event Processing Engine
WallarooLabs/wally
Distributed Stream Processing
quixio/quix-streams
Python Streaming DataFrames for Kafka
halaxa/json-machine
Efficient, easy-to-use, and fast PHP JSON stream parser