- Reading List for Stream Processing
- CS 591 K1: Data Stream Processing and Analytics, Spring 2021, Brown University
- Streaming Systems: The What, Where, When, and How of Large-Scale Data Processing, O'Reilly Media, Inc., 2018
- Stream Processing with Apache Flink: Fundamentals, Implementation, and Operation of Streaming Applications, O'Reilly Media, Inc., 2019
- A Comprehensive Survey on Parallelization and Elasticity in Stream Processing, ACM Computing Surveys, 2019
- Hardware-Conscious Stream Processing: A Survey, ACM SIGMOD Record, 2020
- Resource Management and Scheduling in Distributed Stream Processing Systems: A Taxonomy, Review, and Future Directions, ACM Computing Surveys, 2020
- A Survey on the Evolution of Stream Processing Systems, arXiv, 2020
- S4: Distributed Stream Computing Platform, ICDMW, 2010, Yahoo
- Discretized Streams: An Efficient and Fault-Tolerant Model for Stream Processing on Large Clusters, HotCloud, 2012
- Naiad: a timely dataflow system, SOSP, 2013, Microsoft
- MillWheel: fault-tolerant stream processing at internet scale, VLDB, 2013, Google
- Storm@twitter, SIGMOD, 2014
- Twitter Heron: Stream Processing at Scale, SIGMOD, 2015, Twitter
- The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing, VLDB, 2015, Google
- Apache Flink™: Stream and Batch Processing in a Single Engine, Bulletin of the TCDE, 2015
- StreamScope: Continuous Reliable Distributed Processing of Big Data Streams, NSDI, 2016, Microsoft
- Samza: stateful scalable stream processing at LinkedIn, VLDB, 2017, LinkedIn
- State management in Apache Flink®: consistent stateful distributed stream processing, VLDB, 2017
- Structured Streaming: A Declarative API for Real-Time Applications in Apache Spark, SIGMOD, 2018
- A Cloud Native Platform for Stateful Streaming, arXiv, 2020, IBM
- Drizzle: Fast and Adaptable Stream Processing at Scale, SOSP, 2017
- Prompt: Dynamic Data-Partitioning for Distributed Micro-batch Stream Processing Systems, SIGMOD, 2020
- ChronoStream: Elastic Stateful Stream Computation in the Cloud, ICDE, 2015
- Stela: Enabling Stream Processing Systems to Scale-in and Scale-out On-demand, IC2E, 2016
- Three steps is all you need: fast, accurate, automatic scaling decisions for distributed streaming dataflows (DS2), OSDI, 2018
- Minimizing cost by reducing scaling operations in distributed stream processing, VLDB, 2019
- Elasticutor: Rapid Elasticity for Realtime Stateful Stream Processing, SIGMOD, 2019
- Turbine: Facebook’s Service Management Platform for Stream Processing, ICDE, 2020, Facebook
- Auto-sizing for Stream Processing Applications at LinkedIn (Sage), HotCloud, 2020, LinkedIn
- Move Fast and Meet Deadlines: Fine-grained Real-time Stream Processing with Cameo, NSDI, 2021
- Elastic Stream Processing with Latency Guarantees (Nephele), ICDCS, 2015
- DRS: Auto-Scaling for Real-Time Stream Analytics, IEEE ACM Trans Netw, 2017
- Integrating scale out and fault tolerance in stream processing using operator state management (SEEP), SIGMOD, 2013
- Megaphone: latency-conscious state migration for distributed streaming dataflows, VLDB, 2019
- In support of workload-aware streaming state management, HotStorage, 2020
- Providing streaming joins as a service at Facebook, VLDB, 2018
- AJoin: ad-hoc stream joins at scale, VLDB, 2019
- Nexmark benchmark suite
- Benchmarking cloud serving systems with YCSB, SoCC, 2010
- Benchmarking Distributed Stream Data Processing Systems, ICDE, 2018
- Theodolite: Scalability Benchmarking of Distributed Stream Processing Engines in Microservice Architectures, Big Data Research, 2021
- TerseCades: Efficient Data Compression in Stream Processing, USENIX ATC, 2018 (compression)
- Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center, NSDI, 2011
- Omega: flexible, scalable schedulers for large compute clusters, EuroSys, 2013, Google
- Apache Hadoop YARN: yet another resource negotiator, SoCC, 2013
- Apollo: Scalable and Coordinated Scheduling for Cloud-Scale Computing, OSDI, 2014, Microsoft
- Large-scale cluster management at Google with Borg, EuroSys, 2015, Google
- Hydra: a federated resource manager for data-center scale analytics, NSDI, 2019, Microsoft
- Twine: A Unified Cluster Management System for Shared Infrastructure, OSDI, 2020, Facebook
- HiveD: Sharing a GPU Cluster for Deep Learning with Guarantees, OSDI, 2020, Microsoft
- Autopilot: workload autoscaling at Google, EuroSys, 2020, Google
- Take it to the limit: peak prediction-driven resource overcommitment in datacenters, EuroSys, 2021, Google
- Optimizing space amplification in RocksDB, CIDR, 2017
- FASTER: A Concurrent Key-Value Store with In-Place Updates, SIGMOD, 2018