Arcon

Arcon is a library for building state-first streaming applications in Rust.

Project Status

Arcon is in development and should be considered experimental until further notice.

The APIs may break and you should not be running Arcon with important data!

Rust Version

Arcon builds against the latest stable release and the current MSRV is 1.56.1

Roadmap

See the roadmap here

Highlights

Out-of-order Processing
Event-time & Watermarks
Epoch Snapshotting for Exactly-once Processing
Hybrid Row(Protobuf) / Columnar (Arrow) System
Modular State Backend Abstraction

Example

#[arcon::app]
fn main() {
    (0..100u64)
        .to_stream(|conf| conf.set_arcon_time(ArconTime::Process))
        .filter(|x| *x > 50)
        .map(|x| x * 10)
        .print()
}

More examples can be found here.

Project Layout

arcon: Arcon crate
arcon_build: Protobuf builder
arcon_macros: Arcon derive macros.
arcon_tests: Integration tests
arcon_util: Common Arcon utilities
docs: Project website docs
examples: Example applications

Contributing

See Contributing

Community

Arcon is an ambitious project with many different development & research areas.

If you find Arcon interesting and want to learn more, then join the Zulip community!

Acknowledgements

Arcon is influenced by many great projects whether it is implementation, code practices or project structure:

License

This project is licensed under the Apache-2.0 license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Arcon by you shall be licensed as Apache-2.0, without any additional terms or conditions.

fnbetadev/arcon