[ Homepage ] [ Documentation ]
A domain specific language for building scalable, distributed stream processing applications with custom distribution strategies.
Built in the context of my PhD at the Software Languages Lab.
Skitter is a distributed stream processing system: it makes it possible to define data processing pipelines by combining data processing steps called operations into a workflow.
A key difference between Skitter and other related technologies is the notion of distribution strategies. An operation in a Skitter workflow is paired with a strategy which defines how the operation is distributed over a cluster at runtime. This enables developers to select the appropriate distribution strategy for each operation. Developers can use any of the strategies provided by Skitter or decide to build their own strategies. Strategies can be implemented from scratch or by extending existing strategies.
More information about Skitter can be found at: https://soft.vub.ac.be/~mathsaey/skitter.
We published about Skitter at the following venues:
- Skitter: A DSL for Distributed Reactive Workflows (REBLS, November 2018)
Note that the version of Skitter discussed in this paper differs significantly from the current version. Information on using this earlier version of Skitter can be found here.
We refer developers interested in creating and running Skitter projects to the project's documentation. There, we provide detailed information on the language concepts and abstractions offered by Skitter along with guides which detail how Skitter can be used in practice on your local machine or on a cluster of machines.
The official documentation can be found on hexdocs.