/aestream

Efficient streaming of sparse event data supporting files, network I/O, GPU peripherals (via Torch) and neuromorphic protocols

Primary LanguageC++MIT LicenseMIT

AEStream - Address Event streaming library

Test status chat on Discord DOI

AEStream efficiently reads sparse events from an input source and streams it to an output sink. AEStream supports reading from files, USB cameras, as well as network via UDP and can stream events to files, network over UDP, and peripherals such as GPUs and neuromorphic hardware.

Read more in the AEStream publication.

Installation

Read more in our installation guide

AEStream is usable both as a command-line binary or Python tool.

Source Installation
pip pip install aestream
pip install aestream[torch] (PyTorch support)
pip install aestream --no-binary (Event camera support *)
nix nix run github:aestream/aestream (CLI)
nix develop github:aestream/aestream (Python environment)
docker See Installation documentation
* Event camera support requires available drivers. A step-by-step guide is available in our documentation.

Contributions to support AEStream on additional platforms are always welcome.

Usage (Python)

Read more in our Python usage guide

AEStream can process fixed input sources like files like so:

FileInput("file", (640, 480)).load()

Usage: stream real-time data in Python

AEStream also supports streaming data in real-time without strict guarantees on orders. This is particularly useful in real-time scenarios, for instance when operating with USBInput or UDPInput

# Stream events from a DVS camera over USB
with USBInput((640, 480)) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...
# Stream events from UDP port 3333 (default)
with UDPInput((640, 480), port=3333) as stream:
    while True:
        frame = stream.read() # Provides a (640, 480) tensor
        ...

More examples can be found in our example folder. Please note the examples may require additional dependencies (such as Norse for spiking networks or PySDL for rendering). To install all the requirements, simply stand in the aestream root directory and run pip install -r example/requirements.txt

Example: real-time edge detection with spiking neural networks

We stream events from a camera connected via USB and process them on a GPU in real-time using the spiking neural network library, Norse using fewer than 50 lines of Python. The left panel in the video shows the raw signal, while the middle and right panels show horizontal and vertical edge detection respectively. The full example can be found in example/usb_edgedetection.py

Usage (CLI)

Read more in our CLI usage documentation page

Installing AEStream also gives access to the command-line interface (CLI) aestream. To use aestraem, simply provide an input source and an optional output sink (defaulting to STDOUT):

aestream input <input source> [output <output sink>]

Supported Inputs and Outputs

Input Description Usage
DAVIS, DVXPlorer Inivation DVS Camera over USB input inivation
EVK Cameras Prophesee DVS camera over USB input prophesee
File AEDAT file format as .aedat, .aedat4, or .dat input file x.aedat4
Output Description Usage
STDOUT Standard output (default output) output stdout
Ethernet over UDP Outputs to a given IP and port using the SPIF protocol output udp 10.0.0.1 1234
.aedat4 file Output to .aedat4 format output file my_file.aedat4
CSV file Output to comma-separated-value (CSV) file format output file my_file.txt

CLI examples

Example Syntax
Echo file to STDOUT aestream input file example/sample.aedat4
Stream DVS cameara from iniVation AG to STDOUT (Note, requires Inivation libraries) aestream input inivation output stdout
Stream DVS camera from Prophesee to STDOUT (Note, requires Metavision SDK) aestream input output stdout
Read file to remote IP X.X.X.X aestream input file example/sample.aedat4 output udp X.X.X.X

Installation

Read more in our Installation guide

Acknowledgments

AEStream is developed by (in alphabetical order):

The work has received funding from the EC Horizon 2020 Framework Programme under Grant Agreements 785907 and 945539 (HBP) and by the Deutsche Forschungsgemeinschaft (DFG, German Research Fundation) under Germany's Excellence Strategy EXC 2181/1 - 390900948 (the Heidelberg STRUCTURES Excellence Cluster).

Thanks to Philipp Mondorf for interfacing with Metavision SDK and preliminary network code.

Citation

Please cite aestream if you use it in your work:

@misc{aestream,
  doi = {10.48550/ARXIV.2212.10719},
  url = {https://arxiv.org/abs/2212.10719},
  author = {Pedersen, Jens Egholm and Conradt, Jörg},
  title = {AEStream: Accelerated event-based processing with coroutines},
  publisher = {arXiv},
  year = {2022},
}