Yu-zh/hstream

The database built for IoT streaming data storage and real-time stream processing.

HStreamDB

The database built for IoT streaming data storage and real-time stream processing.

Main Features

• Push real-time data to your apps

  By subscribing to streams in HStreamDB, any update of the data stream will be pushed to your apps in real-time, and this promotes your apps to be more responsive.

  You can also replace message brokers with HStreamDB and everything you do with message brokers can be done better with HStreamDB.

• Stream processing with familiar SQL

  HStreamDB provides built-in support for event-time based stream processing. You can use your familiar SQL to perform basic filtering and transformation operations, statistics and aggregation based on multiple kinds of time windows and even joining between multiple streams.

• Easy integration with a variety of external systems

  With connectors provided, you can easily integrate HStreamDB with other external systems, such as MQTT Broker, MySQL, Redis and ElasticSearch. More connectors will be added.

• Real-time query based on live materailze views

  With maintaining materialized views incrementally, HStreamDB enables you to gain ahead-of-the-curve data insights that response to your business quickly.

• Reliable persistent storage with low latency

  With an optimized storage design based on LogDevice, not only can HStreamDB provide reliable and persistent storage but also guarantee excellent performance despite large amounts of data written to it.

• Seamless scaling and high availability

  With the architecture that separates compute from storage, both compute and storage layers of HStreamDB can be independently scaled seamlessly. And with the consensus algorithm based on the optimized Paxos, data is securely replicated to multiple nodes which ensures high availability of our system.

For more information, please visit HStreamDB homepage.

Installation

Install via HStreamDB Docker Image

docker pull hstreamdb/logdevice
docker pull hstreamdb/hstream

Quickstart

For detailed instructions, follow HStreamDB quickstart.

1. Install HStreamDB.
2. Start a local standalone HStream server.
3. Start HStreamDB's interactive CLI and create your first stream.
4. Run a continuous query.
5. Start another interactive CLI, then insert some data into the stream and get query results.

Documentation

Check out the documentation.

Community, Discussion, Construction and Support

You can reach the HStreamDB community and developers via the following channels:

• Slack
• Twitter
• Reddit

Please submit any bugs, issues, and feature requests to hstreamdb/hstream.

How to build (for developers only)

Pre-requirements

1. Make sure you have Docker installed, and can run docker as a non-root user.
2. You have python3 installed.
3. You can clone the GitHub repository by ssh key.

Get the source code

git clone --recursive git@github.com:hstreamdb/hstream.git
cd hstream/

Update images

script/dev-tools update-images

Start all required services

You must have all required services (including a dev-cluster) started before entering an interactive shell to do further development.

script/dev-tools start-services

This script also involves starting a MySQL server. To inspect the server port, use echo "$MYSQL_LOCAL_PORT".

A dev-cluster is required while running tests. All data are stored under your-project-root/local-data/logdevice

script/dev-tools cluster-start

Enter in an interactive shell

script/dev-tools shell

Build as other Haskell projects

Inside the interactive shell, you have all extra dependencies installed.

I have no name!@649bc6bb75ed:~$ make
I have no name!@649bc6bb75ed:~$ cabal build all

License

HStreamDB is under the BSD 3-Clause license. See the LICENSE file for details.

Acknowledgments

• Thanks LogDevice for the powerful storage engine.