immux: A Rust repository from Immux

Description

The master repository for Immux intergrated computing infrastructure, including:

ImmuxDB, the immutable database, and
ImmuxCompute, the immutable computing service.

This repo builds upon an earlier 1.0 version and is under construction.

ImmuxDB

Run the servers

cargo build
cargo run

HTTP API

GET /grouping/id ("get", outside of transactions)
GET /.transactions/tid/grouping/unit_id ("get" within transaction)
GET /grouping/id/.journal ("inspect_one")
GET /.journal ("inspect all")
GET /grouping?filter=xx&lt=xxx&gtxxx
GET /grouping (get all in grouping)
GET /.groupings (get all the grouping)

PUT /grouping/id {data} ("set")
PUT /grouping/id?height={height} ("revert one")
PUT /.transactions/tid/grouping/unit_id?height={height} ("revert one" within transaction)
PUT /.transactions/tid/grouping/unit_id {data} ("set" within transaction)
PUT /?height={height} ("revert all")

DELETE /grouping/id ("remove one")
DELETE /.transactions/tid/grouping/unit_id ("remove one" with transaction)
DELETE / ("remove all")
DELETE /.groupings {list of groupings} (delete all in these groupings)

POST /.transactions ("create_transaction")
POST /.transactions/tid?commit ("commit_transaction")
POST /.transactions/tid?abort("abort_transaction")

/* Not implemented yet
GET /grouping/.index (get indices)
POST /grouping/.index?field={field} (create index)
DELETE /grouping/.index/index-number (delete index)
*/

Test

Execute storage.tests (both unit storage.tests and end-to-end storage.tests)

cargo test

If you are using Ubuntu or Pop!_OS, remember to run this command before executing cargo test:

sudo apt-get install libssl-dev

Gather test coverage

1. Prepare

rustup toolchain install nightly # add nightly
cargo install grcov # install grcov

2. Gather data

sh coverage.sh

3. Check result

Look at storage.tests/coverage-report/index.html.

Benchmarking

You can run all the benchmarking with a txt output with:

sh benches/benches.sh

There are two sets of benchmarks:

artificial benchmarks uses datasets generated by programs which focuses on regularity of data;
realistic benchmarks uses realistic datasets and aims to reflect database performance under "real" load.

Artificial benchmarks

Currently, we totally have 3 different artificial data set. indexed_set, journal and unindex_set, you can bench specific data set by running the command:

cargo bench --bench indexed_set -- 100000 100 10 0

In the above example, parameters explanation:

100000 is row_count.
100 is num_jsons_per_command.
10 is report_period.
0 is verify_correctness.

Realistic benchmarks

Each benchmark under benches/realistic are independent and requires different datasets, which is not pre-loaded in the Git version control because they are large.

Using berka99 as an example:

1. Download data

cd benches/realistic/berka99
./init_data.sh

2. Execute benchmark

cargo bench --bench berka99

Profiling

1. Install flamegraph

cargo install flamegraph

2. Generate binary executable

cargo bench

3. Run flamegraph

# Use sudo if prompted
flamegraph target/release/realistic-SOME_HASH

Inspect the "flamegraph.svg" in a web browser (it's interactive).

The exact executable name needs to be checked manually at the moment.

License

GPLv3

immux/immux