ToroDB

ToroDB is an open source project that turns your RDBMS into a MongoDB-compatible server, supporting the MongoDB query API and MongoDB's replication, but storing your data into a reliable and trusted ACID database. ToroDB currently supports PostgreSQL as a backend, but others will be added in the future.

ToroDB natively implements the MongoDB protocol, so you can use it with MongoDB tools and drivers, and features a document-to-relational mapping algorithm that transforms the JSON documents into relational tables. ToroDB also offers a native SQL layer and automatic data normalization and partitioning based on JSON documents' implicit schema.

ToroDB follows a RERO (Release Early, Release Often) policy. Current version is considered a "developer preview" and hence is not suitable for production use. Use at your own risk. However, any feedback, contributions, help and/or patches are very welcome. Please join the torodb-dev mailing list for further discussion.

For more information, please see ToroDB's website, this latest presentation or this video recording of a presentation about ToroDB.

Requisites

ToroDB is written in Java and requires:

A suitable JRE, version 7 or higher. It has been mainly tested with Oracle JRE 8.
A PostgreSQL database, version 9.4 or higher. Download.

Download/Installation

Download the compiled file

You may download the latest version (v. 0.40-alpha2) of ToroDB from the release page on the following packaging formats:

tar.bz2
zip

See below for instructions on how to run it.

You can also find binary files on ToroDB's maven repository.

Compile and install from sources

To get the latest version, you may compile ToroDB yourself. All the project is written in Java and managed with Maven, so you need a javac and maven.

ToroDB is based on the Mongo Wire Protocol library (mongowp), which is another library built by 8Kdata to help construct programs that speak the MongoDB protocol. You may also compile this library yourself, or let maven download it from the repository automatically.

Just run mvn package -P assembler on the root directory and execute it from torodb/target/appassembler/bin or choose your prefered packaging format from torodb/target/dist/.

Compile and create docker image

To build the docker image the first time run mvn package -P docker,assembler -Ddocker.skipbase=false on the root directory.

Next time you will be able to build the torodb docker image running mvn package -P docker,assembler.

To run docker containers of ToroDB with PostgreSQL backend just run mvn docker:run -P docker,assembler -pl :standalone. You will have now ToroDB running on port 27018 and will be able to watch created table connection to PostgreSQL on port 15432 (user: postgres, password: postgres, database: torod).

Running ToroDB

ToroDB needs either a configuration file or some command-line parameters to run. But it can also run without any of them if you follow some conventions.

Before running ToroDB it is necessary to configure the RDBMS with the ToroDB user that will be responsible to create namespaces, required data types, tables and indexes.

Create user torodb (this is default user name, see ToroDB configuration to use a different name):

=# CREATE USER torodb WITH  SUPERUSER PASSWORD '<your-password>';

Create the database torod (this is default database name, see ToroDB configuration to use a different name):

=# CREATE DATABASE torod OWNER torodb;

The script $TOROHOME/bin/torodb (or torodb.bat) will run ToroDB. ToroDB can be configured by a configuration file written in YAML or XML formats by passing arguments -c or -x, respectively, to the script $TOROHOME/bin/torodb. For example, to run ToroDB with configuration file torodb.yml, run:

$ $TOROHOME/bin/torodb -c torodb.yml

To print default configuration script in YAML or XML format use the arguments -l an -lx respectively. For example to generate default YAML configuration file:

$ $TOROHOME/bin/torodb -l > torodb.yml

ToroDB connects to the backend database using user torodb (that has been created in previous step). By default ToroDB reads the file $HOME/.toropass (file path can be configured in the configuration), if it exists, that stores the password in PostgreSQL's .pgpass syntax. The password can also be specified in clear text in the configuration file or will be asked at the prompt if the argument -W is issued.

To get general help, pass --help argument:

$ $TOROHOME/bin/torodb --help

Use --help-param to get help on all available parameters of the configuration file:

$ $TOROHOME/bin/torodb --help-param

If you setup a .toropass, use torodb as the user and torod as the database, ToroDB will run without a configuration file (with the rest of the configuration values with their respective defaults).

Once ToroDB is running, connect to it with a normal MongoDB client, like: