DamithaSenevirathne/jasminegraph

High Performance Graph Database Server

JasmineGraph

1. Introduction

JasmineGraph is a C/C++ based distributed graph database server. It has been developed following the Acacia graph database server architecture. JasmineGraph can be run on single computer as well as on a compute cluster.

2. Building JasmineGraph on Single Computer

Prerequisites

The following build tools, applications, and libraries need to be installed before building JasmineGraph. The CMakeLists.txt file's entries need to be updated to match with the installation locations of the dependencies. For examole, if you install SpdLog in a directory called "software" in your Linux system you may have to add a line as "include_directories("$ENV{HOME}/software/spdlog/include")" to get the SpdLog's header files associated with you build process.

• GCC 9.1.0 (recommended) or above

• Cmake version 3.10.3 or above

• Google Flat Buffers (https://github.com/google/flatbuffers)

  • Clone the flatbuffers repository to $ENV{HOME}/software
  • Checkout Tag v1.12.0
  • Follow https://google.github.io/flatbuffers/flatbuffers_guide_building.html link to build flatbuffers

• Metis (http://glaros.dtc.umn.edu/gkhome/metis/metis/download)

  • Follow the installation guidelines provided in this URL.

• SpdLog (https://github.com/gabime/spdlog)

  • Clone or download the repository from the above link
  • Issue "cmake ."
  • Issue "make" followed by "sudo make install"

• SQLite3 (https://www.sqlite.org/download.html)

• cppkafka (https://github.com/mfontanini/cppkafka)

  • Install librdkafka - Follow the Readme in (https://github.com/edenhill/librdkafka)
  • Install boost library - use 'sudo apt-get install libboost-all-dev'
  • Once cppkafka is built install it by running 'sudo make install' from the build directory

• Xerces-c-3.2.2 (https://xerces.apache.org/xerces-c/)

  • Intallation guide is available at http://xerces.apache.org/xerces-c/build-3.html
  • When configuring Xerces skip the use of transcoder ICU by using the flag --disable-transcoder-icu
  • Once installed specify the target_link_libraries path to xerces.

• Jsoncpp(https://github.com/open-source-parsers/jsoncpp)

  • Install the release https://github.com/open-source-parsers/jsoncpp/releases/tag/1.8.4
  • Issue "cmake ."
  • Issue "make" followed by "sudo make install"

• pigz (optional)

  • pigz, which stands for Parallel Implementation of GZip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data.
  • Jasminegraph by default uses pigz for file compression and decompression.
  • However if pigz is not found, gzip is used instead.
  • Download pigz from (http://zlib.net/pigz/) or (https://github.com/madler/pigz.git).
  • Type "make" in pigz directory to build the "pigz" executable.
  • Add pigz executable's path to ~/.bashrc file and refresh.

• python3.5 (https://www.python.org/downloads/release/python-350/)

  • Read this blog for more details on setting up python3.5

• Recent versions of TensorFlow, numpy, scipy, sklearn, and networkx are required (but networkx must be <=1.11). You can install all the required packages using the following command:

  pip install -r ./GraphSAGE/requirements

First, this repository should be cloned into one of your computer's local directory. Then change directory to jasminegraph and run ./build.sh to build the JasmineGraph executable.

3. Running JasmineGraph

JasmineGraph can be run by executing the run.sh script. This will start master on your local computer while workers are created in the list of the hosts mentioned in the conf/hosts.txt file.

4. Contact Information

Please contact [Miyuru Dayarathna](miyurud at yahoo dot co dot uk) for further information. Please let us know about bug reports or any further improvements you wish to have in JasmineGraph.

Open Source License

JasmineGraph is licensed under the Apache License, Version 2.0. Task Scheduler which is integrated in JasmineGraph is using MIT License. GraphSAGE which is integrated in JasmineGraph to generate node embeddings is using MIT License.

References

More details of JasmineGraph/Acacia architecture is available from the following list of papers.

• Miyuru Dayarathna, Sathya Bandara, Nandula Jayamaha, Mahen Herath, Achala Madhushan, Sanath Jayasena, Toyotaro Suzumura. 2017. "An X10-Based Distributed Streaming Graph Database Engine.," 2017 IEEE 24th International Conference on High Performance Computing (HiPC), Jaipur, 2017, pp. 243-252.
• Miyuru Dayarathna, Isuru Herath, Yasima Dewmini, Gayan Mettananda, Sameera Nandasiri, Sanath Jayasena, Toyotaro Suzumura. 2016 "Acacia-RDF: An X10-Based Scalable Distributed RDF Graph Database Engine.," 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), San Francisco, CA, 2016, pp. 521-528. doi: 10.1109/CLOUD.2016.0075
• Miyuru Dayarathna and Toyotaro Suzumura. 2014. "Towards scalable distributed graph database engine for hybrid clouds." In Proceedings of the 5th International Workshop on Data-Intensive Computing in the Clouds (DataCloud '14). IEEE Press, Piscataway, NJ, USA, 1-8.