It's a party of information!
Carnival is a data unification technology that enables the aggregation of data from disparate sources into a unified property graph and provides tools to reason over and interact with graph data using bounded operations. Carnival has a robust architecture for tracking the provenance of data and providing evidence chains for conclusions or reasoning made on those data.
Carnival uses objects called vines to connect to external data sources and reapers encode the domain knowledge specific to that data source. Vines can connect to sources such as MySql or Oracle databases, RedCap projects, and CSV files. Some vine features include:
- Parameterized SQL queries
- Utilities to compose iterative SQL from lists of identifiers and codes
- Caching of query results
- Incremental caching of long running query result data
- Monitor thread to estimate time to completion of long running queries
- Automatic re-establishment of dropped connections
- API layer for REDCap
- H2 database wrapper for CSV data
Carnival’s property graph database:
- Is inherently schema-less enabling the incorporation of new data without restructuring resident data
- Follows data instantiation patterns built for computational efficiency and inspired by OBO Foundry ontologies
- Has a query engine capable of executing queries of arbitrary complexity
The Github pages site is stored in the /docs directory and makes use of jekyll. See the jekyll docs for jekyll installation and usage instructions.
To build the documentation:
cd docs
bundle exec jekyll clean
bundle exec jekyll build
To run the Jekyll server locally:
bundle exec jekyll serve
| Name | Description |
|---|---|
| carnival-util | Contains utility and helper classes such as MappedDataTable, FeatureReport and SqlUtils. |
| carnival-graph | Framework for defining carnival graph schemas (vertex and edge definitions). Contains the basic vertex, edge, and property classes. |
| carnival-core | Basic carnival framework. Implements the basic carnival framework classes (vines, reapers, reasonsers, etc). Defines the basic carnival graph schema (processes, databases). - Core graph schema - Reaper schema - Reasoner schema |
| carnival-gradle | Gradle plugin for building a Micronaut app that relies on Carnival. |
| carnival-gremlin-dsl | Gremlin dsl support for traversing carnival property graphs. |
| Name | Description |
|---|---|
| carnival-clinical | Extension of carnival-core for clinical data. Contains graph schema extensions for concepts such as patients, patient cohorts and healthcare encounters. Implements methods for case-control matching for patient cohorts. - Graph schema |
See developer setup for full documentation on how to set up a development environment, and a tutorial for getting started.