xtdb: A Clojure repository from chrismckelt

XTDB is an open-source immutable database with comprehensive time-travel. XTDB has been built to simplify application development and address complex data compliance requirements. XTDB can be used via SQL and XTQL.

XTDB 2.x is currently in early access; if you are looking for a stable release of an immutable document database with bitemporal query capabilities, we are continuing to develop and support XTDB 1.x at https://github.com/xtdb/xtdb/tree/1.x.

Major features:

Immutable - while it’s optimised for current-time queries, you can audit the full history of your database at any point, with the need for snapshots.
'Bitemporal' - all data is accurately versioned as updates are made ('system' time), but it also allows you to separately record and query when that data is, was, or will become valid in your business domain ('valid' time).
Dynamic - you don’t need to specify schema up-front before documents (rows with arbitrarily nested data) can be inserted.
Speaks both SQL and XTQL.

XTQL is a data-oriented, composable query language - designed from the ground up to be amenable to both hand-written and generated queries. It is heavily inspired by the theoretical bases of both Datalog and the relational algebra.

It also supports SQL, for compatibility with existing experience and tooling. Particularly, it supports the bitemporal functionality as specified in the SQL:2011 standard.
Cloud native - the ACID, columnar engine is built on Apache Arrow and designed for object storage
It is written and supported by JUXT.

Quick links

2.x Documentation
1.x Documentation (see the 1.x branch)
Maven releases
Release notes
Support: discuss.xtdb.com | hello@xtdb.com
Developing XTDB 2.x
Bibliography

Inside-out Architecture

XTDB embraces the transaction log as the central point of coordination when running as a distributed system.

What do we have to gain from turning the database inside out?

Simpler code, better scalability, better robustness, lower latency, and more flexibility for doing interesting things with data.

— Martin Kleppmann

Pre-Release Snapshot Builds

Maven snapshot versions are periodically published under 2.0.0-SNAPSHOT and are used to facilitate support and debugging activities during the development cycle. To access snapshots versions, the Sonatype snapshot repository must be added to your project definition:

<repository>
  <id>sonatype.snapshots</id>
  <name>Sonatype Snapshot Repository</name>
  <url>https://s01.oss.sonatype.org/content/repositories/snapshots</url>
  <releases>
    <enabled>false</enabled>
  </releases>
  <snapshots>
    <enabled>true</enabled>
  </snapshots>
</repository>

;; project.clj
:repositories [["sonatype-snapshots" {:url "https://s01.oss.sonatype.org/content/repositories/snapshots"}]]

;; deps.edn
:mvn/repos {"sonatype-snapshots" {:url "https://s01.oss.sonatype.org/content/repositories/snapshots"}}

In contrast to regular releases which are immutable, a 2.0.0-SNAPSHOT release can be "updated" - this mutability can often be useful but may also cause unexpected surprises when depending on 2.0.0-SNAPSHOT for longer than necessary. Snapshot versions, including full 2.0.0-<timestamp> coordinates (which are useful to avoid being caught out by mutation), can be found here.

XTDB 1.x

XTDB 1.x is a mature product offering that is used in production by many organizations, and its ongoing development is focused on hardening and performance. XTDB 1.x is an embeddable database that emphasizes in-process JVM usage to enable advanced functionality like user-defined transaction functions, speculative transactions, programmatic Datalog rules, and more.

XTDB 2.x’s initial objective is to take the key principles embodied in XTDB 1.x — immutability, schemaless records, and temporal querying — to a mainstream audience.

	XTDB 1.x	XTDB 2.x (early access)
Status	Stable	Experimental (pre-alpha)
Initial Stable Release	2019	TBD
Query languages	EDN Datalog	XTQL + SQL:2011
Bitemporal Querying	Timeslice only (point-in-time)	Fully bitemporal - SQL:2011 and beyond
Query focus	OLTP	OLAP + OLTP ('HTAP')
Storage & Compute	Coupled (nodes are full replicas)	Separated (cost-effective scale out)
Primary Storage Format	Custom Key-Value encodings	Columnar Apache Arrow
-	-	-
Immutable Semantics	Yes	Yes
Online Transactions (ACID, strong consistency)	Yes	Yes
Always-On Bitemporality	Yes	Yes
Dynamism (ad-hoc graph joins, union typing, schemaless, etc.)	Yes	Yes

Repo Layout

2.x is split across multiple projects which are maintained within this repository.

api contains the user API to XTDB 2.x.
core contains the main functional components of XTDB along with interfaces for the pluggable storage components (Kafka, S3 DB etc.). Implementations of these storage options are located in their own projects.
http-server and http-client-jvm contain the HTTP server implementation, and a remote client for JVM users.
Storage and other modules are under modules. Modules are published to Maven independently so that you can maintain granular dependencies on precisely the individual components needed for your application.

Questions, Thoughts & Feedback

We would love to hear from you: hello@xtdb.com

License and Copyright

XTDB is licensed under the Mozilla Public License, version 2 or (at your option) any later version.