Common Lisp client for XTDB 2.x.
Developped by Juxt:
- Mostly Clojure (runs on the JVM).
- Based on Apache Arrow
- HTAP
- Native support for both SQL-2011 and XQTL (inspired from Datalog and relational algebra).
- Accessible from over HTTP (MIME type
application/transit+json
- Some support for Postgrew wire protocol
Currently the only HTTP client released is written in Clojure. This projects open the door to non JVM based languages clients, Rust (with a C binding) being next in line. Note that a JSON API with JSON-LD is being worked on and this library might switch to using that once it is stable.
- Closely maps Clojure's version:
(let* ((table :|users|)
(node (make-xtdb-http-client "http://localhost:3000"))
(xt/id (uuid:make-v4-uuid))
(tx-key (submit-tx
node
(-> (put table (dict :|xt/id| xt/id
:|user-id| (uuid:make-v4-uuid)
:|name| "John Doe"))
(vect))))
(rc (query node
`(-> (from ,table ,(vect 'xt/id 'user-id 'name))
(where (= xt/id $id)))
:args (dict 'id xt/id)
:after-tx tx-key)))
(assert (and (= 1 (length rc))
(uuid:uuid= xt/id (href (car rc) :|xt/id|))))
rc)
returning something like:
((DICT
:|user-id| C25E1960-2BCD-4F6B-9570-924434556033
:|name| "John Doe"
:|xt/id| E6DD10F4-1B65-4193-AA59-47236CD464F9))
Common Lisp doesn't have most of the syntactic suggars Clojure added; we chose not to introduce any (this is Lisp after all) but instead use some from well established CL libs (e.g. Serapeum, Fset etc). Here is how the XTQL version of Q2 from the TPCH suite can be written:
SQL statements are transmitted as strings to the server. Hence the code snippet above would become:
(let* ((table :|users|)
(node (make-xtdb-http-client "http://localhost:3000"))
(xt/id (uuid:make-v4-uuid))
(tx-key (submit-tx node
(-> (put table (dict :|xt/id| xt/id
:|user-id| 123456
:|name| "John Doe"))
(vect))))
(rc (query node
"select u.* from users u where u.xt$id = ?"
:args (vect xt/id)
:after-tx tx-key)))
(assert (and (= 1 (length rc)) (uuid:uuid= xt/id (href (car rc) :|xt$id|))))
rc)
returning something like:
((DICT
:|user_id| 123456 ;; :user-id -> :user_id
:|name| "John Doe"
:|xt$id| F911B6AC-9581-4614-A799-A6961E0D2618)) ;; :xt/id -> :xt$id
Common Lisp has none of Clojure syntactic suggars, and we chose not to introduce any (this is Lisp after all) but instead use some from well established CL libs (e.g. Serapeum, Fset etc). Here is how the XQTL version of Q2 from the TPCH suite can be written:
`(-> (unify (from :part ,(vect (dict :xt/id 'p) 'p-mfgr (dict :p-size 15) 'p-type))
(where (like p-type "%BRASS"))
(from :partsupp ,(vect (dict :ps-partkey 'p :ps-suppkey 's) 'ps-supplycost))
(from :supplier ,(vect (dict :xt/id 's :s-nationkey 'n)
's-acctbal 's-address 's-name 's-phone 's-comment))
(from :nation ,(vect (dict :xt/id 'n :n-regionkey 'r) 'n-name))
(from :region ,(vect (dict :xt/id 'r :r-name "EUROPE")))
(where (= ps-supplycost
(q (-> (unify (from :partsupp ,(vect (dict :ps-partkey '$p :ps-suppkey 's) 'ps-supplycost))
(from :supplier ,(vect (dict :xt/id 's :s-nationkey 'n)))
(from :nation ,(vect (dict :xt/id 'n :n-regionkey 'r)))
(from :region ,(vect (dict :xt/id 'r :r-name "EUROPE"))))
(aggregate ,(dict :min-supplycost '(min ps-supplycost))))
,(dict :args #(p))))))
(order-by ,(dict :val 's-acctbal :dir :desc) n-name s-name p)
(limit 100))
Given XTDB 2.x loaded with a TCPH dataset at factor 0.05, the query above would return (17) documents:
((DICT
:|r| "regionkey_3"
:|s-phone| "17-290-812-8855"
:|s-address| "F4Uy ZQNU6ESTmO3mrL,mI"
:|p-type| "ECONOMY PLATED BRASS"
:|s-name| "Supplier#000000135"
:|n| "nationkey_7"
:|p-mfgr| "Manufacturer#2"
:|n-name| "GERMANY"
:|s| "suppkey_135"
:|s-acctbal| 9767.99d0
:|s-comment| "courts wake slyly instructions. furiously silent requests cajol"
:|p| "partkey_1634"
:|ps-supplycost| 372.29d0)
;; ....
(DICT
:|r| "regionkey_3"
:|s-phone| "17-617-724-5874"
:|s-address| "ZutN4kNIsv7sPWIV9,7"
:|p-type| "STANDARD ANODIZED BRASS"
:|s-name| "Supplier#000000212"
:|n| "nationkey_7"
:|p-mfgr| "Manufacturer#2"
:|n-name| "GERMANY"
:|s| "suppkey_212"
:|s-acctbal| -335.47d0
:|s-comment| "ular requests cajole furiously against the spe"
:|p| "partkey_3080"
:|ps-supplycost| 575.75d0
) )
This library is still a WIP but will be released to QuickLisp soon.
It relies on a recent version of cl-transit not yet picked up by Quicklist. Untill then, just clone cl-transit
in your local projects e.g.,:
$ git clone https://github.com/jsulmont/cl-transit ~/quicklisp/local-projects
For now, you can use the library from a REPL (e.g., Emacs):
- load
xtdb-cl.asd
- compile it (e.g.,
C-C C-K
) - Fire a REPL (e.g.
C-C C-Z
) - load
xtdb-cl
(e.g.(ql:quickload :xtdb-cl)
Although meant as a library, the Makefile allows to build a binary, which assuming you have an XTDB 2.x instance servicing localhost:3000
will loop with each iteration inserting a new document, fetching it then sleeping for 5ms.
This code (and other) will move to a dedicated Package.
$ make build
$ ./xtdb-cl
Tests are defined with Fiveam.
Run them from the terminal with make test
. You should see a failing test.
$ make test
Running test suite TESTMAIN
Running test TEST1 f
Did 1 check.
Pass: 0 ( 0%)
Skip: 0 ( 0%)
Fail: 1 (100%)
---
Licence: MIT