Hydrolix Trino Connector

Overview

This is a Trino plugin that allows Trino to access data stored in Hydrolix clusters directly, without ETL.

Based on connectors-core.
Currently targets version 435 of the Trino SPI. Other Trino versions, or Presto, might come later; Trino/Presto break binary compatibility constantly! If you need a specific version, let us know and we can probably make it happen quickly.

System Requirements

Trino 435
Worker nodes must be recent (>=2022) x64 Linux. For local development on other machines, you can try Docker, using this configuration parameter.

Building

Install SBT in whatever way makes sense for your OS
git clone git@github.com:hydrolix/trino-connector.git hydrolix-trino-connector && cd hydrolix-trino-connector
sbt +assembly will produce ./target/scala-2.13/hydrolix-trino-connector-assembly-0.1.0-SNAPSHOT.jar

Running

Symlink the JAR that was built by the previous step into the right place in the trino-server directory:

cd ~/dev/trino-server-435/plugin
mkdir hydrolix && cd hydrolix
ln -s ~/dev/hydrolix-trino-connector/target/scala-2.13/hydrolix-trino-connector-assembly-0.1.0-SNAPSHOT.jar .

Add a Hydrolix catalog configuration file to your Trino server:

cat ~/dev/trino-server-435/etc/catalog/hydrolix.properties:

connector.name=hydrolix

jdbc_url=jdbc:clickhouse://my-hdx-cluster.net:8088/_local?ssl=true
api_url=https://my-hdx-cluster.net/config/v1/

username=hdx_user
password=hdx_password

### For GCS:
# cloud_cred_1=<base64(gzip(service account JSON key file))>

### For AWS:
# cloud_cred_1=<AWS access key ID>
# cloud_cred_2=<AWS secret key>  

### For local dev in Docker
# turbine_cmd_docker=my_image_name

Run your Trino server:

cd ~/dev/trino-server-435
bin/launcher run

Or, if you want to attach a debugger:

cd ~/dev/trino-server-435
bin/launcher \
    -J-Xdebug \
    -J-Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005 \
    run

Connect a Trino client and run a query!

java -jar trino-cli-435-executable.jar --catalog=hydrolix --user=admin

trino> select count(*), min(timestamp), max(timestamp) from hydro.logs where timestamp > now() - (interval '5' minute);
 _col0 |          _col1          |          _col2          
-------+-------------------------+-------------------------
 29699 | 2024-01-11 21:55:00.061 | 2024-01-11 22:04:01.245 
(1 row)

Query 20240111_220408_00000_dctfe, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
8.63 [29.7K rows, 261KB] [3.44K rows/s, 30.3KB/s]

hydrolix/trino-connector

Hydrolix Trino Connector

Overview

System Requirements

Building

Running

Feature Set

Release Notes

0.1.0