Demeter

The dbt Metrics API

Demeter turns your dbt project into a metrics platform. You get a REST/GraphQL API so you can query, build interactive apps, and use your metrics from anywhere.

View live demo

Demeter

About

Demeter gives you a REST API (including a GraphQL endpoint) to query your dbt metrics

Installation

Prerequisites

A dbt project with metrics
Node
- Run node --version
- If there is no output or node is not found, follow the steps at https://heynode.com/tutorial/install-nodejs-locally-nvm/ to install Node.

Quickstart (local)

📦 Install dbt-metrics-api
- Add the following to your dbt project's packages.yml file:
```
- git: "https://github.com/mjirv/demeter.git"
  revision: main
```
- Run dbt deps
✅ Run
- Run cd dbt_packages/dbt_metrics_api/server && npm i && node dist/
- Navigate to http://localhost:3001/graphql in your browser to start querying your API!

Usage

Routes

dbt-metrics-api has 4 routes:

GET `/metrics`

returns a JSON array of your project's metrics

supports "name", "type", "model", and "package_name" query strings for filtering

$ curl -X GET "http://localhost:3001/metrics?type=count&package_name=jaffle_shop"

[{"unique_id":"metric.jaffle_shop.orders","package_name":"jaffle_shop","model":"ref('orders')","name":"orders","description":"The number of orders","label":"Orders","type":"count","filters":[],"time_grains":["day","week","month","quarter","year"],"dimensions":["status","customer_id"]},{"unique_id":"metric.jaffle_shop.orders2","package_name":"jaffle_shop","model":"ref('orders')","name":"orders2","description":"The number of orders","label":"Orders","type":"count","filters":[],"time_grains":["day","week","month","quarter","year"],"dimensions":["status","customer_id"]}]

GET `/metrics/:metric_name`

returns a JSON object with keys unique_id, package_name, model, name, description, label, type, filters, time_grains, dimensions

$ curl -X GET "http://localhost:3001/metrics/orders"

{"unique_id":"metric.jaffle_shop.orders","package_name":"jaffle_shop","model":"ref('orders')","name":"orders","description":"The number of orders","label":"Orders","type":"count","filters":[],"time_grains":["day","week","month","quarter","year"],"dimensions":["status","customer_id"]}

POST `/metrics/:metric_name`

Queries a metric and returns the result
Accepts a JSON object in the request body with the following properties: grain, dimensions, start_date, end_date (start_date and end_date are optional)

Returns a JSON object or CSV depending on your Accept: header (application/json or text/csv)

$ curl http://localhost:3001/metrics/orders -H "Content-Type: application/json" -H "Accept: application/json" -d '{"grain": "year", "start_date": "2017-01-01", "end_date": "2019-01-01"}'

[{"period": "2017-01-01", "orders": 0.0}, {"period": "2018-01-01", "orders": 99.0}, {"period": "2019-01-01", "orders": 0.0}]

POST `/graphql`

GraphQL API for your metrics
Visit YOUR_SERVER_PATH/graphql (e.g. http://localhost:3001/graphql) to see the GraphiQL client and schema

Advanced

Running a production instance

On your favorite cloud hosting service (e.g. Heroku or GCP Cloud Run) install this repo via the GitHub URL.

Look in .example.env for the available environment variables and configure them as needed.

Authentication

dbt-metrics-api supports Kable for authentication.

To get started:

Note your client ID and secret and add them to your server/.env.local file:

KABLE_CLIENT_ID=
KABLE_CLIENT_SECRET=

# you will need to set to LIVE in production
KABLE_ENV=TEST

Requests will now require authentication with a customer ID and secret that you set up in your Kable dashboard using X-CLIENT-ID and X-API-KEY headers
```
$ curl -X GET "http://localhost:3001/metrics/orders" -H "X-CLIENT-ID: test-customer-1" -H "X-API-KEY: sk_test.some.secret.key"
```

mjirv/demeter