This repository is a reference for converting observability signals (traces, metrics, logs) to/from a common InfluxDB schema.
Steps to run the write full write/query demo follow.
In an InfluxDB Cloud 2 account backed by IOx, create a bucket named otel
.
Then, create a token with permission to read and write to that bucket.
In demo/docker-compose.yml, set values for these keys.
The key INFLUXDB_BUCKET_ARCHIVE
is optional;
if set, it should point to an InfluxDB bucket with longer retention policy than INFLUXDB_BUCKET
,
so that the "Archive Trace" button in Jaeger works properly:
The community addition focuses on the useability of the demo with Grafana. With this being said to improve demo setup we have introduced a .env
file that will allow you to set the following variables:
INFLUXDB_ADDR=eu-central-1-1.aws.cloud2.influxdata.com
INFLUXDB_TOKEN=xxxxxxx
INFLUXDB_ORG=xxxxxxxx
INFLUXDB_BUCKET=otel
INFLUXDB_BUCKET_ARCHIVE=otel-archive
In demo/otelcol-config.yml, set the similar values for these keys:
endpoint: https://< region specific URL - https://region.csp.cloud2.influxdata.com/ >
bucket: otel
token: <the API token you just created>
Run the docker compose:
$ docker-compose --file demo/docker-compose.yml --project-directory . up --abort-on-container-exit --remove-orphans
Traces are generated by "HotRod", an application designed to demonstrate tracing. Browse to HotRod at http://localhost:8080 and click some buttons to trigger trace activity.
Query those traces. Browse to Jaeger at http://localhost:16686 and click "Find Traces" near the bottom left.
Click any trace.
View the dependency graph. Click "System Architecture".
Grafana is available at http://localhost:3000. The default username and password are both admin
. The default datasource of flightSQL is already confgiured.
Note: You can find a dashboard to import under demo/grafana/dashboards/Open Telemetry-1681814438598.json
If you would like to access the Trace node tree. Then Make sure to enable it within the Jaeger datasource. Head to data sources and click on the Jaeger datasource. Then enable Enable Node Graph
. Then click save and test.
The images otelcol-influxdb
and jaeger-influxdb
are automatically built and pushed to Docker at https://hub.docker.com/r/jacobmarble/otelcol-influxdb and https://hub.docker.com/r/jacobmarble/jaeger-influxdb .
Schema reference with conversion tables.
The golang package common
contains simple utilities and common string values,
used in at least two of the above-mentioned packages.
The golang package otel2influx
converts OpenTelemetry protocol buffer objects to (measurement, tags, fields, timestamp) tuples.
It is imported by the OpenTelemetry Collector InfluxDB exporter
and by the Telegraf OpenTelemetry input plugin.
The golang package influx2otel
converts (measurement, tags, fields, timestamp) tuples to OpenTelemetry protocol buffer objects.
It is imported by the OpenTelemtry Collector InfluxDB receiver
and by the Telegraf OpenTelemetry output plugin.
The Jaeger Query Plugin for InfluxDB enables querying traces stored in InfluxDB/IOx via the Jaeger UI.
The golang package tests-integration
contains integration tests.
These tests exercise the above packages against OpenTelemetry Collector Contrib and Telegraf.
To run these tests:
$ cd tests-integration
$ go test
The file demo/docker-compose-development.yml
is similar to the demo above,
but it uses local builds of the otelcol-influxdb
and jaeger-influxdb
images.
Build the needed docker images:
$ docker compose --file demo/docker-compose-development.yml --project-directory . build
Start the development demo environment:
$ docker compose --file demo/docker-compose-development.yml --project-directory . up --abort-on-container-exit --remove-orphans
Changes can be tested on a local branch using the run-checks.sh
tool.
run-checks.sh
verifies go mod tidy
using git diff
,
so any changes must be staged for commit in order for run-checks.sh
to pass.
To update critical dependencies (OpenTelemetry, Jaeger, and intra-repo modules) in the various modules of this repository:
- run
update-deps.sh
- stage the changed
go.mod
andgo.sum
files - run
run-checks.sh
Fork this demo: https://github.com/open-telemetry/opentelemetry-demo