OpenTelemetry Operator for Kubernetes

The OpenTelemetry Operator is an implementation of a Kubernetes Operator.

At this point, it has OpenTelemetry Collector as the only managed component.

Documentation

OpenTelemetryCollector Custom Resource Specification

Getting started

To install the operator in an existing cluster, make sure you have cert-manager installed and run:

kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml

Once the opentelemetry-operator deployment is ready, create an OpenTelemetry Collector (otelcol) instance, like:

$ kubectl apply -f - <<EOF
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: simplest
spec:
  config: |
    receivers:
      jaeger:
        protocols:
          grpc:
    processors:

    exporters:
      logging:

    service:
      pipelines:
        traces:
          receivers: [jaeger]
          processors: []
          exporters: [logging]
EOF

WARNING: Until the OpenTelemetry Collector format is stable, changes may be required in the above example to remain compatible with the latest version of the OpenTelemetry Collector image being referenced.

This will create an OpenTelemetry Collector instance named simplest, exposing a jaeger-grpc port to consume spans from your instrumented applications and exporting those spans via logging, which writes the spans to the console (stdout) of the OpenTelemetry Collector instance that receives the span.

The config node holds the YAML that should be passed down as-is to the underlying OpenTelemetry Collector instances. Refer to the OpenTelemetry Collector documentation for a reference of the possible entries.

At this point, the Operator does not validate the contents of the configuration file: if the configuration is invalid, the instance will still be created but the underlying OpenTelemetry Collector might crash.

Deployment modes

The CustomResource for the OpenTelemetryCollector exposes a property named .Spec.Mode, which can be used to specify whether the collector should run as a DaemonSet, Sidecar, or Deployment (default). Look at the examples/daemonset.yaml for reference.

Sidecar injection

A sidecar with the OpenTelemetry Collector can be injected into pod-based workloads by setting the pod annotation sidecar.opentelemetry.io/inject to either "true", or to the name of a concrete OpenTelemetryCollector from the same namespace, like in the following example:

$ kubectl apply -f - <<EOF
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: sidecar-for-my-app
spec:
  mode: sidecar
  config: |
    receivers:
      jaeger:
        protocols:
          grpc:
    processors:

    exporters:
      logging:

    service:
      pipelines:
        traces:
          receivers: [jaeger]
          processors: []
          exporters: [logging]
EOF

$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: myapp
  annotations:
    sidecar.opentelemetry.io/inject: "true"
spec:
  containers:
  - name: myapp
    image: jaegertracing/vertx-create-span:operator-e2e-tests
    ports:
      - containerPort: 8080
        protocol: TCP
EOF

When there are multiple OpenTelemetryCollector resources with a mode set to Sidecar in the same namespace, a concrete name should be used. When there's only one Sidecar instance in the same namespace, this instance is used when the annotation is set to "true".

The annotation value can come either from the namespace, or from the pod. The most specific annotation wins, in this order:

the pod annotation is used when it's set to a concrete instance name or to "false"
namespace annotation is used when the pod annotation is either absent or set to "true", and the namespace is set to a concrete instance or to "false"

When using a pod-based workload, such as Deployment or Statefulset, make sure to add the annotation to the PodTemplate part. Like:

$ kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  labels:
    app: my-app
  annotations:
    sidecar.opentelemetry.io/inject: "true" # WRONG
spec:
  selector:
    matchLabels:
      app: my-app
  replicas: 1
  template:
    metadata:
      labels:
        app: my-app
      annotations:
        sidecar.opentelemetry.io/inject: "true" # CORRECT
    spec:
      containers:
      - name: myapp
        image: jaegertracing/vertx-create-span:operator-e2e-tests
        ports:
          - containerPort: 8080
            protocol: TCP
EOF

Compatibility matrix

OpenTelemetry Operator vs. OpenTelemetry Collector

The OpenTelemetry Operator follows the same versioning as the operand (OpenTelemetry Collector) up to the minor part of the version. For example, the OpenTelemetry Operator v0.18.1 tracks OpenTelemetry Collector 0.18.0. The patch part of the version indicates the patch level of the operator itself, not that of OpenTelemetry Collector. Whenever a new patch version is released for OpenTelemetry Collector, we'll release a new patch version of the operator.

OpenTelemetry Operator vs. Kubernetes

We strive to be compatible with the widest range of Kubernetes versions as possible, but some changes to Kubernetes itself require us to break compatibility with older Kubernetes versions, be it because of code incompatibilities, or in the name of maintainability.

Our promise is that we'll follow what's common practice in the Kubernetes world and support N-2 versions, based on the release date of the OpenTelemetry Operator.

For instance, when we released v0.27.0, the latest Kubernetes version was v1.21.1. As such, the minimum version of Kubernetes we support for OpenTelemetry Operator v0.27.0 is v1.19 and we tested it with up to 1.21.

The OpenTelemetry Operator might work on versions outside of the given range, but when opening new issues, please make sure to test your scenario on a supported version.