Jaeger Bindings for Go OpenTracing API

Instrumentation library that implements an OpenTracing Tracer for Jaeger (https://jaegertracing.io).

IMPORTANT: The library's import path is based on its original location under github.com/uber. Do not try to import it as github.com/jaegertracing, it will not compile. We might revisit this in the next major release.

✅ import "github.com/uber/jaeger-client-go"
❌ import "github.com/jaegertracing/jaeger-client-go"

How to Contribute

Please see CONTRIBUTING.md.

Installation

We recommended using a dependency manager like glide and semantic versioning when including this library into an application. For example, Jaeger backend imports this library like this:

- package: github.com/uber/jaeger-client-go
  version: ^2.7.0

If you instead want to use the latest version in master, you can pull it via go get. Note that during go get you may see build errors due to incompatible dependencies, which is why we recommend using semantic versions for dependencies. The error may be fixed by running make install (it will install glide if you don't have it):

go get -u github.com/uber/jaeger-client-go/
cd $GOPATH/src/github.com/uber/jaeger-client-go/
git submodule update --init --recursive
make install

Initialization

See tracer initialization examples in godoc and config/example_test.go.

Environment variables

The tracer can be initialized with values coming from environment variables. None of the env vars are required and all of them can be overriden via direct setting of the property on the configuration object.

Property	Description
JAEGER_SERVICE_NAME	The service name
JAEGER_AGENT_HOST	The hostname for communicating with agent via UDP
JAEGER_AGENT_PORT	The port for communicating with agent via UDP
JAEGER_ENDPOINT	The HTTP endpoint for sending spans directly to a collector, i.e. http://jaeger-collector:14268/api/traces
JAEGER_USER	Username to send as part of "Basic" authentication to the collector endpoint
JAEGER_PASSWORD	Password to send as part of "Basic" authentication to the collector endpoint
JAEGER_REPORTER_LOG_SPANS	Whether the reporter should also log the spans
JAEGER_REPORTER_MAX_QUEUE_SIZE	The reporter's maximum queue size
JAEGER_REPORTER_FLUSH_INTERVAL	The reporter's flush interval, with units, e.g. "500ms" or "2s" (valid units)
JAEGER_SAMPLER_TYPE	The sampler type
JAEGER_SAMPLER_PARAM	The sampler parameter (number)
JAEGER_SAMPLER_MANAGER_HOST_PORT	The HTTP endpoint when using the remote sampler, i.e. http://jaeger-agent:5778/sampling
JAEGER_SAMPLER_MAX_OPERATIONS	The maximum number of operations that the sampler will keep track of
JAEGER_SAMPLER_REFRESH_INTERVAL	How often the remotely controlled sampler will poll jaeger-agent for the appropriate sampling strategy, with units, e.g. "1m" or "30s" (valid units)
JAEGER_TAGS	A comma separated list of `name = value` tracer level tags, which get added to all reported spans. The value can also refer to an environment variable using the format `${envVarName:default}`, where the `:default` is optional, and identifies a value to be used if the environment variable cannot be found
JAEGER_DISABLED	Whether the tracer is disabled or not. If true, the default `opentracing.NoopTracer` is used.
JAEGER_RPC_METRICS	Whether to store RPC metrics

By default, the client sends traces via UDP to the agent at localhost:6831. Use JAEGER_AGENT_HOST and JAEGER_AGENT_PORT to send UDP traces to a different host:port. If JAEGER_ENDPOINT is set, the client sends traces to the endpoint via HTTP, making the JAEGER_AGENT_HOST and JAEGER_AGENT_PORT unused. If JAEGER_ENDPOINT is secured, HTTP basic authentication can be performed by setting the JAEGER_USER and JAEGER_PASSWORD environment variables.

Closing the tracer via `io.Closer`

The constructor function for Jaeger Tracer returns the tracer itself and an io.Closer instance. It is recommended to structure your main() so that it calls the Close() function on the closer before exiting, e.g.

tracer, closer, err := cfg.NewTracer(...)
defer closer.Close()

This is especially useful for command-line tools that enable tracing, as well as for the long-running apps that support graceful shutdown. For example, if your deployment system sends SIGTERM instead of killing the process and you trap that signal to do a graceful exit, then having defer closer.Closer() ensures that all buffered spans are flushed.

Metrics & Monitoring

The tracer emits a number of different metrics, defined in metrics.go. The monitoring backend is expected to support tag-based metric names, e.g. instead of statsd-style string names like counters.my-service.jaeger.spans.started.sampled, the metrics are defined by a short name and a collection of key/value tags, for example: name:jaeger.traces, state:started, sampled:y. See metrics.go file for the full list and descriptions of emitted metrics.

The monitoring backend is represented by the metrics.Factory interface from package "github.com/uber/jaeger-lib/metrics". An implementation of that interface can be passed as an option to either the Configuration object or the Tracer constructor, for example:

import (
    "github.com/uber/jaeger-client-go/config"
    "github.com/uber/jaeger-lib/metrics/prometheus"
)

    metricsFactory := prometheus.New()
    tracer, closer, err := config.Configuration{
        ServiceName: "your-service-name",
    }.NewTracer(
        config.Metrics(metricsFactory),
    )

By default, a no-op metrics.NullFactory is used.

Logging

The tracer can be configured with an optional logger, which will be used to log communication errors, or log spans if a logging reporter option is specified in the configuration. The logging API is abstracted by the Logger interface. A logger instance implementing this interface can be set on the Config object before calling the New method.

Besides the zap implementation bundled with this package there is also a go-kit one in the jaeger-lib repository.

Instrumentation for Tracing

Since this tracer is fully compliant with OpenTracing API 1.0, all code instrumentation should only use the API itself, as described in the opentracing-go documentation.

Features

Reporters

A "reporter" is a component that receives the finished spans and reports them to somewhere. Under normal circumstances, the Tracer should use the default RemoteReporter, which sends the spans out of process via configurable "transport". For testing purposes, one can use an InMemoryReporter that accumulates spans in a buffer and allows to retrieve them for later verification. Also available are NullReporter, a no-op reporter that does nothing, a LoggingReporter which logs all finished spans using their String() method, and a CompositeReporter that can be used to combine more than one reporter into one, e.g. to attach a logging reporter to the main remote reporter.

Span Reporting Transports

The remote reporter uses "transports" to actually send the spans out of process. Currently the supported transports include:

Jaeger Thrift over UDP or HTTP,
Zipkin Thrift over HTTP.

Sampling

The tracer does not record all spans, but only those that have the sampling bit set in the flags. When a new trace is started and a new unique ID is generated, a sampling decision is made whether this trace should be sampled. The sampling decision is propagated to all downstream calls via the flags field of the trace context. The following samplers are available:

RemotelyControlledSampler uses one of the other simpler samplers and periodically updates it by polling an external server. This allows dynamic control of the sampling strategies.
ConstSampler always makes the same sampling decision for all trace IDs. it can be configured to either sample all traces, or to sample none.
ProbabilisticSampler uses a fixed sampling rate as a probability for a given trace to be sampled. The actual decision is made by comparing the trace ID with a random number multiplied by the sampling rate.
RateLimitingSampler can be used to allow only a certain fixed number of traces to be sampled per second.

Baggage Injection

The OpenTracing spec allows for baggage, which are key value pairs that are added to the span context and propagated throughout the trace. An external process can inject baggage by setting the special HTTP Header jaeger-baggage on a request:

curl -H "jaeger-baggage: key1=value1, key2=value2" http://myhost.com

Baggage can also be programatically set inside your service:

if span := opentracing.SpanFromContext(ctx); span != nil {
    span.SetBaggageItem("key", "value")
}

Another service downstream of that can retrieve the baggage in a similar way:

if span := opentracing.SpanFromContext(ctx); span != nil {
    val := span.BaggageItem("key")
    println(val)
}

Debug Traces (Forced Sampling)

Programmatically

The OpenTracing API defines a sampling.priority standard tag that can be used to affect the sampling of a span and its children:

import (
    "github.com/opentracing/opentracing-go"
    "github.com/opentracing/opentracing-go/ext"
)

span := opentracing.SpanFromContext(ctx)
ext.SamplingPriority.Set(span, 1)

Via HTTP Headers

Jaeger Tracer also understands a special HTTP Header jaeger-debug-id, which can be set in the incoming request, e.g.

curl -H "jaeger-debug-id: some-correlation-id" http://myhost.com

When Jaeger sees this header in the request that otherwise has no tracing context, it ensures that the new trace started for this request will be sampled in the "debug" mode (meaning it should survive all downsampling that might happen in the collection pipeline), and the root span will have a tag as if this statement was executed:

span.SetTag("jaeger-debug-id", "some-correlation-id")

This allows using Jaeger UI to find the trace by this tag.

Zipkin HTTP B3 compatible header propagation

Jaeger Tracer supports Zipkin B3 Propagation HTTP headers, which are used by a lot of Zipkin tracers. This means that you can use Jaeger in conjunction with e.g. these OpenZipkin tracers.

However it is not the default propagation format, see here how to set it up.

License

Apache 2.0 License.

mariusgiger/jaeger-client-go