sidekiq-datadog

Datadog instrumentation for Sidekiq, integrated via server middleware.

Installation

Add this line to your application's Gemfile:

gem 'sidekiq-datadog'

Or install:

$ gem install sidekiq-datadog

Configure it in an initializer:

Sidekiq.configure_client do |config|
  config.client_middleware do |chain|
    chain.add Sidekiq::Middleware::Client::Datadog
  end
end

Sidekiq.configure_server do |config|
  config.client_middleware do |chain|
    chain.add Sidekiq::Middleware::Client::Datadog
  end

  config.server_middleware do |chain|
    chain.add Sidekiq::Middleware::Server::Datadog
  end
end

Options

Options can be configured to be passed to the middleware constructor when it is added to the chain

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add(Sidekiq::Middleware::Server::Datadog, statsd_port: 3334)
  end
end

Custom tags can be configured using the tags: property

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add(Sidekiq::Middleware::Server::Datadog, tags: ['runtime:jruby'])
  end
end

Dynamic tags can be configured by passing a lambda in the tags array. It is executed at runtime when the job is processed

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add(Sidekiq::Middleware::Server::Datadog, tags: [->(worker_or_worker_class, job, queue, error){
      "source:#{job['source']}"
    }])
  end
end

# NOTE: Your lambda will either receive a `Worker` object for the Server middleware, 
# or a String with the `worker_class` for the Client middleware. 
# If you are using that argument, your lambda should be able to handle both cases.

You can supress some of the default tags from being emitted by passing in skip_tags. This is also useful if you would like to change one of the default tags, you can define a custom lambda and define it as skip_tags

Sidekiq.configure_server do |config|
  config.server_middleware do |chain|
    chain.add(Sidekiq::Middleware::Server::Datadog,
        skip_tags: ["name"], 
        tags: [->(worker_or_worker_class, job, queue, error){
            "name:#{ my_logic_for_name }"
        }])
  end
end

supported options

Both Client and Server middlewares support the same options:

hostname - the hostname used for instrumentation, defaults to system hostname. Can also be set with the INSTRUMENTATION_HOSTNAME env var.
metric_name - the metric name (prefix) to use, defaults to "sidekiq.job".
tags - array of custom tags. These can be plain strings or lambda blocks.
skip_tags - array of tag names that shouldn't be emitted.
statsd_host - the statsD host, defaults to "localhost". Can also be set with the STATSD_HOST env var
statsd_port - the statsD port, defaults to 8125. Can also be set with the STATSD_PORT env var
statsd - custom statsd instance

For more detailed configuration options, please see the Documentation.

Metrics exposed

The client middleware will expose:

sidekiq.job_enqueued counter, with tags: host, env, name (the job name) and queue

The server middleware will expose:

sidekiq.job counter, with tags: host, env, name (the job name), queue, and status (ok or error). If status is error, there will be an additional error tag with the exception class.
sidekiq.job.time timing (ms) metric with the same tags, specifying the job runtime.
sidekiq.job.queued_time timing (ms) metric with the same tags, specifying how long the job was waiting in the queue before starting.

The base metric names sidekiq.job and sidekiq.job_enqueued can be overriden using the metric_name option.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Make a pull request

dogstatsd-ruby Version note: dogstatsd-ruby has major backwards incompatibilities between v4.8.3 and 5.0.0. This gem is compatible with both, and CI tests both versions.

In order to test through both, we have 2 Gemfiles (and their respective .lock files):

Gemfile
Gemfile_dogstats_v4

By default, everything runs against dogstatsd-ruby >= 5.0.

To update Gemfile_dogstats_v4.lock, or run tests with c4.8:

BUNDLE_GEMFILE=Gemfile_dogstats_v4 bundle update