/prometheus-nginxlog-exporter

Export metrics from Nginx access log files to Prometheus

Primary LanguageGoApache License 2.0Apache-2.0

NGINX-to-Prometheus log file exporter

Helper tool that continuously reads an NGINX log file and exports metrics to Prometheus.

Usage

You can either use a simple configuration, using command-line flags, or create a configuration file with a more advanced configration.

Use the command-line:

./nginx-log-exporter \
  -format="<FORMAT>" \
  -listen-port=4040 \
  -namespace=nginx \
  [PATHS-TO-LOGFILES...]

Use the configuration file:

./nginx-log-exporter -config-file /path/to/config.hcl

Collected metrics

This exporter collects the following metrics. This collector can listen on multiple log files at once and publish metrics in different namespaces. Each metric uses the labels method (containing the HTTP request method) and status (containing the HTTP status code).

  • <namespace>_http_response_count_total - The total amount of processed HTTP requests/responses.
  • <namespace>_http_response_size_bytes - The total amount of transferred content in bytes.
  • <namespace>_http_upstream_time_seconds - A summary vector of the upstream response times in seconds. Logging these needs to be specifically enabled in NGINX using the $upstream_response_time variable in the log format.
  • <namespace>_http_response_time_seconds - A summary vector of the total response times in seconds. Logging these needs to be specifically enabled in NGINX using the $request_time variable in the log format.

Additional labels can be configured in the configuration file (see below).

Configuration file

You can specify a configuration file to read at startup. The configuration file is expected to be in HCL format. Here's an example file:

listen {
  port = 4040
  address = "10.1.2.3"
}

consul {
  enable = true
  address = "localhost:8500"
  service {
    id = "nginx-exporter"
    name = "nginx-exporter"
    datacenter = "dc1"
    scheme = "http"
    token = ""
    tags = ["foo", "bar"]
  }
}

namespace "app-1" {
  format = "$remote_addr - $remote_user [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" \"$http_x_forwarded_for\""
  source_files = [
    "/var/log/nginx/app1/access.log"
  ]
  labels {
    app = "application-one"
    environment = "production"
    foo = "bar"
  }
}

namespace "app-2" {
  format = "$remote_addr - $remote_user [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" \"$http_x_forwarded_for\" $upstream_response_time"
  source_files = [
    "/var/log/nginx/app2/access.log"
  ]
}

Running the collector

Systemd

You can find an example unit file for this service in this repository. Simply copy the unit file to /etc/systemd/system:

$ wget -O /etc/systemd/system/prometheus-nginxlog-exporter.service https://raw.githubusercontent.com/martin-helmich/prometheus-nginxlog-exporter/master/systemd/prometheus-nginxlog-exporter.service
$ systemctl enable prometheus-nginxlog-exporter
$ systemctl start prometheus-nginxlog-exporter

The shipped unit file expects the binary to be located in /usr/local/bin/prometheus-nginxlog-exporter and the configuration file in /etc/prometheus-nginxlog-exporter.hcl. Adjust to your own needs.

Credits