Access logs for Section modules are written to the container STDOUT
for collection by the Filebeat daemonset.
This module will intercept logs by reading from a fifo file provided
during a SetupYYYY method. It then parses log lines from which it
makes prometheues metrics. The logs unmodified are also written to an
io.Writer which is generally stdout.
This expects a JSON log format with one line per HTTP request logged with the following properties:
hostname- The value of theHostHTTP request header.status- The value of the response status code.bytesorbytes_sent- The total bytes (header + body) sent downstream by this module .
metrics_test.go has examples of valid log lines.
The metrics collected are:
section_http_request_count_total{ section_io_module_name="module name", status="200" }- Counter of number of HTTP requests by status.section_http_bytes_total{ section_io_module_name="module name", status="200" }- Counter of sum of bytes sent downstream by status.section_http_json_parse_errors_total{ section_io_module_name="module name" }- Counter of the number of times it has been unable to JSON parse a log line.section_http_request_count_by_hostname_total{ hostname="www.example.com" }- Counter of the number of HTTP requests by hostname.section_http_bytes_by_hostname_total{ hostname="www.example.com" }- Counter of sum of bytes sent downstream by hostname.
The by_hostname metrics will only be generated if hostname is included in the additional labels parameter.
The section_io_module_name is configured as a target label on the
service monitor for the module using this module.
The metrics are published as a Prometheus exporter by default on port
9000 with the path /metrics.
Metrics can have additional labels added based on fields in the log
lines. These can be added as a string array when setting up the module
(see code below). The values will pass through the sanitizeValue
function in metrics.go so that fields that have known sanitization
issues (like stripping the additional fields from content_type.)
Additional sanitization can be added to this function as needed. The
additional labels need to be valid Prometheus labels (see
https://prometheus.io/docs/concepts/data_model/#metric-names-and-labels).
This is required, but not currently enforced by the code. The
additional labels also need to match the name of a field in the log
lines. Lines that do not have the field will create a metric with a
blank label value.
There are two ways to use the module.
This will setup the FIFO file at pathToLogFile, start the metrics
collection and redirect all content written the log file to
os.Stdout. Any errors will be written to os.Stderr and the
metrics will have the additional label of content_type.
```
import metrics "github.com/section-io/module-metrics"
...
err := metrics.SetupModule(pathToLogFile, os.Stdout, os.Stderr, "content_type")
```
If the logs are already in an io.Reader you can setup the metrics
and start the reader explicitly.
```
metrics.InitMetrics("content_type")
metrics.StartReader(logReader, os.Stdout, os.Stderr)
```
The setup for GeoIP hashes uses the method SetupWithGeoHash which is
very similar to SetupModule. The difference is that the Geo Hash
precision must be provided. The precision represent an output of 1-12
Geo Hash characters. Since these hash characters will result in
labels on request metrics it's important to choose a number that won't
adversely effect your prometheus scraping. A reasonable starting
point is 2 characters of precision and then slowly grow from there
keeping in mind an exponential growth.
Once we merge changes from feature branch to master after code review & approval, we need to manually tag it
# checkout master and fetch latest merged changes
git checkout master
git pull
# create and push tag ( version in semver format )
git tag -a vX.Y.Z -m'Short message describing change'
git push origin --tags
To create a new release for this module, click on the new release link and provide relevant release details and publish.