upstream servers requests count intermittently reporting spurious metrics
andrewdike opened this issue · 2 comments
upstream servers requests count intermittently reporting spurious metrics
When retrieving the metric nginxplus_upstream_server_requests, intermittently very high values are received. E.g for a system receiving <10 requests per second, the counter can jump from 50 to 850,000 after 2 seconds. The counter can also decrease which is not how the counters should behave according to Prometheus docs.
To reproduce
As it happens intermittently, it can't be programmatically reproduced. In my environment they are hundreds of backends on a single nginx instance and I can graph nginxplus_upstream_server_requests over time (an hour is normally enough) and look for anomalies. These include lines that decrease in value or spike before returning to their true value. See below for an example.
Expected behavior
The counter metric should only increase, and be reset when they reach their maximum value. There shouldn't be extremely high variations from second to second.
Your environment
- Version of the Prometheus exporter - 0.9.0
- Running using the compiled binary on ubuntu 18.04.5
- Version:
# nginx -v
nginx version: nginx/1.21.3 (nginx-plus-r25)
This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 10 days.
This issue was closed because it has been stalled for 10 days with no activity.