Histogram bounds don't seem to behave like in the examples
Closed this issue · 2 comments
sashaafm commented
I've been trying to implement some histograms in my metrics but the bounds never seem to match the ones I defined initially. Even by copying your histogram example it doesn't seem to work the same way:
:prometheus_histogram.new([{:name, :http_request_duration_milliseconds}, {:labels, [:method]}, {:bounds, [100, 300, 500, 750, 1000]}, {:help, "Http Request execution time"}])
:ok
iex(2)> :prometheus_histogram.observe(:http_request_duration_milliseconds, [:get], 95) :ok
iex(3)> :prometheus_histogram.observe(:http_request_duration_milliseconds, [:get], 102)
:ok
iex(4)> :prometheus_histogram.observe(:http_request_duration_milliseconds, [:get], 150)
:ok
iex(5)> :prometheus_histogram.observe(:http_request_duration_milliseconds, [:get], 250)
:ok
iex(6)> :io.format :prometheus_text_format.format
# TYPE erlang_vm_ets_limit gauge
# HELP erlang_vm_ets_limit The maximum number of ETS tables allowed.
erlang_vm_ets_limit 2053
# TYPE erlang_vm_logical_processors gauge
# HELP erlang_vm_logical_processors The detected number of logical processors configured in the system.
erlang_vm_logical_processors 4
# TYPE erlang_vm_logical_processors_available gauge
# HELP erlang_vm_logical_processors_available The detected number of logical processors available to the Erlang runtime system.
erlang_vm_logical_processors_available NaN
# TYPE erlang_vm_logical_processors_online gauge
# HELP erlang_vm_logical_processors_online The detected number of logical processors online on the system.
erlang_vm_logical_processors_online 4
# TYPE erlang_vm_port_count gauge
# HELP erlang_vm_port_count The number of ports currently existing at the local node.
erlang_vm_port_count 29
# TYPE erlang_vm_port_limit gauge
# HELP erlang_vm_port_limit The maximum number of simultaneously existing ports at the local node.
erlang_vm_port_limit 65536
# TYPE erlang_vm_process_count gauge
# HELP erlang_vm_process_count The number of processes currently existing at the local node.
erlang_vm_process_count 310
# TYPE erlang_vm_process_limit gauge
# HELP erlang_vm_process_limit The maximum number of simultaneously existing processes at the local node.
erlang_vm_process_limit 262144
# TYPE erlang_vm_schedulers gauge
# HELP erlang_vm_schedulers The number of scheduler threads used by the emulator.
erlang_vm_schedulers 4
# TYPE erlang_vm_schedulers_online gauge
# HELP erlang_vm_schedulers_online The number of schedulers online.
erlang_vm_schedulers_online 4
# TYPE erlang_vm_smp_support untyped
# HELP erlang_vm_smp_support 1 if the emulator has been compiled with SMP support, otherwise 0.
erlang_vm_smp_support 1
# TYPE erlang_vm_threads untyped
# HELP erlang_vm_threads 1 if the emulator has been compiled with thread support, otherwise 0.
erlang_vm_threads 1
# TYPE erlang_vm_thread_pool_size gauge
# HELP erlang_vm_thread_pool_size The number of async threads in the async thread pool used for asynchronous driver calls.
erlang_vm_thread_pool_size 10
# TYPE erlang_vm_time_correction untyped
# HELP erlang_vm_time_correction 1 if time correction is enabled, otherwise 0.
erlang_vm_time_correction 1
# TYPE erlang_vm_statistics_context_switches counter
# HELP erlang_vm_statistics_context_switches Total number of context switches since the system started
erlang_vm_statistics_context_switches 173394
# TYPE erlang_vm_statistics_garbage_collection_number_of_gcs counter
# HELP erlang_vm_statistics_garbage_collection_number_of_gcs Garbage collection: number of GCs
erlang_vm_statistics_garbage_collection_number_of_gcs 11111
# TYPE erlang_vm_statistics_garbage_collection_words_reclaimed counter
# HELP erlang_vm_statistics_garbage_collection_words_reclaimed Garbage collection: words reclaimed
erlang_vm_statistics_garbage_collection_words_reclaimed 50897993
# TYPE erlang_vm_statistics_garbage_collection_bytes_reclaimed counter
# HELP erlang_vm_statistics_garbage_collection_bytes_reclaimed Garbage collection: bytes reclaimed
erlang_vm_statistics_garbage_collection_bytes_reclaimed 407183944
# TYPE erlang_vm_statistics_bytes_received_total counter
# HELP erlang_vm_statistics_bytes_received_total Total number of bytes received through ports
erlang_vm_statistics_bytes_received_total 28204523
# TYPE erlang_vm_statistics_bytes_output_total counter
# HELP erlang_vm_statistics_bytes_output_total Total number of bytes output to ports
erlang_vm_statistics_bytes_output_total 5403240
# TYPE erlang_vm_statistics_reductions_total counter
# HELP erlang_vm_statistics_reductions_total Total reductions
erlang_vm_statistics_reductions_total 25234130
# TYPE erlang_vm_statistics_run_queues_length_total gauge
# HELP erlang_vm_statistics_run_queues_length_total Total length of the run-queues
erlang_vm_statistics_run_queues_length_total 0
# TYPE erlang_vm_statistics_runtime_milliseconds counter
# HELP erlang_vm_statistics_runtime_milliseconds The sum of the runtime for all threads in the Erlang runtime system. Can be greater than wall clock time
erlang_vm_statistics_runtime_milliseconds 3590
# TYPE erlang_vm_statistics_wallclock_time_milliseconds counter
# HELP erlang_vm_statistics_wallclock_time_milliseconds Information about wall clock. Same as erlang_vm_statistics_runtime_milliseconds except that real time is measured
erlang_vm_statistics_wallclock_time_milliseconds 46868
# TYPE erlang_vm_memory_atom_bytes_total gauge
# HELP erlang_vm_memory_atom_bytes_total The total amount of memory currently allocated for atoms. This memory is part of the memory presented as system memory.
erlang_vm_memory_atom_bytes_total{usage="used"} 974501
erlang_vm_memory_atom_bytes_total{usage="free"} 17932
# TYPE erlang_vm_memory_bytes_total gauge
# HELP erlang_vm_memory_bytes_total The total amount of memory currently allocated. This is the same as the sum of the memory size for processes and system.
erlang_vm_memory_bytes_total{kind="system"} 52162768
erlang_vm_memory_bytes_total{kind="processes"} 24534512
# TYPE erlang_vm_dets_tables gauge
# HELP erlang_vm_dets_tables Erlang VM DETS Tables count
erlang_vm_dets_tables 0
# TYPE erlang_vm_ets_tables gauge
# HELP erlang_vm_ets_tables Erlang VM ETS Tables count
erlang_vm_ets_tables 74
# TYPE erlang_vm_memory_processes_bytes_total gauge
# HELP erlang_vm_memory_processes_bytes_total The total amount of memory currently allocated for the Erlang processes.
erlang_vm_memory_processes_bytes_total{usage="used"} 24532544
erlang_vm_memory_processes_bytes_total{usage="free"} 1968
# TYPE erlang_vm_memory_system_bytes_total gauge
# HELP erlang_vm_memory_system_bytes_total The total amount of memory currently allocated for the emulator that is not directly related to any Erlang process. Memory presented as processes is not included in this memory.
erlang_vm_memory_system_bytes_total{usage="atom"} 992433
erlang_vm_memory_system_bytes_total{usage="binary"} 862136
erlang_vm_memory_system_bytes_total{usage="code"} 28049221
erlang_vm_memory_system_bytes_total{usage="ets"} 2450216
erlang_vm_memory_system_bytes_total{usage="other"} 19808762
# TYPE http_request_duration_milliseconds histogram
# HELP http_request_duration_milliseconds Http Request execution time
http_request_duration_milliseconds_bucket{method="get",le="0.005"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.01"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.025"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.05"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.1"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.25"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.5"} 4
http_request_duration_milliseconds_bucket{method="get",le="1.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="2.5"} 4
http_request_duration_milliseconds_bucket{method="get",le="5.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="10.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="+Inf"} 4
http_request_duration_milliseconds_count{method="get"} 4
http_request_duration_milliseconds_sum{method="get"} 5.97e-4
Why am I getting this:
http_request_duration_milliseconds_bucket{method="get",le="0.005"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.01"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.025"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.05"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.1"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.25"} 4
http_request_duration_milliseconds_bucket{method="get",le="0.5"} 4
http_request_duration_milliseconds_bucket{method="get",le="1.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="2.5"} 4
http_request_duration_milliseconds_bucket{method="get",le="5.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="10.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="+Inf"} 4
If I defined these bounds: [100, 300, 500, 750, 1000]. Shouldn't I get a line for each bound in that array? Could this be an Elixir incompatibility when using the Erlang driver?
sashaafm commented
It seems using the prometheus.ex driver works correctly?
iex(3)> use Prometheus.Metric
[Prometheus.Metric.Counter, Prometheus.Metric.Gauge,
Prometheus.Metric.Histogram, Prometheus.Metric.Summary,
Prometheus.Metric.Boolean]
iex(4)> Histogram.new([name: :http_request_duration_milliseconds,
...(4)> labels: [:method],
...(4)> buckets: [100, 300, 500, 750, 1000],
...(4)> help: "Http Request execution time"])
:ok
iex(5)> Histogram.observe([name: :http_request_duration_milliseconds, labels: [:get]], 95)
:ok
iex(6)> Histogram.observe([name: :http_request_duration_milliseconds, labels: [:get]], 102)
:ok
iex(7)> Histogram.observe([name: :http_request_duration_milliseconds, labels: [:get]], 115)
:ok
iex(8)> Histogram.observe([name: :http_request_duration_milliseconds, labels: [:get]], 155)
:ok
# TYPE http_request_duration_milliseconds histogram
# HELP http_request_duration_milliseconds Http Request execution time
http_request_duration_milliseconds_bucket{method="get",le="100.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="300.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="500.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="750.0"} 4
http_request_duration_milliseconds_bucket{method="get",le="1.0e3"} 4
http_request_duration_milliseconds_bucket{method="get",le="+Inf"} 4
http_request_duration_milliseconds_count{method="get"} 4
http_request_duration_milliseconds_sum{method="get"} 4.67e-4
So is this truly a compatibility error?
deadtrickster commented
Readme was a bit outdated - it's buckets not bounds in erlang lib too.
PS. do not forget to read this: https://github.com/deadtrickster/prometheus.erl/blob/master/doc/prometheus_time.md