Error on ingesting out-of-order samples
gcarmin opened this issue · 6 comments
I have a 3 node C*4 cluster.
Running docker-compse up works
System & Node Metrics works
Cassandra Cluster Condensed doesn't get matrics.
I see in the terminal the following err:
mcac-dashboards_prometheus_1 | level=warn ts=2020-06-08T07:17:05.824Z caller=scrape.go:1234 component="scrape manager" scrape_pool=mcac target=http://172.31.64.165:9103/metrics msg="Error on ingesting out-of-order samples" num_dropped=3
running all the time.
Hi,
That error is mostly harmless, happens when the collectd metrics haven't been updated quickly enough.
The missing metrics in cluster condensed is a bigger issue. Can you tell me how many tables your cluster has?
Hi - So it is a fresh install cluster with the system tables only.....
Are you using a specific build of C* 4?
I'm using the RPM from here:
https://downloads.apache.org/cassandra/redhat/40x/
cassandra-4.0~alpha4-1.noarch.rpm
I got the same log messages.
Collecting from Cassandra 3.11.x cluster of 5 nodes. There are around 20 tables.
I can see that some of the metrics are not updated correctly, for e.g. node status.
Any updates on this issue ?