python-diamond/Diamond

MongoDBCollector reduce sending data

greggy opened this issue · 1 comments

Hello!

I'm using diamond to collect mongodb metrics. I've build docker container with mongodb 3.6 and diamond. It sends metrics via handler GraphiteHandler and collector config is:

[[MongoDBCollector]]
enabled = True
hosts = localhost:27017
databases = production_db
ignore_collections = ^(.*_logs|sessions)
simple = True

Diamond flushes metrics every 10 seconds, in logs I see:

[2020-10-04 16:00:49,276] [ERROR] [MainThread:MongoDBCollector] Took too long to run! Killed!
...
[2020-10-04 16:03:48,101] [DEBUG] [MainThread:MongoDBCollector] Collection took 7172 ms

How to reduce number of sending metrics? Or I have another way to optimize it?
Thank you!

I am guessing one of two things are happening. Either your mongo server is slow to respond to the collector queries or you have a lot of collections to iterate over. You should be able to see the slow queries on your mongo server. Reopen if this doesn't answer your question.