Cleanup monitoring DB
Closed this issue · 4 comments
martinkersner commented
redis_aggregator_completed
742M rowsredis_aggregator_failed
63M rowsredis_request_response_completed
30M rowsredis_request_response_failed
13K rowsredis_vrf_completed
38M rowsredis_vrf_failed
15K rows
jay-kim-kor commented
@martinkersner We don't need to keep redis_*_*
table even it's related to VRF or RR? If so, I'll remove old data from this table when I clean-up monitor DB.
martinkersner commented
@martinkersner We don't need to keep
redis_*_*
table even it's related to VRF or RR? If so, I'll remove old data from this table when I clean-up monitor DB.
All of these tables are seem to be persistent storage or Redis, so I am not sure we can delete it through DB directly without affecting Redis itself.
- Jobs that are
completed
are not used for anything, not even visualization -> we can clean them up - Jobs that are
failed
(as far as I know) are not going to be retried by BullMQ anymore, BullMQ gave up on such jobs. If that is true, we can clean them all up.
If we have some regular cleaning schedule that would be awesome 💪
jay-kim-kor commented
TODO
- If it can remove directly, remove it by SQL cron.
- If can’t, use redis-cli
- Test to remove old data
- Add k8s cronjob
nick-bisonai commented
monitor db will be deprecated after orakl-sentinel implementation