Purge old keys without TTL from Redis
in03 opened this issue · 1 comments
Should be easy enough, but not high priority.
We should figure out how to purge old keys that don't have a TTL. Kombu bindings to old queues with old version constraints lay around until someone cleans them up, which for most end users will be never. Deleting queues that are still in use results in those queues being created again when Celery attempts to use them. There may be some slight overhead there, but no great loss.
There are also a few stray _kombu.binding.reply.celery.pidbox
keys. Ask Solem writes:
These replies are sent for remote control commands. I know flower is constantly sending out control commands, but I'm not sure there is an easy fix. The best option would be if the replies were using PUBSUB instead of being stored in a list.
The keys are very small, but assuming we can just keep them forever may not be realistic. Especially if users have made their Redis instance persistent. We could:
- Create a clean up command to remove old keys
- Clean up after x amount of program runs
- Clean up as part of install / upgrade setup
or a combination of the three.
Here's a sample of the keys from Redis Insight:
Now that we're using the native Celery AsyncResults instead of Redis pubsub, it might be a good idea to bundle this into some kind of general cleanup
command. I'm not sure if RabbitMQ and some of the other popular broker backends need occasional tidying.