[Bug]: tpm/rmp of llm model not managed by litellm in memoryb redis cluster
Opened this issue · 1 comments
What happened?
i have impletemented the aws memorydb redis cluster for production env to manage tpm/rpm but in redis cluster i am not able to see any data. it is showing empty array in redis cluster on port 1992. please any info regarding this will be helpful
Relevant log output
No response
Twitter / LinkedIn details
No response
@ishaan-jaff @krrishdholakia
Adding to this, just a quick question
We have set up Redis caching in the LiteLLM_settings for our proxy, which is deployed on AWS ECS/Fargate with multiple replicas. My question is, do I need to also define Redis caching details separately in the router_settings, or is the existing Redis configuration in LiteLLM_settings is sufficient across all replicas? Would it be necessary to create a separate Redis cluster specifically for the router?
Thanks in advance for your guidance!