BerriAI/liteLLM-proxy

Server failing to start while setting for Redis Semantic Cache

Opened this issue · 0 comments

Getting the following error while deploying the proxy server on kubernetes cluster with semantic cache set as true.

INFO: Started server process [1]
INFO: Waiting for application startup.
ERROR: Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 732, in lifespan
async with self.lifespan_context(app) as maybe_state:
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 608, in aenter
await self._router.startup()
File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 709, in startup
await handler()
File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 3061, in startup_event
await initialize(**worker_config)
File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 2905, in initialize
) = await proxy_config.load_config(router=llm_router, config_file_path=config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 2059, in load_config
self._init_cache(cache_params=cache_params)
File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 1960, in _init_cache
litellm.cache = Cache(**cache_params)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/litellm/caching.py", line 1326, in init
self.cache = RedisSemanticCache(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/litellm/caching.py", line 582, in init
from redisvl.index import SearchIndex
ModuleNotFoundError: No module named 'redisvl'

ERROR: Application startup failed. Exiting.

#------------------------------------------------------------#

'This feature doesn't meet my needs because...'

https://github.com/BerriAI/litellm/issues/new

#------------------------------------------------------------#

Thank you for using LiteLLM! - Krrish & Ishaan

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new