[Bug] Async Caching - object list can't be used in 'await' expression

[ishaan-jaff]

Hello, I'm trying to use async mode - seeing this bug, can I get help ? cc @tylerhutcherson

Code to repro

Create Index

def __init__(
        self,
        host=None,
        port=None,
        password=None,
        redis_url=None,
        similarity_threshold=None,
        use_async = False,
        **kwargs,
    ):
        from redisvl.index import SearchIndex
        from redisvl.query import VectorQuery

        print_verbose(
            "redis semantic-cache initializing INDEX - litellm_semantic_cache_index"
        )
        if similarity_threshold is None:
            raise Exception("similarity_threshold must be provided, passed None")
        self.similarity_threshold = similarity_threshold
        schema = {
            "index": {
                "name": "litellm_semantic_cache_index",
                "prefix": "litellm",
                "storage_type": "hash",
            },
            "fields": {
                "text": [{"name": "response"}],
                "text": [{"name": "prompt"}],
                "vector": [
                    {
                        "name": "litellm_embedding",
                        "dims": 1536,
                        "distance_metric": "cosine",
                        "algorithm": "flat",
                        "datatype": "float32",
                    }
                ],
            },
        }
        if redis_url is None:
            # if no url passed, check if host, port and password are passed, if not raise an Exception
            if host is None or port is None or password is None:
                raise Exception(f"Redis host, port, and password must be provided")
            redis_url = "redis://:" + password + "@" + host + ":" + port
        
        print_verbose(f"redis semantic-cache redis_url: {redis_url}")
        if use_async == False:
            self.index = SearchIndex.from_dict(schema)
            self.index.connect(redis_url=redis_url)
            try:
                self.index.create(overwrite=False)  # don't overwrite existing index
            except Exception as e:
                print_verbose(f"Got exception creating semantic cache index: {str(e)}")
        elif use_async == True:
            print("creating async semantic cache index")
            schema["index"]["name"] = "litellm_semantic_cache_index_async"
            self.index = SearchIndex.from_dict(schema, use_async=True)
            self.index.connect(redis_url=redis_url)
            try:
                self.index.create(overwrite=False)  # don't overwrite existing index
            except Exception as e:
                print_verbose(f"Got exception creating semantic cache index: {str(e)}")

Query index

        from redisvl.query import VectorQuery
        import numpy as np

        # query

        # get the messages
        messages = kwargs["messages"]
        prompt = ""
        for message in messages:
            prompt += message["content"]

        # convert to embedding
        embedding_response = await litellm.aembedding(
            model="text-embedding-ada-002",
            input=prompt,
            cache={"no-store": True, "no-cache": True},
        )

        print("async got embedding_response", embedding_response)

        # get the embedding
        embedding = embedding_response["data"][0]["embedding"]

        query = VectorQuery(
            vector=embedding,
            vector_field_name="litellm_embedding",
            return_fields=["response", "prompt", "vector_distance"],
        )
        print("async querying semantic cache", query)
        results = await self.index.aquery(query)
        print("async got results", results)

stacktrace

An exception occurred: Traceback (most recent call last):
  File "/Users/ishaanjaffer/Github/litellm/litellm/caching.py", line 972, in async_get_cache
    cached_result = await self.cache.async_get_cache(cache_key, messages=messages)
  File "/Users/ishaanjaffer/Github/litellm/litellm/caching.py", line 472, in async_get_cache
    results = await self.index.aquery(query)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/redisvl/index.py", line 106, in wrapper
    await check_async_redis_modules_exist(_redis_conn.client)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/redisvl/utils/utils.py", line 60, in check_async_redis_modules_exist
    installed_modules = await client.module_list()
TypeError: object list can't be used in 'await' expression

[ishaan-jaff]

It looks like this doc is wrong

I needed to do this:

self.index = SearchIndex.from_dict(schema, use_async=True)
self.index.connect(redis_url=redis_url, use_async=True)

[tylerhutcherson]

Hi @ishaan-jaff thanks for opening this! Will take a look today.

Fair warining -- we are rapidly approaching a 0.1.0 release, as up until now the package has been in pre-release mode. This new version will drop this week and contains a number of updates and improvements, including redis connection handling and async work.

You can see the latest state on the tip of this branch/PR:
#105

Apologies for the confusion. One of the core updates will be separating the SearchIndex and AsyncSearchIndex classes for ultimate clarity :)

[ishaan-jaff]

this was fixed by adding this to my code - but I think it should be added to your docs @tylerhutcherson

self.index = SearchIndex.from_dict(schema, use_async=True)
self.index.connect(redis_url=redis_url, use_async=True)

[tylerhutcherson]

@ishaan-jaff Given you are building this as an integration into your library, I'd strongly recommend waiting until our stable release at 0.1.0. You can pip install from the branch linked above.

The new and improved async flow will look like this:

from redisvl.index import AsyncSearchIndex

index = AsyncSearchIndex.from_dict(schema)
index.connect(redis_url)

await index.create()

await index.load(data)

...

[ishaan-jaff]

• when will 0.1.0 be out?

(I'll mark semantic caching as a beta-feature until then)

[tylerhutcherson]

• when will 0.1.0 be out?

(I'll mark semantic caching as a beta-feature until then)

Targeting thursday or friday this week.

[rohanrehman]

@tylerhutcherson Pylance keeps throwing errors in vscdode for all async and awaits.
"await" allowed only within async function Pylance