SemanticCache with filterable_fields - consider extra metadata for creating the cache entry key

[gacerioni]

They @tylerhutcherson and @justin-cechmanek - Hope you are doing well!

As we discussed:

When I try to store some semantic caches with filterable_fields (as in the docs example):

private_cache = SemanticCache(
    name="private_cache",                     # underlying search index name
    redis_url="redis://default:blabla@redis-16962.c11.us-east-1-2.ec2.cloud.redislabs.com:16962",  # redis connection url string
    distance_threshold=0.1,               # semantic cache distance threshold
    filterable_fields=[{"name": "user_id", "type": "tag"}]
)

private_cache.store(
    prompt="What is the phone number linked to my account?",
    response="The number on file is 123-555-0000",
    filters={"user_id": "gabs"},
)

private_cache.store(
    prompt="What's the phone number linked in my account?",
    response="The number on file is 123-555-9999",
    filters={"user_id": "cerioni"},
)

private_cache.store(
    prompt="What's the phone number linked in my account?",
    response="The number on file is 123-555-1111",
    filters={"user_id": "bart"},
)

It only stores the 1st and last items (gabs and bart) - keeping this cap of 2 items.
Is there a way to override this limit?

Or, as discussed: utilize a purely random UUID for the key (if this is possible).

Thanks!

[justin-cechmanek]

Hi @gacerioni. We've merged a fix to address this. Cache entries should no longer overwrite each other unless both the prompt and all metadata are identical.