Getting a `KeyError` in `cleanup`

[aconrad]

I'm getting a KeyError once in a while (but not always). Here's my code:

    @lru_cache_function(max_size=1024, expiration=60)
    def shard_resolver(*resolver_args, **resolver_kwargs):
        # ... my code ...

And here is the traceback (removed irrelevant bits):

<snip>
    shard_id = shard_resolver(*resolver_args, **resolver_kwargs)
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 249, in __call__
    self.cache[key] = value
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 34, in withlock
    return func(self, *args, **kwargs)
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 160, in __setitem__
    self.cleanup()
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 34, in withlock
    return func(self, *args, **kwargs)
  File "/opt/webapp/txtasvc/local/lib/python2.7/site-packages/lru/__init__.py", line 185, in cleanup
    if self.__expire_times[k] < t:
KeyError: '((74300113,), {})#shard_resolver'

Since it happen only infrequently, I would suspect it's exposes a race condition bug somewhere? Any idea?

[aconrad]

When taking a closer look at the code, it looks like the key disappears between the time we iterate over it on line 184 and by the time we access it on line 185, which is crazy! I have a hard time wrapping my head around why this would happen. But since threading is involved, you never know...

https://github.com/stucchio/Python-LRU-cache/blob/master/lru/__init__.py#L184-L185

This would explain why this is a rare occurrence.

[aconrad]

Anyone?

[sontek]

@aconrad By default this library doesn't expect to be in a threaded environment. You have to pass concurrent=True if you want threadlocks put in place to prevent this error.

Docs for that are here:
https://github.com/stucchio/Python-LRU-cache/blob/master/lru/__init__.py#L64-L66