Smart CacheStores
Opened this issue · 1 comments
Surprisingly, with all the choices cachetools made with extendibility in mind, the choice of the dict as the cache store is hard-coded.
Indeed "caching in memory" might be the most frequent case, but one can cache even "lower" (e.g. the three CPU cache levels), or higher: If you don't want to call a remote service every time you need some data, you'd better cache it (in a persisting way) locally.
What we need (or at least can use) is the ability to use wrap a store so that it acts as a cache endowed with some "smart" features such as TTL (time to live), or other processes that are triggered by the cache itself, given some conditions.
Targeted usage
Time based conditions such as TTL.
Different datas require different refresh/update rates, so there is also a need to specify cache parameters at the key level.
Keep the following use case in mind:
- Refreshing a data source regularly (when data sources need to be updated (like the covid data during the pandemic)).
Cached keys.
cached_keys (formerly "cache_iter") allows one to keep the keys in memory, but at the point of writing this, needed to iterate through the keys first in order to know what they are. We'd like to also be able to use the same mechanisms (in a separate, or the same decorator) to provide the source of the keys as a store. This would allow one to both persist and retrieve the keys from a file for example.
Keep the following use cases in mind:
- Cached filtering (i.e. indexing?)
- An instance of which: FlatZipFilesReader with ExplicitKeymapReader