aiohttp-client-cache

See full documentation at https://aiohttp-client-cache.readthedocs.io

aiohttp-client-cache is an async persistent cache for aiohttp requests, based on requests-cache.

Not to be confused with aiohttp-cache, which is a cache for the aiohttp web server. This package is, as you might guess, specifically for the aiohttp client.

Development Status

This is an early work in progress!

The current state is a working drop-in replacement (or mixin) for aiohttp.ClientSession, with a fully async SQLite backend. Additional backends are provided, but cache operations are still synchronous, and have had minimal testing.

Breaking changes should be expected until a 1.0 release.

Installation

Requires python 3.7+

Install the latest stable version with pip:

pip install aiohttp-client-cache

Note: You will need additional dependencies depending on which backend you want to use; See Cache Backends section below for details. To install with extra dependencies for all supported backends:

pip install aiohttp-client-cache[backends]

To set up for local development:

$ git clone https://github.com/JWCook/aiohttp-client-cache
$ cd aiohttp-client-cache
$ pip install -Ue ".[dev]"
$ # Optional but recommended:
$ pre-commit install --config .github/pre-commit.yml

Usage example

See the examples folder for more detailed usage examples.

Here is a simple example using an endpoint that takes 1 second to fetch. After the first request, subsequent requests to the same URL will return near-instantly; so, fetching it 10 times will only take ~1 second instead of 10.

from aiohttp_client_cache import CachedSession, SQLiteBackend

async with CachedSession(cache=SQLiteBackend()) as session:
    for i in range(10):
        await session.get('http://httpbin.org/delay/1')

aiohttp-client-cache can also be used as a mixin, if you happen have other mixin classes that you want to combine with it:

from aiohttp import ClientSession
from aiohttp_client_cache import CacheMixin

class CustomSession(CacheMixin, CustomMixin, ClientSession):
    pass

Cache Backends

Several backends are available. If one isn't specified, a simple in-memory cache will be used.

SQLiteBackend: Uses a SQLite database (requires aiosqlite)
DynamoDBBackend: Uses a Amazon DynamoDB database (requires boto3)
RedisBackend: Uses a Redis cache (requires redis-py)
MongoDBBackend: Uses a MongoDB database (requires pymongo)
- GridFSBackend: Uses a MongoDB GridFS database, which enables storage of documents greater than 16MB (requires pymongo)

You can also provide your own backend by subclassing aiohttp_client_cache.backends.BaseCache.

Expiration

If you are using the expire_after parameter, expired responses are removed from the storage the next time the same request is made. If you want to manually purge all expired items, you can use CachedSession.delete_expired_responses. Example:

session = CachedSession(expire_after=3)   # Cached responses expire after 3 hours
await session.remove_expired_responses()  # Remove any responses over 3 hours old

Conditional Caching

Caching behavior can be customized by defining various conditions:

Response status codes
Request HTTP methods
Request headers
Specific request parameters
Custom filter function

See CacheBackend docs for details.

Credits

Thanks to Roman Haritonov and contributors for the original requests-cache!

This project is licensed under the MIT license, with the exception of storage backend code adapted from requests-cache, which is licensed under the BSD license (copy included).

jophish/aiohttp-client-cache