Scrapy Memcached Cache
Memcached HTTP cache storage backend for Scrapy.
Install
Install the package with pipenv install scrapy-memcached-cache
or pip install memcached-cache
.
Usage
In your Scrapy settings.py
:
- Enable HTTP cache with
HTTPCACHE_ENABLED = True
- Set the cache expiration (in seconds), for example, with
HTTPCACHE_EXPIRATION_SECS = 600
- Set
MemcachedCacheStorage
as your cache storage withHTTPCACHE_STORAGE = 'scrapy_memcached_cache.MemcachedCacheStorage'
- Set the location where your Memcached is running, for example, with
MEMCACHED_LOCATION=localhost:11211
- Done
🎉
Contributing
Please, format your code with Black.