Improve cache mechanism
szyb opened this issue · 3 comments
szyb commented
Due to the large increase in inquiries to Synopackage we need to improve our cache mechanism to minimize number of requests to the source servers. For now, caching responses from the servers is setup for CPU & DSM version (build number) & channel (stable/bet) and expiration is set to 10h.
- Cache expiration should be able to be set per repository as well as globally. This should be released as soon as possible.
- Fix health checks
- Caching per CPU, DSM version & channel might not be accruable to all repositories. This might be changed (need to investigate first)
- Add header If-Modified-Since for every GET request
- Consider using DSM major[.minor] version (instead of build number) to cache responses
- GET request to the server should be default HTTP method (not related to the issue, but it can be done together)
- Store cache files in separate folders for different sources
- Add worker to create cache subfolders (when above checkpoint will be completed)
- Add cleanup worker (on startup) to remove cached files older than X days for active repositories (leave cache files for inactive sources - if any)
szyb commented
A temprorary hack for filebot has been deployed with 1.0.7 version
szyb commented
GET request to the server should be default HTTP method (not related to the issue, but it can be done together) is not accurate for all (legacy) repositories