[Bug] Cache implementation on BE isn't working as expected
Closed this issue · 5 comments
I got the proxy to work in react and now i am able to hit the /random_game endpoint but the create react app is setup so that when you save the file, it automatically refreshes the page. For some reason I got this long message from the BE :
HTTPSConnectionPool(host='boardgamegeek.com', port=443): Max retries exceeded with url: /xmlapi2/collection?username=JDGiardino (Caused by ResponseError('too many 429 error responses'))
Ugh yeah I was experiencing something similar where I was running locally and hit random game too many times it would take awhile to load and hang there for a bit like this :
And than would load with that same error :
I DO know that cacheing worked at some point. Where I could just spam that submit button and would keep regenerating and never hang. I only noticed this when I included the image of the game. But this is definitely a bug to look into!
It might be that the @cached decorator here: https://github.com/JDGiardino/BGG-Companion/blob/main/src/bgg_companion_api.py#L23-L24 is caching the exception being thrown, when instead we only want to cache the parsed XML from these lines: https://github.com/JDGiardino/BGG-Companion/blob/main/src/bgg_companion_api.py#L25-L26
Perhaps what we could do instead is use the lower-level cache API and check the cache before making the request. We'd remove the function decorator and use the cachetools.TTLCache cache directly, like so:
cache = cachetools.TTLCache(maxsize=500, ttl=300)
def get_collection(self, user: str) -> dict:
xml_parse = cache.get(user)
if xml_parse is None:
string_xml = self.request(f"https://boardgamegeek.com/xmlapi2/collection?username={user}")
xml_parse = xmltodict.parse(string_xml)
cache[user] = xml_parse
# ...