Fallback gracefully when pricing data is missing

Question

Fallback gracefully when pricing data is missing

lajarre opened this issue 2 years ago · 0 comments

Traceback:

2022-08-03T20:12:15.285237+00:00 app[web.1]: 10.1.35.190:19920 - "GET /api/portfolio/0x1a9c8182c09f50c8318d769245bea52c32be35bc/2021-08-02 HTTP/1.1" 500
2022-08-03T20:12:15.286006+00:00 app[web.1]: [2022-08-03 20:12:15 +0000] [14] [ERROR] Exception in ASGI application
2022-08-03T20:12:15.286008+00:00 app[web.1]: Traceback (most recent call last):
2022-08-03T20:12:15.286009+00:00 app[web.1]: File "pandas/_libs/index.pyx", line 545, in pandas._libs.index.DatetimeEngine.get_loc
2022-08-03T20:12:15.286009+00:00 app[web.1]: File "pandas/_libs/hashtable_class_helper.pxi", line 2131, in pandas._libs.hashtable.Int64HashTable.get_item
2022-08-03T20:12:15.286010+00:00 app[web.1]: File "pandas/_libs/hashtable_class_helper.pxi", line 2140, in pandas._libs.hashtable.Int64HashTable.get_item
2022-08-03T20:12:15.286011+00:00 app[web.1]: KeyError: 1659398400000000000
2022-08-03T20:12:15.286012+00:00 app[web.1]:
2022-08-03T20:12:15.286012+00:00 app[web.1]: During handling of the above exception, another exception occurred:
2022-08-03T20:12:15.286013+00:00 app[web.1]:
2022-08-03T20:12:15.286013+00:00 app[web.1]: Traceback (most recent call last):
2022-08-03T20:12:15.286013+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3621, in get_loc
2022-08-03T20:12:15.286014+00:00 app[web.1]: return self._engine.get_loc(casted_key)
2022-08-03T20:12:15.286014+00:00 app[web.1]: File "pandas/_libs/index.pyx", line 513, in pandas._libs.index.DatetimeEngine.get_loc
2022-08-03T20:12:15.286015+00:00 app[web.1]: File "pandas/_libs/index.pyx", line 547, in pandas._libs.index.DatetimeEngine.get_loc
2022-08-03T20:12:15.286015+00:00 app[web.1]: KeyError: Timestamp('2022-08-02 00:00:00+0000', tz='tzutc()')
2022-08-03T20:12:15.286016+00:00 app[web.1]:
2022-08-03T20:12:15.286016+00:00 app[web.1]: The above exception was the direct cause of the following exception:
2022-08-03T20:12:15.286016+00:00 app[web.1]:
2022-08-03T20:12:15.286017+00:00 app[web.1]: Traceback (most recent call last):
2022-08-03T20:12:15.286017+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/indexes/datetimes.py", line 679, in get_loc
2022-08-03T20:12:15.286017+00:00 app[web.1]: return Index.get_loc(self, key, method, tolerance)
2022-08-03T20:12:15.286018+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3623, in get_loc
2022-08-03T20:12:15.286018+00:00 app[web.1]: raise KeyError(key) from err
2022-08-03T20:12:15.286019+00:00 app[web.1]: KeyError: Timestamp('2022-08-02 00:00:00+0000', tz='tzutc()')
2022-08-03T20:12:15.286019+00:00 app[web.1]:
2022-08-03T20:12:15.286019+00:00 app[web.1]: The above exception was the direct cause of the following exception:
2022-08-03T20:12:15.286020+00:00 app[web.1]:
2022-08-03T20:12:15.286020+00:00 app[web.1]: Traceback (most recent call last):
2022-08-03T20:12:15.286020+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 375, in run_asgi
2022-08-03T20:12:15.286021+00:00 app[web.1]: result = await app(self.scope, self.receive, self.send)
2022-08-03T20:12:15.286021+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
2022-08-03T20:12:15.286021+00:00 app[web.1]: return await self.app(scope, receive, send)
2022-08-03T20:12:15.286022+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 269, in __call__
2022-08-03T20:12:15.286022+00:00 app[web.1]: await super().__call__(scope, receive, send)
2022-08-03T20:12:15.286022+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 124, in __call__
2022-08-03T20:12:15.286023+00:00 app[web.1]: await self.middleware_stack(scope, receive, send)
2022-08-03T20:12:15.286023+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
2022-08-03T20:12:15.286023+00:00 app[web.1]: raise exc
2022-08-03T20:12:15.286024+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
2022-08-03T20:12:15.286024+00:00 app[web.1]: await self.app(scope, receive, _send)
2022-08-03T20:12:15.286025+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 93, in __call__
2022-08-03T20:12:15.286025+00:00 app[web.1]: raise exc
2022-08-03T20:12:15.286025+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__
2022-08-03T20:12:15.286026+00:00 app[web.1]: await self.app(scope, receive, sender)
2022-08-03T20:12:15.286026+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
2022-08-03T20:12:15.286026+00:00 app[web.1]: raise e
2022-08-03T20:12:15.286027+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
2022-08-03T20:12:15.286027+00:00 app[web.1]: await self.app(scope, receive, send)
2022-08-03T20:12:15.286027+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 670, in __call__
2022-08-03T20:12:15.286028+00:00 app[web.1]: await route.handle(scope, receive, send)
2022-08-03T20:12:15.286028+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 418, in handle
2022-08-03T20:12:15.286028+00:00 app[web.1]: await self.app(scope, receive, send)
2022-08-03T20:12:15.286029+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 269, in __call__
2022-08-03T20:12:15.286029+00:00 app[web.1]: await super().__call__(scope, receive, send)
2022-08-03T20:12:15.286029+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 124, in __call__
2022-08-03T20:12:15.286030+00:00 app[web.1]: await self.middleware_stack(scope, receive, send)
2022-08-03T20:12:15.286030+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
2022-08-03T20:12:15.286030+00:00 app[web.1]: raise exc
2022-08-03T20:12:15.286031+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
2022-08-03T20:12:15.286031+00:00 app[web.1]: await self.app(scope, receive, _send)
2022-08-03T20:12:15.286031+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 93, in __call__
2022-08-03T20:12:15.286032+00:00 app[web.1]: raise exc
2022-08-03T20:12:15.286032+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__
2022-08-03T20:12:15.286032+00:00 app[web.1]: await self.app(scope, receive, sender)
2022-08-03T20:12:15.286033+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__
2022-08-03T20:12:15.286033+00:00 app[web.1]: raise e
2022-08-03T20:12:15.286038+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
2022-08-03T20:12:15.286038+00:00 app[web.1]: await self.app(scope, receive, send)
2022-08-03T20:12:15.286038+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 670, in __call__
2022-08-03T20:12:15.286039+00:00 app[web.1]: await route.handle(scope, receive, send)
2022-08-03T20:12:15.286039+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 266, in handle
2022-08-03T20:12:15.286039+00:00 app[web.1]: await self.app(scope, receive, send)
2022-08-03T20:12:15.286040+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 65, in app
2022-08-03T20:12:15.286040+00:00 app[web.1]: response = await func(request)
2022-08-03T20:12:15.286041+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 227, in app
2022-08-03T20:12:15.286041+00:00 app[web.1]: raw_response = await run_endpoint_function(
2022-08-03T20:12:15.286041+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 160, in run_endpoint_function
2022-08-03T20:12:15.286042+00:00 app[web.1]: return await dependant.call(**values)
2022-08-03T20:12:15.286042+00:00 app[web.1]: File "/app/app/endpoints.py", line 120, in get_portfolio
2022-08-03T20:12:15.286042+00:00 app[web.1]: portfolio = Portfolio.from_treasury_with_assets(
2022-08-03T20:12:15.286043+00:00 app[web.1]: File "/app/app/endpoints.py", line 84, in from_treasury_with_assets
2022-08-03T20:12:15.286043+00:00 app[web.1]: eth_series.loc[end].price - eth_series.loc[start].price
2022-08-03T20:12:15.286043+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/indexing.py", line 967, in __getitem__
2022-08-03T20:12:15.286044+00:00 app[web.1]: return self._getitem_axis(maybe_callable, axis=axis)
2022-08-03T20:12:15.286044+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/indexing.py", line 1202, in _getitem_axis
2022-08-03T20:12:15.286044+00:00 app[web.1]: return self._get_label(key, axis=axis)
2022-08-03T20:12:15.286045+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/indexing.py", line 1153, in _get_label
2022-08-03T20:12:15.286045+00:00 app[web.1]: return self.obj.xs(label, axis=axis)
2022-08-03T20:12:15.286045+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/generic.py", line 3864, in xs
2022-08-03T20:12:15.286046+00:00 app[web.1]: loc = index.get_loc(key)
2022-08-03T20:12:15.286046+00:00 app[web.1]: File "/usr/local/lib/python3.9/site-packages/pandas/core/indexes/datetimes.py", line 681, in get_loc
2022-08-03T20:12:15.286046+00:00 app[web.1]: raise KeyError(orig_key) from err
2022-08-03T20:12:15.286047+00:00 app[web.1]: KeyError: '2022-08-02'
2022-08-03T20:12:15.452919+00:00 heroku[router]: at=info method=GET path="/api/portfolio/0x1a9c8182c09f50c8318d769245bea52c32be35bc/2021-08-02" host=whip.buttery.money request_id=f6034c14-32a4-45a2-baee-80ae0bd25006 fwd="2001:8a0:7adb:c300:9cd1:3b68:292f:3f3c,172.68.102.178" dyno=web.1 connect=0ms service=1206ms status=500 bytes=193 protocol=https

A simple fallback seems to be to query the data again whenever this happens. It would be good logging something when this happens so we know that the cache is not working.