500 internal server error on GET /passes/<norad-id>
viconnex opened this issue · 9 comments
Hello! Thank you for this project!
I get 500 errors when calling the GET /passes/<norad-id> endpoint.
Did you also notice this? Do you know what is happening?
Or am I making a mistake in my query?
Below a sample of the performed query:
curl 'https://satellites.fly.dev/passes/25544?lat=-34.911221&lon=-57.9372988&limit=100&days=7&visible_only=false' \
-H 'accept: application/json' \
-H 'accept-language: fr,en-US;q=0.9,en;q=0.8,fr-FR;q=0.7,es;q=0.6,de;q=0.5,it;q=0.4' \
-H 'cache-control: no-cache' \
-H 'pragma: no-cache' \
-H 'priority: u=1, i' \
-H 'referer: https://satellites.fly.dev/docs/' \
-H 'sec-ch-ua-mobile: ?0' \
-H 'sec-fetch-dest: empty' \
-H 'sec-fetch-mode: cors' \
-H 'sec-fetch-site: same-origin'
Result:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or
there is an error in the application.</p>
Thank you for your help!
Hi, thanks for alerting. Reading the logs and seems like the redis cache reached the daily limit 🥵
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] cache.set(cache_key, tle, CACHE_TIMEOUT)
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/client.py", line 1801, in set
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] return self.execute_command('SET', *pieces)
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/client.py", line 901, in execute_command
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] return self.parse_response(conn, command_name, **options)
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/client.py", line 915, in parse_response
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] response = connection.read_response()
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] File "/usr/local/lib/python3.8/site-packages/redis/connection.py", line 747, in read_response
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] raise response
2024-04-24T14:58:35.835 app[9080e205a22087] gru [info] redis.exceptions.ResponseError: max daily request limit exceeded. Limit: 10000, Usage: 10000. See https://upstash.com/docs/redis/troubleshooting/max_daily_request_limit for details
2024-04-24T14:58:35.836 app[9080e205a22087] gru [info] 172.16.136.66 - - [24/Apr/2024:14:58:35 +0000] "GET /passes/25544?lat=-34.911221&lon=-57.9372988&limit=100&days=7&visible_only=false HTTP/1.1" 500 290 "https://satellites.fly.dev/docs/" "curl/8.4.0"
I guess I could skip cache writes if hitting this limit, I'll take a look.
I could add Cloudflare on front to cache cached requests, but that will require a domain change.
Done. Now use https://sat.terrestre.ar and let's wait until tomorrow to reset the daily quota. I blocked a heavy bot that was consuming the quota.
Hello @redraw! Thanks for setting this up! However I get 403 Forbidden errors now 😅
I am blocked by the Cloudflare protection. The error message invites me to share my Cloudflare Ray ID: 88394dda2a0a9e8d. Does it help for debugging?
Thanks! 🚀
@viconnex based on the Cloudflare Ray ID, you are THE one I've blocked 👀. You were flooding the server, exceeding the redis operations quota. I can unblock your IP if you can fix your code. Seems like you were hitting the /passes endpoint in a while True loop maybe?
LOL indeed! Sorry! My code calls the /passes endpoint every 10 seconds. What would be an acceptable delay for you?
No prob. Why every 10 seconds? passes don't change. Calculations tends to drift after 15 days of prediction, as those will require updated TLEs in the future. Every 1 day you should be fine.
Just unblocked your IP 👍
Thank you! Yes you're right, I will change the code.