simple-watcher.py error: Request Timeout Retry RPC Call on node: https://api.hive.blog
Opened this issue · 3 comments
ryan-lp commented
$ python simple-watcher.py
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (1/5)
Retrying in 0 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (2/5)
Retrying in 2 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (3/5)
Retrying in 3 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (4/5)
Retrying in 5 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (5/5)
Retrying in 6 seconds
output: None of identifier 81906994
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (1/5)
Retrying in 0 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (2/5)
Retrying in 2 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (3/5)
Retrying in 3 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (4/5)
Retrying in 5 seconds
Error: Request Timeout
Retry RPC Call on node: https://api.hive.blog (5/5)
Retrying in 6 seconds
Traceback (most recent call last):
File "/home/ryan/build/podping-hivewatcher/simple-watcher.py", line 89, in <module>
main()
File "/home/ryan/build/podping-hivewatcher/simple-watcher.py", line 56, in main
start_block = block_num_back_in_minutes(blockchain, 15)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ryan/build/podping-hivewatcher/simple-watcher.py", line 45, in block_num_back_in_minutes
block_num = blockchain.get_estimated_block_num(back_time)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ryan/.local/lib/python3.11/site-packages/beem/blockchain.py", line 325, in get_estimated_block_num
last_block = self.get_current_block()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ryan/.local/lib/python3.11/site-packages/beem/blockchain.py", line 300, in get_current_block
return Block(
^^^^^^
File "/home/ryan/.local/lib/python3.11/site-packages/beem/block.py", line 70, in __init__
super(Block, self).__init__(
File "/home/ryan/.local/lib/python3.11/site-packages/beem/blockchainobject.py", line 137, in __init__
self.refresh()
File "/home/ryan/.local/lib/python3.11/site-packages/beem/block.py", line 163, in refresh
raise BlockDoesNotExistsException("output: %s of identifier %s" % (str(block), str(self.identifier)))
beem.exceptions.BlockDoesNotExistsException: output: None of identifier 81906994
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ryan/build/podping-hivewatcher/simple-watcher.py", line 92, in <module>
main()
File "/home/ryan/build/podping-hivewatcher/simple-watcher.py", line 56, in main
start_block = block_num_back_in_minutes(blockchain, 15)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ryan/build/podping-hivewatcher/simple-watcher.py", line 45, in block_num_back_in_minutes
block_num = blockchain.get_estimated_block_num(back_time)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ryan/.local/lib/python3.11/site-packages/beem/blockchain.py", line 325, in get_estimated_block_num
last_block = self.get_current_block()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ryan/.local/lib/python3.11/site-packages/beem/blockchain.py", line 300, in get_current_block
return Block(
^^^^^^
File "/home/ryan/.local/lib/python3.11/site-packages/beem/block.py", line 70, in __init__
super(Block, self).__init__(
File "/home/ryan/.local/lib/python3.11/site-packages/beem/blockchainobject.py", line 137, in __init__
self.refresh()
File "/home/ryan/.local/lib/python3.11/site-packages/beem/block.py", line 163, in refresh
raise BlockDoesNotExistsException("output: %s of identifier %s" % (str(block), str(self.identifier)))
beem.exceptions.BlockDoesNotExistsException: output: None of identifier 81907007
daveajones commented
@agates Is the simple watcher deprecated at this point? I think it basically is right?
ryan-lp commented
FYI, the same node also fails in hive-watcher.py
among others:
2024-01-16T02:09:14+1100 | INFO backoff MainThread : | Backing off _req(...) for 0.6s (requests.exceptions.HTTPError: 521 Server Error: for url: https://rpc.mahdiyari.info/)
2024-01-16T02:09:15+1100 | INFO backoff MainThread : | Backing off _req(...) for 0.4s (requests.exceptions.HTTPError: 521 Server Error: for url: https://rpc.mahdiyari.info/)
2024-01-16T02:09:16+1100 | ERROR backoff MainThread : | Giving up _req(...) after 3 tries (requests.exceptions.HTTPError: 521 Server Error: for url: https://rpc.mahdiyari.info/)
2024-01-16 02:09:16,325 lighthive ERROR 521 Server Error: for url: https://rpc.mahdiyari.info/
2024-01-16T02:09:16+1100 | ERROR lighthive MainThread : | 521 Server Error: for url: https://rpc.mahdiyari.info/
2024-01-16 02:09:16,325 lighthive WARNING Ignoring node https://rpc.mahdiyari.info for 3600 seconds: ('get_dynamic_global_properties',), {}
2024-01-16T02:09:16+1100 | WARNING lighthive MainThread : | Ignoring node https://rpc.mahdiyari.info for 3600 seconds: ('get_dynamic_global_properties',), {}
2024-01-16T02:09:22+1100 | INFO backoff MainThread : | Backing off _req(...) for 0.5s (requests.exceptions.HTTPError: 521 Server Error: for url: https://rpc.mahdiyari.info/)
2024-01-16T02:09:23+1100 | INFO backoff MainThread : | Backing off _req(...) for 1.0s (requests.exceptions.HTTPError: 521 Server Error: for url: https://rpc.mahdiyari.info/)
2024-01-16T02:09:24+1100 | ERROR backoff MainThread : | Giving up _req(...) after 3 tries (requests.exceptions.HTTPError: 521 Server Error: for url: https://rpc.mahdiyari.info/)
2024-01-16 02:09:24,725 lighthive ERROR 521 Server Error: for url: https://rpc.mahdiyari.info/
2024-01-16 02:09:24,725 lighthive ERROR 521 Server Error: for url: https://rpc.mahdiyari.info/
2024-01-16T02:09:24+1100 | ERROR lighthive MainThread : | 521 Server Error: for url: https://rpc.mahdiyari.info/
2024-01-16 02:09:24,726 lighthive WARNING Ignoring node https://rpc.mahdiyari.info for 3600 seconds: ('get_block', {'block_num': 81941038}), {}
2024-01-16 02:09:24,726 lighthive WARNING Ignoring node https://rpc.mahdiyari.info for 3600 seconds: ('get_block', {'block_num': 81941038}), {}
2024-01-16T02:09:24+1100 | WARNING lighthive MainThread : | Ignoring node https://rpc.mahdiyari.info for 3600 seconds: ('get_block', {'block_num': 81941038}), {}
2024-01-16T02:09:32+1100 | INFO backoff MainThread : | Backing off _req(...) for 0.6s (requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.hive.blog', port=443): Read timed out. (read timeout=3))
2024-01-16T02:09:37+1100 | INFO backoff MainThread : | Backing off _req(...) for 0.4s (requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.hive.blog', port=443): Read timed out. (read timeout=3))
2024-01-16T02:09:41+1100 | ERROR backoff MainThread : | Giving up _req(...) after 3 tries (requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.hive.blog', port=443): Read timed out. (read timeout=3))
2024-01-16 02:09:41,779 lighthive ERROR HTTPSConnectionPool(host='api.hive.blog', port=443): Read timed out. (read timeout=3)
2024-01-16 02:09:41,779 lighthive ERROR HTTPSConnectionPool(host='api.hive.blog', port=443): Read timed out. (read timeout=3)
2024-01-16T02:09:41+1100 | ERROR lighthive MainThread : | HTTPSConnectionPool(host='api.hive.blog', port=443): Read timed out. (read timeout=3)
2024-01-16 02:09:41,780 lighthive WARNING Ignoring node https://api.hive.blog for 3600 seconds: ('get_block', {'block_num': 81941039}), {}
2024-01-16 02:09:41,780 lighthive WARNING Ignoring node https://api.hive.blog for 3600 seconds: ('get_block', {'block_num': 81941039}), {}
2024-01-16T02:09:41+1100 | WARNING lighthive MainThread : | Ignoring node https://api.hive.blog for 3600 seconds: ('get_block', {'block_num': 81941039}), {}
2024-01-16T02:09:42+1100 | INFO backoff MainThread : | Backing off _req(...) for 0.7s (requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://rpc.ecency.com/)
2024-01-16T02:09:44+1100 | INFO backoff MainThread : | Backing off _req(...) for 0.7s (requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://rpc.ecency.com/)
2024-01-16T02:09:46+1100 | ERROR backoff MainThread : | Giving up _req(...) after 3 tries (requests.exceptions.HTTPError: 502 Server Error: Bad Gateway for url: https://rpc.ecency.com/)
2024-01-16 02:09:46,440 lighthive ERROR 502 Server Error: Bad Gateway for url: https://rpc.ecency.com/
2024-01-16 02:09:46,440 lighthive ERROR 502 Server Error: Bad Gateway for url: https://rpc.ecency.com/
2024-01-16T02:09:46+1100 | ERROR lighthive MainThread : | 502 Server Error: Bad Gateway for url: https://rpc.ecency.com/
2024-01-16 02:09:46,440 lighthive WARNING Ignoring node https://rpc.ecency.com for 3600 seconds: ('get_block', {'block_num': 81941039}), {'num_retries': 2}
2024-01-16 02:09:46,440 lighthive WARNING Ignoring node https://rpc.ecency.com for 3600 seconds: ('get_block', {'block_num': 81941039}), {'num_retries': 2}
2024-01-16T02:09:46+1100 | WARNING lighthive MainThread : | Ignoring node https://rpc.ecency.com for 3600 seconds: ('get_block', {'block_num': 81941039}), {'num_retries': 2}
agates commented
I've never used simple-watcher for what it's worth... but it might want to remove the failing server from the accepted node list if it continues.
They come and go, so I'm not surprised. That's why I added the backoff logic to lighthive.