tradingstrategy-ai/web3-ethereum-defi

Handle chain reorg in event reader

Closed this issue · 4 comments

Current the event reader can't handle chain reorg situation, so it's likely to fail here:

if extract_timestamps is not None:
timestamps = extract_timestamps(web3, start_block, end_block)
for log in logs:
block_hash = log["blockHash"]
block_number = int(log["blockNumber"], 16)
# Retrofit our information to the dict
event_signature = log["topics"][0]
log["context"] = context
log["event"] = filter.topics[event_signature]
try:
log["timestamp"] = timestamps[block_hash] if extract_timestamps else None
except KeyError as e:
raise RuntimeError(f"Timestamp missing for block number {block_number:,}, hash {block_hash}, our timestamp table has {len(timestamps)} blocks") from e

if extract_timestamps is not None:
    timestamps = extract_timestamps(web3, start_block, end_block)

for log in logs:
    block_hash = log["blockHash"]
    block_number = int(log["blockNumber"], 16)
    # Retrofit our information to the dict
    event_signature = log["topics"][0]
    log["context"] = context
    log["event"] = filter.topics[event_signature]
    if extract_timestamps:
        try:
            log["timestamp"] = timestamps[block_hash]
        except KeyError as e:
            # If the block hash is not found in the timestamps table, it could be due to a chain reorg.
            # In this case, we can try to retrieve the timestamp using the block number instead.
            try:
                block = web3.eth.getBlock(block_number)
                log["timestamp"] = block["timestamp"]
            except Exception as e:
                raise RuntimeError(
                    f"Timestamp missing for block number {block_number:,}, hash {block_hash}, our timestamp table has {len(timestamps)} blocks"
                ) from e
    else:
        log["timestamp"] = None

This code first checks if extract_timestamps is not None, and if it is not, it retrieves the timestamps for the blocks within the specified range using the extract_timestamps function. Then, for each log in the logs list, it retrofits the information to the dict and tries to retrieve the timestamp for the block using the block hash. If a KeyError is raised, it means that the block hash is not found in the timestamps table, which could be due to a chain reorg. In this case, it tries to retrieve the timestamp using the block number instead using the web3.eth.getBlock method. If this also fails, it raises a RuntimeError with a message indicating that the timestamp is missing.

#59 can you check this once @miohtama @hieuh25

@swarna1101 Thank you for your feedback. I am sorry your comment is not relevant for this issue. Please do not post comments, unless you have more context on the issue.

Fixed in #62