paritytech/subxt

Unable to decode extrinsics from blocks older than 24H

Closed this issue · 4 comments

f-squirrel commented

Hi all,

I am using subtext to decode extrinsic from blocks.
I receive the following error:

value: Decode(Error { context: Context { path: [] }, kind: VisitorDecodeError(VariantNotFound(5, TypeDefVariant { variants: [Variant { name: "Disabled", fields: [], index: 0, docs: [] }, Variant { name: "Enabled", fields: [], index: 1, docs: [] }] })) })

This only happens for blocks older than 24 hours.
Blocks older than 24 hours do contain events.

  • Is it a problem of subxt or node configuration?
  • Is it possible to retrieve extrinsic information from events?
jsdw commented

Subxt, by default, doesn't carry any guarantees about being able to decode historic blocks. The reasons for this are:

  1. Subxt is built primarily to interact with the head of the chain, essentially setting state (via submitting transacitons) and getting state (via accessing storage entries) from the "database".
  2. A technical reason is that the metadata it obtains from a node on connecting describes current blocks, but if a runtime update happened recently, then that metadata may not accurately describe things prior to it.
  3. Subxt will eventually migrate to using the new chainHead RPC APIs for interacting with the head of a chain. These APIs enforce fairly strict limits/guarantees about which blocks can be obtained, restricting usage to very recent blocks.
  4. Non-archive RPC nodes don't carry around very much history anyway; they will contain block bodies but not storage entries (including events) for older blocks (I can't remember how old offhand; something like the last 500-1k blocks only will have this stuff available).

There is ongoing work on historic block decoding in Rust, but this will likely result in separate libraries for working with histroic state (which would rely on the archive methods from the new RPC spec).

That said, it is currently possible to manually set the metadata used by Subxt if you know what you're doing and can connect to an archive node. Doing this will break some things (like submitting transactions), but may allow slightly older blocks and storage to be decoded properly:

let mut api = OnlineClient::<PolkadotConfig>::from_url("wss://url.to.archive.node").await?;

// Set the metadata to some older metadata which covers the block(s) you're interested in. Even doing this,
// you won't be able to decode any blocks older than when V14 metadata was released a couple of years ago.
api.set_metadata(metadata_for_old_block);

// Now, obtain some older block events etc (that is from the same runtime as the metadata provided above)
// and subxt should be able to decode it properly at the moment:
api.blocks().at(block_hash).await?.events();

But note that this will not work once Subxt moves to using the new RPC APIs by default (although we will probably keep the legacy stuff around for a while, and so there will probably continue to be a way to do this for now).

Is it possible to retrieve extrinsic information from events?

Could you elaborate on what you want here? Given jsut an event, you can't locate the related extrinsic, but given an extrinsic you can access the related events if the metadata is able to decode them and the node is an archive node in case of older blocks)

f-squirrel commented

Hi @jsdw
Thank you for your prompt reply.
I am using the spec from the old blocks, i.e., I think this is not the case.

jsdw commented

In that case, I'd guess that your issue might be 4?:

Non-archive RPC nodes don't carry around very much history anyway; they will contain block bodies but not storage entries (including events) for older blocks (I can't remember how old offhand; something like the last 500-1k blocks only will have this stuff available).

jsdw commented

I'#ll close this for now, but please re-open if you'd like to discuss more :)

  • 👍1