Error: Metadata(IncompatibleCodegen)

[paulormart]

Hi @niklasad1,

I believe the issue below was the original one I stumbled since the runtime upgrade 1003000 and lead me to open the previous issue #1754

The particular storage item storage().para_inherent().on_chain_votes() does not decode correctly for older blocks if the metadata file is already on v1003000.

Example tested:

#![allow(missing_docs)]
use subxt::backend::{legacy::LegacyRpcMethods, rpc::RpcClient};
    // rpc::reconnecting_rpc_client::{RpcClient as ReconnectingRpcClient, ExponentialBackoff} };
use subxt::{OnlineClient, PolkadotConfig};

// Generate an interface that we can use from the node's metadata.
#[subxt::subxt(runtime_metadata_path = "../artifacts/kusama_metadata_new.scale")]
pub mod node_runtime {}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create a new API client, configured to talk to Polkadot nodes.
    // let api = OnlineClient::<PolkadotConfig>::from_url("wss://rpc.turboflakes.io/kusama").await?;

    // First, create a raw RPC client:
    let rpc_client = RpcClient::from_url("wss://rpc.turboflakes.io/kusama").await?;

    // Use this to construct our RPC methods:
    let rpc = LegacyRpcMethods::<PolkadotConfig>::new(rpc_client.clone().into());

    // We can use the same client to drive our full Subxt interface too:
    let api = OnlineClient::<PolkadotConfig>::from_rpc_client(rpc_client.clone()).await?;

    
    // let block_number = 24779298_u32;
    let block_number = 24783412_u32;

    let block_hash_metadata = rpc
        .chain_get_block_hash(Some((block_number).into()))
        .await?;

    let metadata = rpc.state_get_metadata(block_hash_metadata).await?;
    
    // assign metadata to the api
    api.set_metadata(metadata);
    
    if let Some(block_hash) = rpc.chain_get_block_hash(Some(block_number.into())).await? {

        println!("block_hash: {:?}", block_hash);

        let events = api.events().at(block_hash).await?;

        if let Some(new_session_event) = events.find_first::<node_runtime::session::events::NewSession>()? {
            println!("new_session_event: {:?}", new_session_event);
        }

        // Fetch on chain votes
        let on_chain_votes_addr = node_runtime::storage().para_inherent().on_chain_votes();
        if let Some(backing_votes) = api
            .storage()
            .at(block_hash)
            .fetch(&on_chain_votes_addr)
            .await?
        {
            println!("backing_votes: {:?}", backing_votes);
        }

    }

    Ok(())
}

Output: Error: Metadata(IncompatibleCodegen)

[jsdw]

The Error: Metadata(IncompatibleCodegen) comes from the fact that the hardcoded metadata you're using to generate the interface here:

#[subxt::subxt(runtime_metadata_path = "../artifacts/kusama_metadata_new.scale")]
pub mod node_runtime {}

Is different from the runtime metadata you are using for the block (which you fetch here: let metadata = rpc.state_get_metadata(block_hash_metadata).await?;).

To use the statically generated interface eg on the line events.find_first::<node_runtime::session::events::NewSession>()?, the metadata used to generate the static interface needs to match the runtime metadata used in the client (or more precisely, it needs to line up w.r.t whatever calls/events/storage you're using it for).

[niklasad1]

Yeah so this is a bit tricky/overwhelming....

There is a breaking change in storage for the storage key ParachainHosts::on_chain_votes between these two versions i.e, you can't use the static interface generated by subxt for both these versions (v1003000 and v1002006)

Unfortunately for this case the metadata you set on that block most match the one you generated to the codegen for :(

Thus, you need download metadata from a node with v1002006 and re-generate the subxt codegen then this snippet should work again.

[paulormart]

Ok, I think we can close this issue.

One workaround to get it to work for this particular case was to use the block_metadata to iterate over the events and reassign the static_metadata again to the api to decode storage.. something along the lines below:

#![allow(missing_docs)]
use subxt::backend::{legacy::LegacyRpcMethods, rpc::RpcClient};
use subxt::{OnlineClient, PolkadotConfig};

// Generate an interface that we can use from the node's metadata.
#[subxt::subxt(runtime_metadata_path = "../artifacts/kusama_metadata_v1003000.scale")]
pub mod node_runtime {}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create a new API client, configured to talk to Polkadot nodes.
    // let api = OnlineClient::<PolkadotConfig>::from_url("wss://rpc.turboflakes.io/kusama").await?;

    // First, create a raw RPC client:
    let rpc_client = RpcClient::from_url("wss://rpc.turboflakes.io/kusama").await?;

    // Use this to construct our RPC methods:
    let rpc = LegacyRpcMethods::<PolkadotConfig>::new(rpc_client.clone().into());

    // We can use the same client to drive our full Subxt interface too:
    let api = OnlineClient::<PolkadotConfig>::from_rpc_client(rpc_client.clone()).await?;

    // Clone the static metadata (v1003000) so it can be reassigned later if needed.
    let static_metadata = api.metadata().clone();
    
    let block_number = 24783412_u32;

    if let Some(block_hash) = rpc.chain_get_block_hash(Some(block_number.into())).await? {

        println!("block_hash: {:?}", block_hash);

        println!("runtime_version: {:?}", api.runtime_version());

        // Assign _block_metadata_ to the api to be able to iterate events without the 
        // `Error: Codec(Error { cause: None, desc: "Not enough data to fill buffer" })` being raised 
        // for this particular block 24783412:
        
        let block_hash_metadata = rpc.chain_get_block_hash(Some((block_number).into())).await?;
        let block_metadata = rpc.state_get_metadata(block_hash_metadata).await?;
        api.set_metadata(block_metadata);

        let events = api.events().at(block_hash).await?;

        if let Some(new_session_event) = events.find_first::<node_runtime::session::events::NewSession>()? {
            println!("new_session_event: {:?}", new_session_event);
        }

        // Reassign _static_metadata_ to the api so we can get on_chain_votes properly decoded without `Error: Metadata(IncompatibleCodegen)`
        api.set_metadata(static_metadata);

        // Fetch on chain votes
        let on_chain_votes_addr = node_runtime::storage().para_inherent().on_chain_votes();
        if let Some(backing_votes) = api
            .storage()
            .at(block_hash)
            .fetch(&on_chain_votes_addr)
            .await?
        {
            println!("backing_votes: {:?}", backing_votes);
        }

    }

    Ok(())
}

[paulormart]

Closing this issue, it has been been working well, thanks guys.