Solving contract state determinism for metadata

Question

Solving contract state determinism for metadata

dr-orlovsky opened this issue 2 years ago · 4 comments

Motivation

Currently RGB metadata fields are used to represent a part of the contract state which is not specific to any UTXO/single-use-seal or state transition - like in case of the asset name. Still, the metadata values can be upgraded with state transition, leaving us in the void with a generic way of finding what is the state of metadata for a specific contract.

This is a proposal on how this issue can be solved. This requires consensus changes to the RGB schema and existing/previously issued RGB smart contracts; the Compatibility section gives an options on how this problem can be mitigated.

Overview

The proposal consists of the following components:

Define a consensus ordering for state transitions.
Separate concept of state node-specific metadata (which never becomes part of the contract state) from contract-level "unowned" state, updatable with state transitions
Define schema-level rules for updating contract unowned state, in form of consensus-defined previous_unowned_state_atom, state_transition_unowned_state_update -> new_unowned_state_atom

Specification

Consensus ordering of state transition

General provisions

All state transitions in an RGB contract (both stash-stored and new one coming with consignment) have a deterministic ordering, which will be the same on the consensus level, meaning that each RGB Node (or other software with RGB Core engine) will come to the same order of state transition for a given contract. If two peers have a different subset of state transitions, the order they get will still be non-contradictory, specifically: if peer A has state transitions a < c, peer B has state transitions a < b, and peer C has state transitions b < c, than their orders are in concord between each other and with a global ordering of a < b < c.

Consensus ordering of state transition linearizes the DAG of state transitions under a contract in a well defined order. It is done using witness transaction (i.e. transaction committing to the state transition): the order of state transition follows the bitcoin consensus ordering of witness transaction, i.e. using block height and position in the block of the witness transaction.

Ordering of bundeled state transitions

If two or more state transitions under the same RGB contract are committed into the same witness transaction (which is possible with "bundles", where state transitions "spends" previous state from different witness transaction inputs) their order is the order of them within the bundle (i.e. the same order as the order in which they are serialized according to the RGB consensus rules).

Ordering of state extensions

State extension do not have a consensus ordering; they are just "extensions" to the state transition referencing them, and the unowned state in the state extension just becomes part of the unowned state of the state transition. If the state transition references more than just a single state extension, the set of state extensions must not contain a conflicting unowned state, otherwise the state transition will be invalid under RGB consensus rules.

Node-specific metadata

Each contract node (genesis, state transition, state extension) gets new set of data, called "transition-specific metadata", up to 255 items, typed using the same type system and identified with a node-level id (8 bit). These data do not become part of the RGB contract state and are used only as a supplementary information requried for validating a specific transition (for instance, they are used as a part of Pedersen commitment validation for the initial issuance).

Unowned contract state

Previous concept of metadata is transformed into a concept of unowned contract state (here "unowned" means that the state is not owned by any of UTXO owners, i.e. not assigned to any single-use-seal).

RGB schema defines which unowned state can be a part of the contract; each unowned state is defined by a unique type id (or simply type). Unowned state of each type is composed of atoms which are kept by the contract in form of FILO queue of valid state atoms. RGB schema defines the minimum and maximum size of the queue for each of the state types, for instance:

schema Some
  unowned ContractName :: Ascii          -- exactly one atom, equivalent to `Ascii[1..=1]`
  unowned CollectedData :: Bytes[0..100]   -- queue up to 100 atoms

Note: one as the maximum size of queue means that there is only a single valid unowned state value for a given type.

Schema defines which state transitions may update which types of the unowned state, and also specify maximum number of updated atoms they can provide (which must be less or equal to the maximum size of the contract unowned state array for the same state type).

  -- let's define some single-use-seals used below
  right Renomination

  -- this specifies that "renomination" state transition closing exactly one renominate seal 
  -- and optionally defining a new one must also define one and exactly one new atom of 
  -- contract name, which will replace the previous name
  transition Renominate :: Renomination -> Renomination?, ContractName

  right Collect

  -- this allows adding to the unowned state one or up to 10 more elements (state atoms)
  -- by the holders of the `Collect` rights under this contract
  transition AddData :: Collect -> Collect?, CollectedData[1..10]

If some unowned state is mandatory at the contract level (i.e. has a minimum number of atoms above zero), genesis must provide this state and must have exactly the same minimum requirements; otherwise the schema is invalid.

Contract unowned state is updated in the consensus ordering of the state transitions, such that new state provided by the transition is "pushed" into the end of the state queue - "popping" the oldest state atom out of the queue if the maximum size of the queue is reached (FILO order). This also means that if the maximum size is one, the state simply gets updated with the new state.

None of the contract nodes must have higher maximum number of the state atoms than in the definition of the unowned state.

schema WillNotCompile
   unowned Data :: Bytes[0..2]

   genesis :: Data[1..3] -- this is invalid and will result in a compile-time & schema-validation time errors

Compatibility

The proposal breaks consensus serialization of RGB schema. This can be mitigated by adding a special table to the RGB Core, where an old RGB schema id is mapped to the new RGB schema, providing in-flight replacement for already existing schemata as a hardcoded bugfix.
The proposal breaks consensus serialization of the contract nodes (genesis, state transitions, state extensions). The solution for the problem is the work in progress

Answer 1 · 2022-11-21T13:12:51.000Z

It's good to attempt to build in a way that old token state can be migrated, but if no easy fix can be found, the community will have to decide if this feature is worth the breaking change. I think it's worth it, since it could enable some really interesting things in each transfer, such as marking a transfer as unspendable (burning it), or adding other properties to it, such as a written message or note.

However, as we keep building, there will eventually be a point where we'll need to handle consensus-breaking changes more gracefully. I think effort should be made towards supporting migrations or leaving old code paths for various features whenever possible.

Answer 2 · 2022-11-24T15:05:51.000Z

This is a very interesting proposal, but at this stage I feel we should avoid breaking changes and new vectors of complexity as much as possible, if not for serious bug fixes. We have multiple products being built on top of RGB as we speak, and it would be very hard to ship them to market if the protocol and the RGB code base is not stable.

Answer 3 · 2022-11-24T16:48:56.000Z

I'm not sure anyone is using RGB actively on mainnet yet, and no 1.0 guarantees have been made yet, so I think adding such a powerful feature could be worth the breaking change this early on.

Answer 4 · 2022-11-25T07:54:35.000Z

My original plan was not to have zero consensus-breaking changes (other than bugfixes) after v0.8
While working on bugfixes I have found that some of them require consensus changes
I think there is a way to do some (many) of the things from this proposal in a way that would not affect wallet devs and issued asses users in a significant way
At some point (week-two) from now I propose to have a call and discuss what consensus- and API-breaking changes are required due to bug fixes; which features can be added using the opportunity of the neccesety of that consensus changes etc. I will prepare a report explaining what types of "breaking changes" can be out there and which of them caused by which feature/bug; plus ways how to mitigate the impact of these changes on users and wallet devs.
In the meanwhile, my priority remains, in the order of referencing, (a) to do fixes for all reported and found issues (b) do a self-audit, test coverage and full separation of everything consensus-related from other stuff for the v0.9, (c1) do basics of tooling for the schema and other devs for the work with RGB (not to repeat the mistakes like happened with NFT schema in a future), (c2) document most critical information and parts of the RGB.