/cartesi-vm-ext-data

External Data Availability to the Cartesi VM

cartesi-vm-ext-data

External Data Availability to the Cartesi VM

Introduction

Cartesi is an Optimistic rollup running on top of layer 1 (or 2, etc.) Ethereum Virtual Machine (EVM) blockchains. The Cartesi Virtual Machine (VM) is a RISC-V virtual machine which has the capability to mark specific locations in its state as "dirty" and occasionally save its state in a Merkle tree, which is quite diff-friendly. This allows it to not only commit snapshots of its state, but efficiently keep its entire history of state changes. If and when a fraud dispute needs to be generated, this allows the "faulty" instruction to be pinpointed and proven fraudulent by emulating it on the EVM.

In addition the Cartesi VM is deterministic. It is single processor (but possibly multi-threaded) RISC-V VM. This allows multiple instances of it to produce the exact same state changes and output, given the exact same initial state and input. As such, multiple nodes can run the same execution VM and consent upon the correctness of the execution.

Even if all but one nodes produce fraudulent output, the honest node can prevail, while fraud proof(s) can be generated for the fraudulent node(s) by emulating them on the EVM.

Problem

Compared to the EVM, RISC-V is significantly more powerful and allows for execution of entire operating system (Cartesi Linux build) with its entire ecosystem of libraries and utilities. If RISC-V acts as a computer, EVM would compare to a programmable pocket calculator.

However, there is a problem: the Cartesi VM can perform powerful calculations but on what data? The only way to feed it with significant amount of data is upon initialization. After that, all input has to be recorded on the underlying EVM, in order to get to Cartesi and guarantee that all nodes receive the exact same input.

Implementation Issues

In order to guarantee determinism, the Cartesi VM cannot communicate with the "outside world". It cannot have network access, external file systems or input outside of the one recorded on the underlying EVM.

If such external data is to be accessed, it would have to be guaranteed to get to all Cartesi nodes (actually, those involved in a particular matched set, working on a common task). In addition, the eventual fraud proof would have to be able to not emulate the RISC-V on the EVM, but also produce proofs that a specific node did not get the correct data (this needs ellaboration!).

External Data Commitment

Sometimes it is enough to have data commitment instead of availability. Using Merkle trees, large data sets can be recorded, where the root uniquely identifies the entire data set (almost; well, intractably hard to produce two distinct Merkle trees with identical Merkle root). This is in case the program running on the Cartesi node needs to check on the data at arbitrary points. In other words, to verify part of the pre-committed data.

The commitment can be recorded on the EVM as input to Cartesi, and each consequent request in the pre-committed data can be accessed or proven by passing the Merkle path through the EVM, again as input.

This is still not enough! The Cartesi VM can do more than just verification - it can compute on big data, only if that data were available.

External Data Availability

Let's propose a new architecture:

  1. The data is recorded externally to the Cartesi VM. It is committed to the Cartesi VM in a content-addressable manner, for example IPFS CID.
  2. The committed data is mounted as file system in the OS running on the Cartesi VM. It can be accessed in its entirety as a tree of directories and files in them.
  3. In addition to the Cartesi VM an IPFS node is attached to each Cartesi node. In the (1.) above, only "pinned" roots can be passed to the EVM. This would guarantee data availability.
  4. To verify (3.) above, the EVM would be an FEVM (Filecoin EVM), with a built-in pre-compile which verifies that a particular CID is pinned in the IPFS node. The Cartesi server would call this EVM pre-compile to make sure no input can be passed as CID without a guarantee that it is available in its entirety.
  5. The IPFS data does not have to be pinned forever. As soon as the dispute grace period for the Cartesi epoch passes, the IPFS Pin can be removed and the related data can flush out of the IPFS node's cache as needed.

The above architecture would guarantee that each Cartesi node experiences exactly the same data intake.

Cartesi Dispute Proof Issue

In addition to the forward execution, we have to consider what happens when at least one Cartesi node differs from the rest. In such case fraud proof(s) need to be generated by emulating the Cartesi VM on the underlying EVM. In such case, all committed (and available) data can be fed into the emulated RISC-V machine as constants, or a read-only file system (this needs elaboration!).

Is this Feasible? How much work?

A more detailed look into the feasibility is needed. We need to consider the following:

  • Does the current implementation of the Cartesi Linux build have read-only file systems (FS) available? Do we maybe need to build a device driver for such read only IPFS FS?
  • How complicated is to inject code into the Cartesi Server to verify the validity and availability of the input CIDs?
  • Is the CID checking pre-compile on FEVM available now? If not, how easy is to build it? The Filecoin blockchain blockchain executes WASM and should at least be able to check on a specific IPLD node. Then, we may have to build a traversal mechanism to check the availability and consistency of the entire tree. See Filecoin VM (FVM).

This is not a project for a hackathon. Even if everything goes smoothly, several developer-months may be required. It may be even more difficult as FVM is a work-in-progress project. Maybe Cartesi and Protocol Labs can help with advice? Before this is started as a project, some research has to be completed to discover what components are available and reliable today.