Add Web3 1.0 support
owocki opened this issue · 1 comments
(this is a continuation of #2350 (comment) from @danfinlay )
The core team has been very busy with other issues at the moment, and we've always planned to do this eventually, and we might just do it ourselves soon. The community doesn't need to feel responsible for completing this.
That said, since this issue clearly needs a re-focusing, I will now write a brief summary of what this issue involves.
The Current State of MetaMask's Provider
MetaMask's most popular open source component to date has by far been web3-provider-engine. This express inspired middleware architecture allowed @kumavis to compose MetaMask's custom provider logic into a stack of "subproviders" that handle different requests, or portions of requests, and then either pass the mutated request down the stack, or reply directly.
This engine was designed for the original Ethereum JSON RPC spec. Since then, there has also emerged a websocket API that includes subscriptions called the RPC Pub Sub API. This new API allowed push subscriptions, and Web3 1.0 to exist.
Currently, MetaMask still uses polling & HTTP to get all of its data, and so the most obvious way to add Web3 1.0 support is to allow us to move onto a websocket based infrastructure.
A Problem With That
Over time, it has become increasingly clear that some of the ethereum-opinionated aspects of provider-engine were actually getting in the way of creating a good provider, and things were being hacked on top of it.
For example, our cache-subprovider needs block awareness to know when to clear its cache, and so block tracking was built into provider-engine, and bit by bit, these pieces of opinionated architecture has made provider-engine a bit harder to work with.
Furthermore, since provider-engine was never designed for handling subscriptions, since it's a middleware for responding to requests, it has no natural way of representing a websocket connection.
The Websocket Solution
Since much of the difficulty of adding a new connection type is that provider-engine encompasses all the middleware, and makes them hard to interact with, we're moving MetaMask over to json-rpc-engine. You'll see us use it a few places in MetaMask instead of provider-engine. This is basically the same general product as provider-engine, except with all the ethereum-opinionated aspects torn out, so they can more easily be accessed by external consumers, and so some types of features (like subscriptions) can more easily be added.
By removing the specialized logic from the engine itself, that kind of wiring can be left to the subproviders themselves. For example, the new arrangement might look a little more manual, but it allows much more diversity in subproviders (pseudocode below):
var engine = new JsonRpcEngine()
var blockTracker = new BlockTrackerSubprovider()
var cache = new CacheSubprovider({ blockTracker })
engine.add(blockTracker)
engine.add(cache)
engine.start()Moving to json-rpc-engine means every subprovider in the zero subprovider needs to be re-written or modified for json-rpc-engine, so its functionality can be swapped in place. Many of these subproviders have already been written here:
https://github.com/MetaMask/eth-json-rpc-middleware
In particular, the final subprovider that will need to be written for this issue is the websocket subprovider. Once we have this all ready, we'll probably need to compose the zero provider which is currently only imported, because only by composing the subproviders will we have access to the block-tracker's events. Or we come up with a way of accessing specific subproviders from the parent controller.
Anyways, this proposal is hardly trivial, as provider-engine is basically the heart of MetaMask, and so this is something like MetaMask heart surgery. We've already seen some regressions related to moving provider-engine over to json-rpc-engine, and so this needs to be done slowly and carefully, and json-rpc-engine needs to get really solid really fast.
An MVP Solution
While everything above has been accurate and true, there is a way we can provide web3 1.0 support a bit faster, but without the full performance benefits of websockets.
This is to polyfill the subscription API, the way Ben Burns did here already.
If that subprovider got a little polish and QA, it's possible it would be ready to go very soon, and so someone looking for the shortest path to letting their dapp use Web3 1.0 might just take that on, and I think it would still win the bounty for this issue, since this issue is really about web3 1.0 support, not websockets themselves.
The subscription data event would still need to be exposed in some way, and maybe a new module could be used to wrap provider-enigne to expand its API to include the full PUB SUB API. Figuring out those details would still be part of this "MVP".