Common genesis.json format scheme across all client implementations

Question

Common genesis.json format scheme across all client implementations

Closed this issue 3 years ago · 29 comments

Simple Summary

Client should be able to parse genesis.json config file that is commonly accepted and supported by all other client implementations.

Motivation

Currently every ethereum client has it's own genesis.json format which makes it impossible to move one node configuration to another node. (parity config to geth and vice versa). And it is not about unique options that some clients have, it's about using common fields but with different format. Same options has different style and names in .json configurations across client implementation.

Hive tool uses bash script in order to make a config file for each client individually.

Users have to search for correct genesis.json description for each client on different web pages.

Suggestion

Make a list of minimum required fields for genesis.json config file that each client would be able to parse and understand.

Example:

{
       "version" : "1",
       "params": {
                "miningMethod" : "NoProof",
            	"homesteadForkBlock": "0x00",
		"EIP150ForkBlock": "0x00",
		"EIP158ForkBlock": "0x00",
		"byzantiumForkBlock": "0x00",
		"constantinopleForkBlock": "0x00",
		"chainID": "0x01",
       },
       "genesis" : {
            	"nonce": "0x0000000000000042",
		"difficulty": "0x020000",
		"author": "0x0000000000000000000000000000000000000000",
		"timestamp": "0x00",
		"extraData": "0x11bbe8db4e347b4e8c937c1c8370e4b5ed33adb3db69cbdb7a38e1e50b1b82fa",
		"gasLimit": "0x1388"
       },
       "accounts": {
            "0x095e7baea6a6c7c4c2dfeb977efac326af552d87" : {
                "balance" : "0x0de0b6b3a76586a0",
                "code" : "0x6001600101600055",
                "nonce" : "0x00",
                "storage" : {
                    "0x00" : "0x02"
                }
            },
            "0x2adc25665018aa1fe0e6bc666dac8fc2697ff9ba" : {
                "balance" : "0x29a2241af62ca034",
                "code" : "",
                "nonce" : "0x00",
                "storage" : {
                }
            }
       }
}

Answer 1 · 2018-05-18T09:51:31.000Z

Hi, thanks for submitting this EIP, I think this could be useful to find some common ground here.

In the current format I see different problems with the data structure, which would probably hold me back to use this in production:

To have the hardforks added in a centrally managed/updated file makes things unflexible. If a client relies on this and would want to e.g. add some draft Casper hardfork params based an this list, one would have to wait for an updated released genesis.json file. This is not practical.
If a hardfork list should be maintained/included I would propose not to include this into the genesis file. Beside that this actually is no genesis data, this takes off the static nature of a genesis.json file. This prevents to e.g. provide a checksum/hash for the file, which would be useful to verify its integrity. So I would instead maintain a separate hardforks.json file.
Having the genesis accounts integrated in the file makes the file very large. If clients just need the genesis parameters (e.g. to instantiate a genesis block) they always have to load the whole huge file. This makes things unnecessarily inefficient. I would also propose to use a separate genesisAccounts.json file here.
I think for this to be maximally useful this should be defined with different networks in mind, not just the Ethereum mainnet.

Sidenote: have seen that this is discussed in the core devs meeting today, sorry, can't join on this for time reasons.

Answer 2 · 2018-05-18T09:57:10.000Z

One thought further: I am generally skeptical if this can be provided in a way that clients want to use/parse this "as is". It will nevertheless be helpful if this information is provided officially and from a trustful source, even if only parts are taken and/or not parsed or used directly but just copied.

Answer 3 · 2018-05-21T10:52:01.000Z

Speaking of splitting the config file. I think it is not a good idea. If you split options into several different files everything will become messy pretty fast.
However you don't want to edit a large file with the genesis state.
An idea for solution of this was to allow the field accounts (or state) that contains all the state data for the genesis block to be of two types.

if the state is small (like in tests) then it could easily fit into config file
if the state is huge, the field accounts could contain a path to the file that contains all the state.

a node reads the link with a path to a file that contains huge state and loads up the state data from it.
thus you could have one small config file with all the options for node and a huge state file separated.

Answer 4 · 2018-05-29T15:32:44.000Z

hmm. I wonder, could we abstract EVM gasPrices to some config file as well.
It looks like some other parties doing their own implementation of ethereum would like to use our test source files

Answer 5 · 2018-09-17T21:41:04.000Z

This should probably be updated to use the Fancy names for forks as is being done here: ethereum/tests#488

Answer 6 · 2018-09-18T21:35:28.000Z

@winsvega while working to implement this in trinity I've found the spec to be lacking a bit. What do you think about adding a JSON-schema definition to this EIP? I've written one up which can be found in this PR: ethereum/py-evm#1299 though I think there's some necessary discussion on things like required fields and the required formats for data.

Answer 7 · 2018-09-18T23:08:29.000Z

We've tried already. Making a complicated hard to read several pages jsob scheme does not help. Could we just formalize fields and its values? Please look

Answer 8 · 2018-09-18T23:10:04.000Z

Set chsin params at https://github.com/ethereum/retesteth/wiki/RPC-Methods#test_setchainparams

Answer 9 · 2018-09-19T16:56:43.000Z

Could we just formalize fields and its values? Please look

Yes, that was my intent with JSON-schema, but I understand that format is not the most accessible. It sounds like you are thinking about more of a natural language spec which defines the fields, formats, required/optional, etc? If so, 👍 as I'm indifferent to how it is specified, only that there is a spec with what I think is necessary information:

what fields are required
what each field means
what formats are allowed (like is the 0x prefix required/optional?)
additional validation (like is it invalid to have homesteadForkBlock < byzantiumForkBlock?)

Answer 10 · 2018-10-04T18:24:33.000Z

I guess explicitly mention that network ID is not identified in genesis (so the same genesis can be used for different networks), but chainID must be specified if the SpuriousDragon fork (or later) is included.

Answer 11 · 2018-11-21T20:09:10.000Z

Now that I have to run so many different clients with different ways approaching this issue, I would say, the ones implementing the Genesis format / EIP-1085 are the most annoying ones.

I think we should not stop at defining the genesis and forks only but give a complete network description. This is what Parity Ethereum is doing. The advantage is, you can simply pass a single file to a client and it runs without any additional need for parameters or descriptors.

My main critique is the lack of networkID and bootnodes support. I don't see why a Genesis should be reusable across different networks. For example, I run the following four commands to launch a Görli testnet node with four different clients:

Parity Ethereum and Nethermind with chain spec:

parity --chain goerli.json

dotnet run --project Nethermind.Runner --config configs/goerli.cfg

Geth and Pantheon with EIP-1085 Genesis:

geth init goerli.genesis
geth --networkid 6284 --bootnodes enode://04fb7acb86f47b64298374b5ccb3c2959f1e5e9362158e50e0793c261518ffe83759d8295ca4a88091d4726d5f85e6276d53ae9ef4f35b8c4c0cc6b99c8c0537@40.70.214.166:40303,enode://17de5580bbc1620081a21f82954731c7854305463630a0d677ed991487609829a6bf1ffcb8fb8ef269eff4829690625db176b498c629b9b13cb39b73b6e7b08b@213.186.16.82:1345,enode://22da3ef3707626a92a32b0527d0846f88228daa0536c62d83c9ac7e96660bc8e4ac70a9aa8f8cedf71b580cd41449ad46c6e5a06ecf138b142f38a9d1b2b856a@85.7.110.224:30303,enode://3897b1a5786948f643d9755df92dc56d0b2284f36730dc198ef371aebf191b24b5cbe8162c2032b09b2f14ba73460bfc3f7d4ef1e26bcc59297d4f235dc5cdc5@54.88.169.219:30303,enode://3d197d65ed92af6d0adf280ce486714fb641ef9f9f38f0bdd5ddd552666fc1132f033eb249a87f7f30086902c131f30f054f872ae80ac83eea6bd3760a7bbce2@40.70.214.166:30405,enode://3d8d6698d2d4d730d896c7c1e3602ff845343f71bacbf8cb614b0e94fcb3b10e1a49ac2a5063c76617182a1c5928a4a63d4be897e54ae1cb858a1b94d0d275b8@188.166.20.30:30303,enode://46add44b9f13965f7b9875ac6b85f016f341012d84f975377573800a863526f4da19ae2c620ec73d11591fa9510e992ecc03ad0751f53cc02f7c7ed6d55c7291@94.237.54.114:30313,enode://5065d5221b507764771a8b74abc69df0351217eae09b96ec0df4275576a8b2bbba9986ce3037e6fb3c933b5b301364e18030c1ada8cec4ae00f1fa4dfff32eb8@13.113.211.0:30303

pantheon --genesis=goerli.genesis --network-id=6284 --bootnodes=enode://04fb7acb86f47b64298374b5ccb3c2959f1e5e9362158e50e0793c261518ffe83759d8295ca4a88091d4726d5f85e6276d53ae9ef4f35b8c4c0cc6b99c8c0537@40.70.214.166:40303,enode://17de5580bbc1620081a21f82954731c7854305463630a0d677ed991487609829a6bf1ffcb8fb8ef269eff4829690625db176b498c629b9b13cb39b73b6e7b08b@213.186.16.82:1345,enode://22da3ef3707626a92a32b0527d0846f88228daa0536c62d83c9ac7e96660bc8e4ac70a9aa8f8cedf71b580cd41449ad46c6e5a06ecf138b142f38a9d1b2b856a@85.7.110.224:30303,enode://3897b1a5786948f643d9755df92dc56d0b2284f36730dc198ef371aebf191b24b5cbe8162c2032b09b2f14ba73460bfc3f7d4ef1e26bcc59297d4f235dc5cdc5@54.88.169.219:30303,enode://3d197d65ed92af6d0adf280ce486714fb641ef9f9f38f0bdd5ddd552666fc1132f033eb249a87f7f30086902c131f30f054f872ae80ac83eea6bd3760a7bbce2@40.70.214.166:30405,enode://3d8d6698d2d4d730d896c7c1e3602ff845343f71bacbf8cb614b0e94fcb3b10e1a49ac2a5063c76617182a1c5928a4a63d4be897e54ae1cb858a1b94d0d275b8@188.166.20.30:30303,enode://46add44b9f13965f7b9875ac6b85f016f341012d84f975377573800a863526f4da19ae2c620ec73d11591fa9510e992ecc03ad0751f53cc02f7c7ed6d55c7291@94.237.54.114:30313,enode://5065d5221b507764771a8b74abc69df0351217eae09b96ec0df4275576a8b2bbba9986ce3037e6fb3c933b5b301364e18030c1ada8cec4ae00f1fa4dfff32eb8@13.113.211.0:30303

Note the scrollbar. When I talk with client implementers I encourage them to use chain specs rather than genesis files. /cc @cburgdorf @ajsutton

Happy to draft an additional ERC for that, however, meanwhile what about networkID and bootnodes support?

Answer 12 · 2018-11-21T20:52:58.000Z

networkID and bootnodes could be supported.
what are the other issues?

Answer 13 · 2018-11-21T22:00:45.000Z

Happy to draft an additional ERC for that

I hope you do @5chdn! It's pretty compelling to be able to launch the client with only the chain spec file.

It seems that chain spec is a proper subset of the genesis file spec, which is a nice property: implementing chain spec means that you necessarily support genesis files, too.

Answer 14 · 2018-11-21T23:10:22.000Z

I only became aware of this EIP just a moment ago and I was thinking about this topic today. The Hive updates now allow simulations to run multiple flavours of client at once, and I am now in the process of adding Trinity, Nethermind, Harmony,, into the available clients. Simulators now determine which clients should be invoked, rather than letting the Hive engine specify. In this case, the Simulators can now also specify which concrete configuration/genesis each client instance can use. The current Hive code places the responsibility of parsing the genesis.json (supplied by the engine) onto some fairly fragile shell scripts in the client container entrypoints. I want to get rid of all that. So until I noticed the convo on AllCoreDevs and here, I was about to go the route of having the Simulator instantiate the client with a client specific config that the Sim supplies. Based on this new info here, I think the best route instead would be for me to have Hive accept a common format and then implement the conversions there, so the Simulators can be agnostic about Client launch details.
Hive will expose some kind of common API in that case anyway, allowing Simulator authors to launch Clients and hopefully not worry too much about how the Client is sparked up. I will implement the conversions where possible. Perhaps whether or not a common format is agreed upon, the conversion tools could become a useful component, or there may be other synergies.

Answer 15 · 2018-11-22T00:01:58.000Z

Supporting networkId and bootnodes in the genesis config makes a lot of sense to me. In Pantheon a predefined network is essentially a tuple of genesis, networkId and bootnodes which has been enough to describe MainNet, Rinkeby, Ropsten and Goerli.

Chain spec also looks quite good, the only concern I have is that it enables significantly more customisation in a couple of areas which I suspect wouldn't be supported by a number of clients (such as defining precompiles in the chain spec file). It could be trimmed it down to the configurable options that are widely supported and/or required by major networks, or for some things maybe it's a reasonable lever to make them more configurable (e.g. some of the ethash params seem sensible to make configurable).

Answer 16 · 2018-11-22T00:05:09.000Z

One other consideration is what the default values should be for things that aren't specified in the config and/or which fields are strictly required. That's important for cross-client compatibility but also for upgradability as new fields are added as part of future hard forks.

Answer 17 · 2018-11-22T01:12:04.000Z

The common genesis format should be as a yellowpaper rule for all clients to implement. Writing scripts that convert one format to another for specific client is not a good design.

Answer 18 · 2018-11-22T07:45:47.000Z

@winsvega That is true. Hive, however, needs to support scenarios like starting simulations involving different versions of client concurrently, including those prior to a commonly agreed format, and even when different versions of the later common format exist. A common format however would make the implementation of those requirements a lot simpler. In the meantime, the code from Hive (when implemented) could serve as a config/genesis generator for whomever (eventually)

Answer 19 · 2018-11-22T21:47:33.000Z

With "miningMethod" : "NoProof", defining the consensus algorithm

Would it be clearer to actually call this consensusAlgorithm since PoA schemes like Clique technically don't mine blocks but seal them? Mining is commonly used for those anyway so not a big deal.
Where would options for the consensus algorithm go? For example block period and epoch length in clique or IBFT?

Answer 20 · 2018-11-24T05:15:26.000Z

We lately have introduced a new https://github.com/ethereumjs/ethereumjs-common library - providing generic access to network and hardfork params - where a lot of design thinking has been put in, it might be worth to take some inspiration for some structures. On the other way we are also ready to rework if things are suboptimal, over- or under-engineered.

Just as a first note, have no time right now to go into more detail but will join the discussion during the next days.

Answer 21 · 2018-11-24T05:37:58.000Z

Have opened an issue https://github.com/ethereumjs/ethereumjs-common/issues/30 on genesis format changes, if you have a first look on our library and stumble over things which you think we should definitely change, feel free to post there.

Answer 22 · 2018-12-03T13:52:14.000Z

Sorry, due to personal reasons I am delayed on this, still plan to jump in though and give some thought.

Answer 23 · 2019-02-28T21:25:50.000Z

Is there a standardized name for the new constantinople fork yet? We picked petersburgForkBlock in trinity, but it's easy to change (for now).

Answer 24 · 2019-02-28T21:27:21.000Z

Pantheon picked constantinopleFix which I believe matches what the reference tests are using.

Answer 25 · 2019-02-28T21:29:24.000Z

So your genesis key name is constantinopleFixForkBlock?

Answer 26 · 2019-02-28T21:32:27.000Z

constantinopleFixBlock. e.g.

    "homesteadBlock": 1150000,
    "daoForkBlock": 1920000,
    "daoForkSupport": true,
    "eip150Block": 2463000,
    "eip150Hash": "0x2086799aeebeae135c246c65021c82b4e15a2c451340993aacfd2751886514f0",
    "eip155Block": 2675000,
    "eip158Block": 2675000,
    "byzantiumBlock": 4370000,
    "constantinopleBlock": 7280000,
    "constantinopleFixBlock": 7280000,

Answer 27 · 2019-02-28T21:37:33.000Z

Interesting, that's definitely something this EIP would have to get consensus on. All of the key names in trinity (and in the issue summary above) include Fork, eg: https://github.com/ethereum/trinity/blob/master/trinity/assets/eip1085.schema.json#L70

Would be good to check what other major clients are doing too. But for now, I have to go push a trinity release.

Answer 28 · 2021-12-06T03:11:51.000Z

There has been no activity on this issue for two months. It will be closed in a week if no further activity occurs. If you would like to move this EIP forward, please respond to any outstanding feedback or add a comment indicating that you have addressed all required feedback and are ready for a review.

Answer 29 · 2021-12-20T03:12:00.000Z

This issue was closed due to inactivity. If you are still pursuing it, feel free to reopen it and respond to any feedback or request a review in a comment.