attestantio/go-eth2-client

Loading validator set does not work for prysm clients with default flags

pk910 opened this issue · 1 comments

pk910 commented

Loading the validator set does currently not work with prysm clients without supplying special flags.

The ValidatorsProvider.Validators function uses the /eth/v2/debug/beacon/states/{state_id} endpoint to fetch the full state.
I guess this was made because the regular /eth/v1/beacon/states/{state_id}/validators endpoint does not support SSZ encoding?

But /debug/ apis are not available for prysm without supplying the --enable-debug-rpc-endpoints client flag.

It would be great to have a possibility to disable using the /debug/ endpoint if it's not available :)

mcdee commented

It would be great to have a possibility to disable using the /debug/ endpoint if it's not available :)

Not realistic, I'm afraid. With the larger networks (mainnet, holesky) using JSON for items such as validators is basically unmanageable. This is a combination of the size of JSON, and the limitations on the number of validator indices/pubkeys that can be requested in a single operation. A number of beacon nodes also struggle to provide information piecemeal. So fetching validators via state SSZ is the only viable option.

To give you an idea of what I mean: attempting to fetch 50,000 validators on holesky using the validators endpoint as currently available on a particular beacon node takes around 20 minutes. Doing the same thing by fetching state takes around 15 seconds. As such, there really isn't a viable fallback.

There are potential options coming up, with a POST-based /validators call but that also needs SSZ support to be useful (something for which I'm on the hook, and hoping to get to over the holidays).

Perhaps it makes sense to ask the prysm folks to enable the debug namespace by default?