Boot nodes refuse connection when running InitializeBootnodes() @ DiscoveryApp.cs | Optimism mainnet

Description

Nethermind 1.28.0, every single bootnode from ChainSpec returns connection reset RST,ACK when attempting to contact them during normal startup procedures in Nethermind.Network.Discovery's DiscoveryApp.cs.

Default op-mainnet.cfg;
Client execution command line args: -c op-mainnet -dd \"x:/xxxxxxxx\" --JsonRpc.JwtSecretFile \"x:/xxxxxxxx/jwt/jwtsecret\" --JsonRpc.Host \"x.x.x.x\" --JsonRpc.EnginePort 8552 --JsonRpc.EngineHost \"x.x.x.x\" --JsonRpc.Enabled true --HealthChecks.Enabled true --HealthChecks.UIEnabled true --Network.MaxActivePeers 250 --Pruning.Mode=Hybrid --Pruning.FullPruningTrigger=StateDbSize --Pruning.FullPruningThresholdMb=500000 --JsonRpc.EnabledModules=\"[Eth, Subscribe, Trace, TxPool, Web3, Personal, Proof, Net, Parity, Health, Rpc, Admin]\"

Consensus Client op-node compiled from source (op-stack v1.9.1)

Tested on two separate systems with different WAN IPs and the same exact config:
Ubuntu 22.04/i9 14900KF/64GB RAM
Windows 10/i9 13900KF/32GB RAM

Screenshots

PingMsg object's fields when constructed and sent:

Logs

Could not communicate with any nodes (bootnodes, trusted nodes, persisted nodes). Since none of the nodes actually establish connection and therefore NodeLifecycleState.Active is never set.

Bootnodes are just for discovery(UDP), still the client tries to connect to them via devp2p(TCP) to query headers and other things, and it fails, which is OK.

Also the screenshot displays PingMsg usage which is from discv4 - previous version of discovery, which is not expected, op-mainnet has Discv5 enabled. Could you confirm op-mainnet config is used?

Ah, then I suggest adding "DiscoveryVersion": "V5" to op-mainnet.cfg's Discovery config object, since otherwise it wont be expressly set from any IConfigSource and will default to V4 unless user specifies it. You probably meant to refactor this line

nethermind/src/Nethermind/Nethermind.Runner/configs/op-mainnet.cfg

Lines 24 to 26 in 1fc24b6

    
           "Discovery": { 
        
             "Discv5Enabled": true 
        
           },

Indeed! This is definitely a config issue, thanks for the report! Let me make a pr