DNS TTL Not Respected by celestia-core Leading to Potential Future Sync Issues

[smuu]

Description:

We identified a potential issue where changes to the DNS entries of nodes in celestia-core are not respected, similar to the issue observed in celestia-node. Although we have not yet encountered sync issues due to this behavior, it could pose a problem in the future. It appears that nodes resolve DNS entries only once at startup and continue using the same IP address indefinitely, ignoring DNS TTL.

Steps to Reproduce:

1. Change the DNS entries for nodes.
2. Observe that nodes continue to use the old IP address without re-resolving the DNS entries.

Suspected Cause:
Nodes resolve DNS entries only once at startup and continue using the same IP address without respecting the TTL. This could affect:

• Nodes connecting to other nodes within the network.

Relevant Code:

• DNS resolution implementation in celestia-core:

  celestia-core/p2p/peer.go

  Line 140 in c2e6331

  func (pc peerConn) RemoteIP() net.IP {

Potential Fix:

1. Periodically re-resolve DNS entries based on the TTL.
2. Update active connections if the resolved IP address changes.

Repositories Potentially Needing Changes:

• celestiaorg/celestia-core
• cometbft/cometbft (upstream implementation of Tendermint)

Impact:
Not respecting DNS TTL can lead to potential connectivity and sync issues, affecting network reliability in the future.

Request for Assistance:

1. Implement periodic DNS resolution based on TTL.
2. Test changes to ensure nodes dynamically update connections based on DNS updates.

[smuu]

For reference the issue in celestia-node: celestiaorg/celestia-node#3570

[smuu]

One workaround for this issue would be to recreate the connection once it fails after the IP address changes. This way, we don't need to add support to handle the DNS TTL, and the node would request the new IP address from the DNS server.

[cmwaters]

Yeah, we could add code such that if the connection failed it would try resolve the DNS again to see if there had been an IP change and if so reconnect to the peer. Ideally we'd only try connect if the error specifically had to do with the network connection and not some malicious behaviour from the peer