This library bundles different components for lower-level peer-to-peer connection and message exchange:
- Distributed Peer Table (DPT) / Node Discovery
- RLPx Transport Protocol
- Ethereum Wire Protocol (ETH)
- Light Ethereum Subprotocol (LES/2)
The library is based on ethereumjs/node-devp2p as well
as other sub-libraries (node-*
named) (all outdated).
This library has to be compiled with typescript to a Node 6
and above friendly source format.
For triggering a (first) build to create the dist/
directory run:
npm run build
You can also use ts-node
to run a script without first transpiling to js (you need to npm i --save-dev ts-node
first):
node -r ts-node/register [YOUR_SCRIPT_TO_RUN.ts]
All components of this library are implemented as Node EventEmitter
objects
and make heavy use of the Node.js network stack.
You can react on events from the network like this:
dpt.on('peer:added', (peer) => {
// Do something...
})
Basic example to connect to some bootstrap nodes and get basic peer info:
Communicate with peers to read new transaction and block information:
Run an example with:
DEBUG=devp2p:* node -r ts-node/register ./examples/peer-communication.ts
Maintain/manage a list of peers, see ./src/dpt/, also includes node discovery (./src/dpt/server.ts)
Create your peer table:
const dpt = new DPT(Buffer.from(PRIVATE_KEY, 'hex'), {
endpoint: {
address: '0.0.0.0',
udpPort: null,
tcpPort: null
}
})
Add some bootstrap nodes (or some custom nodes with dpt.addPeer()
):
dpt.bootstrap(bootnode).catch((err) => console.error('Something went wrong!'))
See the following diagram for a high level overview on the library.
Distributed Peer Table. Manages a Kademlia DHT K-bucket (Kbucket
) for storing peer information
and a BanList
for keeping a list of bad peers. Server
implements the node discovery (ping
,
pong
, findNeighbours
).
Creates new DPT object
privateKey
- Key for message encoding/signing.options.refreshInterval
- Interval in ms for refreshing (callingfindNeighbours
) the peer list (default:60s
).options.createSocket
- A datagram (dgram)createSocket
function, passed toServer
(default:dgram.createSocket.bind(null, 'udp4')
).options.timeout
- Timeout in ms for serverping
, passed toServer
(default:10s
).options.endpoint
- Endpoint information to send with the serverping
, passed toServer
(default:{ address: '0.0.0.0', udpPort: null, tcpPort: null }
).
Uses a peer as new bootstrap peer and calls findNeighbouts
.
peer
- Peer to be added, format{ address: [ADDRESS], udpPort: [UDPPORT], tcpPort: [TCPPORT] }
.
Adds a new peer.
object
- Peer to be added, format{ address: [ADDRESS], udpPort: [UDPPORT], tcpPort: [TCPPORT] }
.
For other utility functions like getPeer
, getPeers
see ./src/dpt/dpt.ts.
Events emitted:
Event | Description |
---|---|
peer:added | Peer added to DHT bucket |
peer:removed | Peer removed from DHT bucket |
peer:new | New peer added |
listening | Forwarded from server |
close | Forwarded from server |
error | Forwarded from server |
Connect to a peer, organize the communication, see ./src/rlpx/
Create your RLPx
object, e.g.:
const rlpx = new devp2p.RLPx(PRIVATE_KEY, {
dpt: dpt,
maxPeers: 25,
capabilities: [
devp2p.ETH.eth63,
devp2p.ETH.eth62
],
listenPort: null
})
Manages the handshake (ECIES
) and the handling of the peer communication (Peer
).
Creates new RLPx object
privateKey
- Key for message encoding/signing.options.timeout
- Peerping
timeout in ms (default:10s
).options.maxPeers
- Max number of peer connections (default:10
).options.clientId
- Client ID string (default example:ethereumjs-devp2p/v2.1.3/darwin-x64/nodejs
).options.remoteClientIdFilter
- Optional list of client ID filter strings (e.g.['go1.5', 'quorum']
).options.capabilities
- Upper layer protocol capabilities, e.g.[devp2p.ETH.eth63, devp2p.ETH.eth62]
.options.listenPort
- The listening port for the server ornull
for default.options.dpt
-DPT
object for the peers to connect to (default:null
, noDPT
peer management).
Manually connect to peer without DPT
.
peer
- Peer to connect to, format{ id: PEER_ID, address: PEER_ADDRESS, port: PEER_PORT }
.
For other connection/utility functions like listen
, getPeers
see ./src/rlpx/rlpx.ts.
Events emitted:
Event | Description |
---|---|
peer:added | Handshake with peer successful |
peer:removed | Disconnected from peer |
peer:error | Error connecting to peer |
listening | Forwarded from server |
close | Forwarded from server |
error | Forwarded from server |
Upper layer protocol for exchanging Ethereum network data like block headers or transactions with a node, see ./src/eth/.
Send the initial status message with sendStatus()
, then wait for the corresponding status
message
to arrive to start the communication.
eth.once('status', () => {
// Send an initial message
eth.sendMessage()
})
Wait for follow-up messages to arrive, send your responses.
eth.on('message', async (code, payload) => {
if (code === devp2p.ETH.MESSAGE_CODES.NEW_BLOCK_HASHES) {
// Do something with your new block hashes :-)
}
})
See the peer-communication.ts
example for a more detailed use case.
Handles the different message types like NEW_BLOCK_HASHES
or GET_NODE_DATA
(see MESSAGE_CODES
) for
a complete list. Currently protocol versions PV62
and PV63
are supported.
Normally not instantiated directly but created as a SubProtocol
in the Peer
object.
version
- The protocol version for communicating, e.g.63
.peer
-Peer
object to communicate with.send
- Wrappedpeer.sendMessage()
function where the communication is routed to.
Send initial status message.
status
- Status message to send, format{ networkId: CHAIN_ID, td: TOTAL_DIFFICULTY_BUFFER, bestHash: BEST_HASH_BUFFER, genesisHash: GENESIS_HASH_BUFFER }
.
Send initial status message.
code
- The message code, seeMESSAGE_CODES
for available message types.payload
- Payload as a list, will be rlp-encoded.
Events emitted:
Event | Description |
---|---|
message | Message received |
status | Status info received |
Upper layer protocol used by light clients, see ./src/les/.
Send the initial status message with sendStatus()
, then wait for the corresponding status
message
to arrive to start the communication.
les.once('status', () => {
// Send an initial message
les.sendMessage()
})
Wait for follow-up messages to arrive, send your responses.
les.on('message', async (code, payload) => {
if (code === devp2p.LES.MESSAGE_CODES.BLOCK_HEADERS) {
// Do something with your new block headers :-)
}
})
See the peer-communication-les.ts
example for a more detailed use case.
Handles the different message types like BLOCK_HEADERS
or GET_PROOFS_V2
(see MESSAGE_CODES
) for
a complete list. Currently protocol version LES/2
running in client-mode is supported.
Normally not instantiated directly but created as a SubProtocol
in the Peer
object.
version
- The protocol version for communicating, e.g.2
.peer
-Peer
object to communicate with.send
- Wrappedpeer.sendMessage()
function where the communication is routed to.
Send initial status message.
status
- Status message to send, format{ networkId: CHAIN_ID, headTd: TOTAL_DIFFICULTY_BUFFER, headHash: HEAD_HASH_BUFFER, headNum: HEAD_NUM_BUFFER, genesisHash: GENESIS_HASH_BUFFER }
.
Send initial status message.
code
- The message code, seeMESSAGE_CODES
for available message types.reqId
- Request ID, will be echoed back on response.payload
- Payload as a list, will be rlp-encoded.
Events emitted:
Event | Description |
---|---|
message | Message received |
status | Status info received |
There are unit tests in the test/
directory which can be run with:
npm run test
This library uses debug debugging utility package.
For the debugging output to show up, set the DEBUG
environment variable (e.g. in Linux/Mac OS:
export DEBUG=*,-babel
).
Use the DEBUG
environment variable to active the logger output you are interested in, e.g.:
DEBUG=devp2p:dpt:*,devp2p:eth node -r ts-node/register [YOUR_SCRIPT_TO_RUN.ts]
For more verbose output on logging (e.g. to output the entire msg payload) use the verbose
logger
in addition:
DEBUG=devp2p:dpt:*,devp2p:eth,verbose node -r ts-node/register [YOUR_SCRIPT_TO_RUN.ts]
Exemplary logging output:
Add peer: 52.3.158.184:30303 Geth/v1.7.3-unstable-479aa61f/linux-amd64/go1.9 (eth63) (total: 2)
devp2p:rlpx:peer Received body 52.169.42.101:30303 01c110 +133ms
devp2p:rlpx:peer Message code: 1 - 0 = 1 +0ms
devp2p:rlpx refill connections.. queue size: 0, open slots: 20 +1ms
devp2p:rlpx 52.169.42.101:30303 disconnect, reason: 16 +1ms
Remove peer: 52.169.42.101:30303 (peer disconnect, reason code: 16) (total: 1)
To update the structure diagram files in the root folder open the devp2p.drawio
file in draw.io, make your changes, and open a PR with the updated files. Export svg
and png
with border
width=20
and transparency=false
. For png
go to "Advanced" and select 300 DPI
.
The following is a list of major implementations of the devp2p
stack in other languages:
- Python: pydevp2p
- Go: Go Ethereum
- Elixir: Exthereum
- Blog article series on implementing Ethereum protocol stack
MIT