threefoldtech/mycelium

Improve contention on source table when sending updates

Closed this issue · 0 comments

Currently sending an update to a peer first checks if the source table needs to be updated, taking a write lock to do so (and if necessary apply said update). This happens just before the update is queued to the peer, in line with the RFC. The issue here is that this fuction is called for every peer which gets the update. This means that for every peer that gets the update, the source table lock is acquired. Since processing an update also takes a (read) lock on the source table, this leads to contention. For instance, every 1 minute, the peers currently get a full route table dump (also a specified in the rfc). This means that the lock is acquired in write mode M*N times, where M is the size of the network, and N is the amount of local peers. Especially for the public nodes this leads to a rather large amount of contention. While the source table should be reworked a bit to perform more of a RCU pattern for updates, we can already improve now by only calculating the effect on the source table once per route (since we advertise the same update to all peers).