slpMempoolIgnoreList is limited to 10,000 items
jcramer opened this issue · 2 comments
slpMempoolIgnoreList is limited to 10000 items, with blocks larger
than 2MB this number may need to be larger. When this limit is exceeded
every non-SLP zmq transaction notification (prior to block) will cause
a RPC call to getRawTransaction(). To fix this, we will want to switch to
using a raw transaction hex notification, rather than txn hash notification,
raw transaction hex notification.
Line 148 in 894d147
slpMempoolIgnoreList.includes does a linear scan, we can probably improve this by creating a slpMempoolIgnoreSet as well and doing lookups from that, as in we add a txid to both the List and the Set, then once the list is over N size we just do Set.delete(List.shift());
Using this with 1 million txids (100x increase in size) i have timing of 1ms vs 26ms for running has/includes on something this size.
It would be nice if N size was operator configurable for low memory applications (or for easy upgrade with bigger blocks).
Closing as this solved this issue, ty @blockparty-sh