SkipBlockDB.getAllSkipchains is very slow
ineiti opened this issue · 0 comments
ineiti commented
That method is used to get all skipchain IDs. It loads every skipblock, which is very slow, probably due to the network.Unmarshal
, which gets the whole structure, including the roster and the forward-links with every possible new roster.
For the skipchain IDs, it would be enough to only load the Hash
, GenesisID
and Index
fields. With protobuf, this could be done like:
type SkipBlockShort struct {
Index int `protobuf:"0"`
GenesisID []byte `protobuf:"6"`
Hash []byte `protobuf:"9"`
}
Then the decoding in getAllSkipchains
should be done using protobuf.Decode
:
var sbs SkipBlockShort
protobur.Decode(v[16:], &sbs)
This should give a big speed-up in decoding and getting all the skipchain-IDs.
Another way would be to add an additional field that holds all skipchain-IDs, but that would be more work...