dedis/cothority

SkipBlockDB.getAllSkipchains is very slow

ineiti opened this issue · 0 comments

That method is used to get all skipchain IDs. It loads every skipblock, which is very slow, probably due to the network.Unmarshal, which gets the whole structure, including the roster and the forward-links with every possible new roster.

For the skipchain IDs, it would be enough to only load the Hash, GenesisID and Index fields. With protobuf, this could be done like:

type SkipBlockShort struct {
    Index int `protobuf:"0"`
    GenesisID []byte `protobuf:"6"`
    Hash []byte `protobuf:"9"`
}

Then the decoding in getAllSkipchains should be done using protobuf.Decode:

var sbs SkipBlockShort
protobur.Decode(v[16:], &sbs)

This should give a big speed-up in decoding and getting all the skipchain-IDs.

Another way would be to add an additional field that holds all skipchain-IDs, but that would be more work...