ipfs/go-graphsync

Support caching layer for metadata traversals on the responder to answer future requests quickly

hannahhoward opened this issue · 0 comments

Once we build Graphsync metadata only requests, the next step is to build a way to cache them.

Rather than try to build this in go-graphsync, I'd like for it to be an optional dependency (passed through config options at initialization) that satisfies an interface something like this:

type MetadataWriter interface {
   Write(path ipld.Path, c cid.Cid, linkAction graphsync.LinkAction) error
   Finish() error
}

type MetadataCache interface {
   Has(root cid.Cid, selector ipld.Node) (bool, error)
   Get(root cid.Cid, selector ipld.Node) (graphsync.LinkMetadata, error)
   Begin(root cid.Cid, selector ipld.Node) (MetadataWriter, error)
}

When present, at the beginning of serving a metadata request, a response manager would check for the presence of a cached set of responses, and short circuit the process if they were present.

If not present, it call Begin and then as the response executed, call Write on each CID traversed, and call Finish at the end.

It's worth considering if there is a generalized version of this cache that might be useful for selector traversals in general, and perhaps this should be tackled in collaboration with IPLD stewards.

The path parameter exists on the Write call so that an implementer wanting to build a more semantic cache can do so.

Perhaps it even makes sense to add a parameter to write that captures the current selector at the given path, but this probably doesn't make sense to do until we have a way to convert back to a non-compiled selector at a given node.