How should `OpLog`s exchange edits?

[noib3]

Hi, I'm starting to explore the API and internals of DT starting from this simple example, but I must be using the API incorrectly since it panics at the first unwrap().

// Insert "ab" at site 1 and send the document over to 2 other sites.

let mut oplog1 = OpLog::new();

let site1 = oplog1.get_or_create_agent_id("1");

let ab = oplog1.add_insert(site1, 0, "ab");

let ab = oplog1.encode_from(EncodeOptions::default(), &[ab]);

let mut oplog2 = OpLog::new();

oplog2.decode_and_add(&ab).unwrap(); // -> BaseVersionUnknown

let mut oplog3 = OpLog::new();

oplog3.decode_and_add(&ab).unwrap();

// At this point all the documents should contain "ab".

let doc1 = oplog1.checkout_tip();
let doc2 = oplog2.checkout_tip();
let doc3 = oplog3.checkout_tip();

assert_eq!(doc1.content(), "ab");
assert_eq!(doc2.content(), "ab");
assert_eq!(doc3.content(), "ab");

Unfortunately the BaseVersionUnknown error variant is not documented and a first (albeit cursory) look at the source code wasn't illuminating.

I'm using the latest version published on crates.io which should be 8fc685d.

[josephg]

Thanks for the bug report. I can reproduce it on master. I'll take a look!

[josephg]

Oh, the problem is that you're calling encode_from(EncodeOptions::default(), &[ab]);. This encodes all the changes from some specific version (in your case, the head version of ab). So it makes a dataset based on that version, with no changes. Then you try to merge it into an empty document - but it can't because its missing the parent versions.

Changing the encode_from line to encode makes the test pass:

    let ab = oplog1.encode(EncodeOptions::default());

[noib3]

Oh I see, it encodes from but not including that version. In fact it also works if I use encode_from with an empty slice.

[josephg]

Yeah, an empty slice is semantically equivalent to the start of the document's history.

(long winded reason):

In diamond types we have the idea of a frontier (called HEADs in automerge). This is a set of versions (usually 1) which also implicitly includes everything that version depends on. And everything those versions depend on, and so on back to the start of the document's history. Thats what you're passing to encode_from - a frontier (a set of versions) which also transitively implies all the parents, and so on. encode_from is a function which returns an encoded copy of all operations not included in frontier you passed via the parameter. It exists to let you gather the operations that a remote peer is missing to bring them up to date.

An empty slice includes no versions. So everything minus the empty set, which is everything. I'd have to check, but I think encode is just an alias for encode_from(&[]).

[noib3]

Yeah I've heard you mention the "frontier" a bunch of times in your Braid talks but I never had a good definition for it.

Thanks a lot for this extended explanation, it might be a good idea to add it to the docs of either encode_from or OpLog.