GetRangeAsync with LastLessOrEqual seems broken when reading your own writes

[packetship]

Hi,

This is in the Nuget release version 6.2.0-preview (can't test in 7.x, sorry)

The following code works when reading back keys in a previous transaction, but not if the keys were created in the same transaction:

      // startKey, endKey are Slices
      KeySelector begin = KeySelector.FirstGreaterOrEqual(startKey);
      KeySelector end = KeySelector.LastLessOrEqual(endKey);
      var range = await tr.GetRangeAsync(begin, end, null);
      // Check length of range

So if I have a transaction creating Key{0..9} and then read back between startKey='Key2' and endKey='Key7' with the above code within the same transaction, I only get one key back (Key2). If I do the creation in a previous transaction, I get 5 keys, Key{2..6} as expected.

If I use KeySelector.LastLessThan it returns 4 keys as expected, even in the same transaction (but not what's required).

Not sure if this is actually the C# API or the underlying C library!

Many thanks

Paul

[KrzysFR]

The KeySelector type is wrapping as-is the underlying C library.

Are you sure the startKey/endKey are properly formatted? Are you using tuples?

There are common issues with key selectors that could explain your issue.

• the key selector resolve a key that exists in the database (or in the local mutations if the transaction is not yet comitted)
• for the end selector, the range read will stop BEFORE the resolved key. So the resolved key will NOT be returned in the range. You have to select the key that would be after the last one you want.
• Key are not encoded as you would expect so they are not ordered "correctly" (using tuple encoding helps this)
• GetRangeAsync(...) does not guarantee that it will return everything in a single call! it's returning chunks and you are supposed to call back again if HasMore is true. That's why using tr.GetRange(...).ToListAsync() is preferred (the ToListAsync will call GetRangeAsync(...) multiple times as needed. The GetRangeAsync method will only return a single page, while GetRange creates an AsyncEnumerable that will call GetRangeAsync repeatedly.
• Doing snapshot reads instead of normal reads (snapshot reads don't see the local changes) but I don't think that's the case here.

Could you include a bit more of the test code? Seeing how keys are encoded and how you create the transaction could help.

[packetship]

Wow, thanks for your fast response!

The keys are literally just Key0, Key1 ... created through essentially

var startKey = Slice.Copy(Encoding.UTF8.GetBytes("Key"+i).Span)
(that's a bunch of code coalesced!). They read back as just "Key0" in fdbcli too, and this works perfectly with separate write and read transactions.

Understood about local mutations and the half-open interval etc. I think your point about the chunking could be the issue - I will try that. But weird how that would happen with LastLessOrEqual and not LastLessThan?

Thanks again

Paul

[packetship]

Yes, that fixes it, thanks!

So I my guess is that I just got lucky when using LastLessThan but not with LastLessOrEqual (although it was very consistent...)

TL;DR for any others passing this way: Use tr.GetRange(...).ToListAsync() rather than tr.GetRangeAsync(...) if you want all the values in one hit.

[KrzysFR]

You should probably not use binary keys, and use tuples (look at TuPack). Also, if you want to create slice more easily, look at the static methods on the Slice type, there are a lot to create from utf8 string, ints, longs, guid, etc...

But TuPack.Pack(....) will create a better key for complex keys (one or more elements). This uses the same encoding as all other layers so is compatible with everything.