holepunchto/hyperbee

Error in `createDiffStream` when using sub()

Closed this issue · 6 comments

pfrazee commented

I have some code which calls createDiffStream() on a sub():

    const co = this.db.bee.checkout(other).sub(this._schemaDomain).sub(this._schemaName)
    return new Promise((resolve, reject) => {
      pump(
        co.createDiffStream(this.bee.version),
        concat(resolve),
        err => {
          pend()
          if (err) reject(err)
        }
      )
    })

This will sometimes produce an error:

TypeError: Cannot read property 'key' of undefined
    at TreeIterator.nextKey (/Users/paulfrazee/work/ctzn/node_modules/hyperbee/iterators/diff.js:110:21)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async DiffIterator._next (/Users/paulfrazee/work/ctzn/node_modules/hyperbee/iterators/diff.js:168:43)
    at async DiffIterator.next (/Users/paulfrazee/work/ctzn/node_modules/hyperbee/iterators/diff.js:154:17)

I can currently work around it by running createDiffStream() on the root and filtering the results:

    const co = this.db.bee.checkout(other)
    const diffs = await new Promise((resolve, reject) => {
      pump(
        co.createDiffStream(this.bee.version),
        concat(resolve),
        err => {
          pend()
          if (err) reject(err)
        }
      )
    })
    const prefix = `${this._schemaDomain}\x00${this._schemaName}\x00`
    return diffs.filter(diff => {
      const key = (diff.right||diff.left).key
      if (key.startsWith(prefix)) {
        if (diff.left) diff.left.key = diff.left.key.slice(prefix.length)
        if (diff.right) diff.right.key = diff.right.key.slice(prefix.length)
        return true
      }
      return false
    })
andrewosh commented

@pfrazee Not surprised that the sub key fiddling causes trouble for diffs. Do you have a good test case for this?

pfrazee commented

@andrewosh psh you expect me to make a test case, don't you know who I am, I have over 900 subscribers on youtube

Okay but seriously I am pretty lazy so I'll just quick-write one here:

await bee.sub('sub1').bar('sub1').put('test1_1', 1)
await bee.sub('sub1').bar('sub1').put('test1_1', 2)
await bee.sub('sub1').bar('sub2').put('test1_2', 1)
await bee.sub('sub1').bar('sub1').put('test1_1', 3)
await bee.sub('sub1').bar('sub2').put('test1_2', 2)
await bee.sub('sub1').bar('sub2').put('test1_2', 3)
await bee.sub('sub1').put('test1', 1)
await bee.sub('sub1').put('test1', 2)
await bee.sub('sub1').bar('sub2').put('test1_2', 4)
await bee.sub('sub1').put('test1', 3)

const co = this.db.bee.checkout(2).sub('sub1').sub('sub2')
const res = await new Promise((resolve, reject) => {
  pump(
    co.createDiffStream(bee.version),
    concat(resolve),
    err => {
      pend()
      if (err) reject(err)
    }
  )
})

I'm pretty sure this or some variation of it will cause createDiffStream to throw.

pfrazee commented

(Also a funny side-note, if you try to do createDiffStream() from a checkout seq lower than 2 it fails completely, afaict because on seq=0 and seq=1 the root nodes aren't in the hypercore yet so hyperbee chokes)

andrewosh commented

Sir this is a Hyperbee's and those kinds of subs aren't relevant here. I'll muck around with your code, interpreting the bar method as I see fit, and see what's up

pfrazee commented
mafintosh commented

This is fixed (at least this doesn't crash anything on latest and there was some diff bug fixes there)