holepunchto/hyperbee

Possible to create history stream on sub database?

Closed this issue · 2 comments

My project has two sub databases in a single hyperbee called payload DB and a meta DB that each use the same key but have different values
Given a sub database called metaDB expect only keys from that sub database to stream.
Currently this returns "data" twice once for each sub DB and no clear way to deconflict which "data" other than inspecting the values

  watch(cb) {
    const rs = this.metaDB.createHistoryStream({ live: true })
    rs.on("data", (data) => {
      console.log("watch", data);
      cb(null, data);
    });
  }

Here is how they are initialized

db = new Hyperbee(core, {
  keyEncoding: "utf-8",
  valueEncoding: "binary",
});

const metaDB = db.sub("meta", {
  valueEncoding: "utf-8",
});

const dataDB = db.sub("data", {
  valueEncoding: "binary",
});

returns data like

watch {
type: 'put',
seq: 105,
key: '1ea746dc-d050-41fd-bbba-5d33a50afdc3',
value: '{"type":"text/html","etag":"a622c40426aedb88f24559877cb8c6df66ed527c","template":"article"}'
}
WATCH [ '2d723249-5d0e-46ba-b1c4-df0014c107ab' ]
SSE:RELOAD 2d723249-5d0e-46ba-b1c4-df0014c107ab
watch {
type: 'put',
seq: 106,
key: '1ea746dc-d050-41fd-bbba-5d33a50afdc3',
value: '<h2>hello world</h2>'
}
WATCH [ '2d723249-5d0e-46ba-b1c4-df0014c107ab' ]
SSE:RELOAD 2d723249-5d0e-46ba-b1c4-df0014c107ab

My use case is live reloading large dependency trees in the browser when a file changes so I only need the meta data to build that graph not the values of the "payload"
Maybe there is another pattern that would better support? thinking about 2 hyperbees now.

We should parse the history here, but a bit tricky here as we don’t know if the key there is meant to look like that or is a sub.

However, you pattern sounds like you wanna do a diff on the subs instead triggered by the append event on the main instance or core. Will be much more efficient

Ill dive into this more in the future, for now I am just listening to the entire stream and refreshing any time I see the key show up.