Possible to create history stream on sub database?
Closed this issue · 2 comments
My project has two sub databases in a single hyperbee called payload
DB and a meta
DB that each use the same key but have different values
Given a sub
database called metaDB
expect only keys from that sub database to stream.
Currently this returns "data" twice once for each sub DB and no clear way to deconflict which "data" other than inspecting the values
watch(cb) {
const rs = this.metaDB.createHistoryStream({ live: true })
rs.on("data", (data) => {
console.log("watch", data);
cb(null, data);
});
}
Here is how they are initialized
db = new Hyperbee(core, {
keyEncoding: "utf-8",
valueEncoding: "binary",
});
const metaDB = db.sub("meta", {
valueEncoding: "utf-8",
});
const dataDB = db.sub("data", {
valueEncoding: "binary",
});
returns data like
watch {
type: 'put',
seq: 105,
key: '1ea746dc-d050-41fd-bbba-5d33a50afdc3',
value: '{"type":"text/html","etag":"a622c40426aedb88f24559877cb8c6df66ed527c","template":"article"}'
}
WATCH [ '2d723249-5d0e-46ba-b1c4-df0014c107ab' ]
SSE:RELOAD 2d723249-5d0e-46ba-b1c4-df0014c107ab
watch {
type: 'put',
seq: 106,
key: '1ea746dc-d050-41fd-bbba-5d33a50afdc3',
value: '<h2>hello world</h2>'
}
WATCH [ '2d723249-5d0e-46ba-b1c4-df0014c107ab' ]
SSE:RELOAD 2d723249-5d0e-46ba-b1c4-df0014c107ab
My use case is live reloading large dependency trees in the browser when a file changes so I only need the meta data to build that graph not the values of the "payload"
Maybe there is another pattern that would better support? thinking about 2 hyperbees now.
We should parse the history here, but a bit tricky here as we don’t know if the key there is meant to look like that or is a sub.
However, you pattern sounds like you wanna do a diff on the subs instead triggered by the append event on the main instance or core. Will be much more efficient
Ill dive into this more in the future, for now I am just listening to the entire stream and refreshing any time I see the key show up.