Scalability in p2p network
av8ta opened this issue · 3 comments
Hi I'm using a p2p network that has an immutable log for each user (secure scuttlebutt). It's a kappa architecture / event sourced p2p database. What I want to do is make it CQRS by not using the database for querying, but instead run through the log inserting messages into eliasdb. That way eliasdb can be the read side only via graphql or EQL.
I'm guessing scaling should be no problem because even though ssb comprises a global network each user generally only stores their data, their friends, and friends of friends. Blobs are stored separately to the log which is json so the log isn't actually that big. I have a huge log stored locally after setting hops=6 to get a lot of data! If it works on my log then it will be fine! It's 1.3 gig, but a normal log would only be 200-300 meg of json messages.
contact messages are used to declare follow, unfollow, or block. I threw them all with directed edges into a graphology graph the other day and there are about 390,000 edges between users. I forget how many nodes. So that's what a large graph would look like. What do you think? Would eliasdb handle that scale, and would it handle the type of queries I would need for a social network?
Also, will it run in the browser with wasm? ssb is mostly server side but there are some browser based solutions.
Hey there, unfortunately, I don't know the answer 😃 - EliasDB doesn't have a hard scale limit as such. So I guess it depends mostly on the resources it has available. re: WASM - I am not sure either. I don't see why not but running a DB in the browser sounds a bit strange - though I don't have a lot of knowledge here.
Yes running databases in the browser is a bit strange in our master/slave (server/client) architecture of the current web! But in the p2p world it makes perfect sense to have a database in the browser because it helps to make peers equal. Sir Tim's original browser was both client and server, it was easy to publish. Unfortunately when an intern was tasked to build a cross platform browser it was easier to build only a client... and so the web was broken at the beginning :(
Databases in the browser are a great step in the right direction :)