Refactor: The Great Refactor
Opened this issue · 0 comments
The Great Refactor
This is essentially a new Epic that describes a merging of Database, Store, and Client into a single, unified JS ThreadDB API.
Overview
The primary plan is to “shift” support for threads-database
in favor of threads-client
. This limits our API flexibility in some respects, but what we lose in control, we gain in accessibility and a more streamlined development focus. Additionally, we will deprecate threads-network-client
and threads-network
, which are low-level Threads protocol libraries. These were never really designed to be publicly consumed modules, so this isn’t likely to be a controversial move. Finally, threads-encoding
will be moved to its own repo and maintained for demos and examples that want to build on Threads more directly. The primary change will be to marry threads-database
and threads-store
on top of threads-client
, to provide an offline-first database with fast MongoDB style APIs and queries, and a Realm/MongoDB like local/remote sync interface.
Goals/Outcomes
The goals/outcomes of this refactor are to be able to provide a PouchDB + CouchDB style experience for developers that syncs data to remote peers that can then serve said data over decentralized protocols. Something like, “the dynamic database that syncs to IPFS”. Another analogy would be MongoDB + Realm, where you have a free, mobile/browser database that syncs with a remote MongoDB service. The resultant library be familiar to anyone who has built an application using something like Realm, Pouch, or Mongo. The difference here is everything is available over decentralized protocols such as PubSub/GossipPub, IPFS, IPNS, HTTP gateways, and more.
Roadmap
- Migration
- Drop
threads-network-client
andthreads-network
- Move
threads-encoding
to its own repo (which already exists) - Consolidate and combine
threads-database
withthreads-store
- Import
threads-client
intothreads-database
as the primary networking/DB layer - Deprecate
threads-client
in favor of the newthreads-database
(which is now a nicely wrapped threads-client) - Update local (within repo/APIs) documentation to reflect these migrations
- Drop
- Local-first
- Upgrade local-first behavior to support
threads-client
updates - Implement required remote sync API(s) in
go-threads
to support faster local caching - Update local (within repo/APIs) documentation to reflect these new features
- Upgrade local-first behavior to support
- P2P
- Expose Pubsub updates at the database level
- Expose APIs for accessing database collections and entities directly from the JS clients
- Return IPFS CIDs for all database updates (rather than Instance IDs alone)
- Update local (within repo/APIs) documentation to reflect these new features
- DX
- Refocus primary developer entry-point to
js-hub
(this is mostly already done) - Move from current low level auth access to higher-level, token-based, browser-first implementation. For NodeJS applications, the low-level APIs will remain, or they can utilize a compatibility layer for cookie/header management.
- Update local (within repo/APIs) documentation to reflect these changes
- Refocus primary developer entry-point to
- Documentation - All of the above will require major documentation updates
- Update README of
js-threads
to clearly state primary goals/functionality of the updated library - Additional information on Threads inner-workings should be directed to high-level white paper overview or Go Threads
- Re-working of primary auth flow (now more "browser native") examples and demos
- Port all existing Database demos to the new Database design (this is likely only the Chat demo)
- Refocus primary JS client focus to Buckets where appropriate
- Update README of