apankrat/nullboard

Switch to IndexedDB

Opened this issue · 15 comments

apankrat commented

Get rid of localStorage, basically.

manup commented

localForage might help to make the switch easy https://github.com/localForage/localForage

transiient commented

Why, though? NB is built to be dead simple, there isn't a lot of data which needs to be stored, so I don't think there's anything wrong with using localStorage.

Edit: NB allows exporting/importing so the cache issue is slightly less of an issue. I guess it's a tradeoff between staying super lightweight, and allowing additional data persistence.

  • 👍1
apankrat commented

Using localStorage starts causing stalls in Firefox when there are several boards that are well populated and active (so there's quite a bit of undo data). Changing to something a bit more robust is a very reasonable idea. I just need to find time to do it.

  • 👍1
transiient commented

Ah, fair enough then! localForage uses Promises, which I'm fairly new to, so I'll attempt to migrate but don't count on it

quantuumsnot commented

Using localStorage starts causing stalls in Firefox when there are several boards that are well populated and active (so there's quite a bit of undo data). Changing to something a bit more robust is a very reasonable idea. I just need to find time to do it.

There was a Web SQL directly supported by Chrome and FF and they just dropped it. Was perfect because it was SQLite directly accessed from the browser, now we have the annoying IndexedDB. And yeah, browsers get laggy when a lot of things are saved to localStorage/IndexedDB

Arteneko commented

As I have both a need for a more reliable persistent local storage and possible synchronization, I'm planning to find time to see into doing this switch, may submit a PR in time.

tionis commented

How about using pouchdb?

apankrat commented

@Arteneko - I hear you on synchronization. Obviously, the tricky part is the conflict resolution or, more generally, reconciling changes made to the same original on different devices.

A couple of thoughts on this I had was:

  1. First, do a simple backup/restore to/from a remote storage first. This will at least enable switching between 2+ devices for editing the DB (i.e. edit-on-A, save, pull-from-B, edit-on-B, save, pull-from-A, edit-on-A...) and it will also allow sharing copies of it. This doesn't require anything more from the storage host than an option of storing blobs of data. Blobs can also be encrypted at source, so the storage doesn't even need to be trusted by the NB devices.

  2. Second, see if to adopt git repos for local storage. This will take care of undo/redo functionality and it should also simplify merges and conflict resolution.

  3. I am also looking at making a small native program (e.g. a regular Windows executable) that the page will be able to talk to via websockets (or even plain HTTP). The program will take care of maintaining NB's "database" on the local file system.

Something like this, a rudimentary "roadmap". No ETA however, not at the moment.

@tionis - I'll have a look, thanks.

  • 👍1
tionis commented

Just to make the case for pouchdb:

  • PouchDB is javascript implementation for client and server of apache couchdb
  • It is a document store (in json) that is designed to sync between many devices
  • It allows you to handle conflicts explicitly
  • It's easy to sink to an upstream database and such databases are quite simple to setup.
  • Many of the advantages of local storage are still there, as pouchdb holds a local copy of the database backed by IndexedDB (or an alternative if it is not available)
  • The database also save the history with a settable maximum for the kept revisions.

This comment is also highly related to #2

  • 👍2
jamie-lord commented

To allow seamless syncing, perhaps https://github.com/automerge/automerge is an option?

rosano commented

https://github.com/0dataapp/0datawrap is a single API for remoteStorage (mentioned in #2) and Fission, both of which use IndexedDB and provide cloud sync.

apankrat commented

Update 20210410 includes a new Storage facility, used to read/write stuff into permanent storage. It has an interface-ish design with 5 methods that require concrete implementation (Ctrl-F for "implement-me"). There's a single implementation for now - Storage_Local, which uses LocalStorage API.

The rework is still incomplete, because the main code basically assumes that the API never fails. However the main goal of this rework was to allow for backing up boards to the secondary storage, and this is now doable and it will be in the next update.

  • 👍2
good-unit commented

Second, see if to adopt git repos for local storage. This will take care of undo/redo functionality and it should also simplify merges and conflict resolution.

Yes, I think git backends would be fantastic (over local AND HTTP protocols)... git over "smartHTTP" would allow secure team collaboration using the same kanban! (While still keeping the minimalist philosophy). Has this been considered?

Of course these backends could be optional and the localStorage could still be the default option for single users.

I love the minimalist approach you've taken... it gives great power/flexibility (e.g choosing fields for each item (dates, assigned-to, dependency-id etc)). Being stored in json -- these can be powerfully manipulated/filtered/gantt charts auto generated etc...
I was thinking adding colors to each item might be useful -- but again I saw the wisdom in minimalism (ie simply reordering the items in the lists or creating categories).

(Maybe the dark theme could use a bit lighter greys for the items to make them more distinct from the black background though.)

rpavlik commented

I think that would probably best be done in the backup/sync application.

good-unit commented

I think that would probably best be done in the backup/sync application.

I thought that was also just a temporary solution...

  • It's windows only
  • Doesn't allow for team collaboration (e.g merging concurrent changes by different people etc)

Advantages of git based backend:

  • Web-server serving git over HTTP can be run in a docker container on any platform / anywhere
    • No code to maintain (modular/minimalist approach)
    • HTTPS authentication built in (security done!)
  • Many git hosting services already available (minimalist/reuse approach)
  • Git allows team collaboration (merge conflicting/concurrent changes)

Right now NullBoard seems very tied to single user mode -- that's fine.
But I'm sure many people would like to use it for team work (common boards between multi-users).
I don't think this "breaks" the minimalist approach?! What do you guys think?

A nice (optional) team work feature (in my opinion) would be that while the NullBoard tab of the browser is
active: NullBoard polls the git backend for updates from other team members -- and if there are any it
updates those changes without the user needing to manually import.