Improve result of 'reset' sync when data supplied is from an older version peer
Closed this issue · 4 comments
Scenario: a new version is released, a truant client upgrades and connects. It is told to reset to server state, but the server state is from an older client with a lower version.
This truant client receives a bunch of invalid data. It prunes it, leaving not a lot left. This looks like lost data.
Desired outcome: truant client receives data, resets to the appropriate version of the db, then migrates it up and syncs it back.
Open question: how will the new replica know for sure what version the data it receives comes from? Do we need at least one touch op per migration to ensure there is an op from the latest version? Or is it logically consistent that if a migration doesn't create any operations, there is no invalid data?
Clients now receive all available historical schemas in context, which can be used to reset the database and remigrate up to latest.
I'm going to say for the moment that any client resetting to some data should go ahead and reset to the latest version number encoded in an operation/baseline timestamp and migrate up.
Dropping the queryable db isn't necessary. The migration can all happen in-memory and it will land on the latest version anyway, at which point data can be safely inserted/replaced in the queryable db.
Receive data -> materialize and migrate documents to newest version -> write documents to db, which is already at newest version since the store was intialized.
This relates highly to backups, too. Same principle.
Main difficulty in implementation here is system organization. Today, the reset sync message is handled at the EntityStore layer, which is too late to do migration.
The client also just isn't really designed to migrate after initialization.
If actually migrating IDB versions isn't necessary (see above), this is still hopefully possible without too much refactoring. But that comment disregarded migration queries, which need the queryable db available at the current migration version.
I think the actual procedure will be...
- Receive old reset sync
- Open a temporary queryable db at reset version
- Load in reset sync to meta db
- Run migration up to current version
- Copy data from temp queryable to real queryable
Temp is only necessary to avoid hot-swapping the live queryable db with a new db at an older version. If that can be done, it isn't needed.
Hot swapping live queryable db was the roadblock last time I tried this as I recall.