superfly/litefs

Manual editing of data and forcing sync

Opened this issue · 1 comments

So I am using local development and the litefs-example as a base to build out my schema and test some stuff.

One of the things I did was use docker compose to mount the volumes locally, so I view them in my IDE so I can use the SQL tools with the file directly, since it's not exposed like a traditional server.

One of my observations, is that I can not manually add data for testing purposes without causing a checksum issue with LTX as manual changes are not identified by the LTX file and checksum does not match.

I discovered by deleting the LTX file of the primary, and then triggering a insert (generate button on the example repo), a new LTX file is generated for both the primary and replica which basically resets the replica to match the primary.

This feels like a command that should be possible, where I can restart the docker images and pass in a argument to run a lifefs command to wipe the LTX file, and reset the replica to match the primary db without inserting data.

I know i can export and import. And I played with the idea of doing a export, and then reimporting and having a shared mount to transition the file back and forth. But this was problematic with version 0.5.0. Version 0.5.4 seems to handle these issues better, and will automatically recover on the above.

litefs -force
or
litefs -reset
or something like git to specify the source
litefs -rebase primary

Just a thought, but loving this. I know this is a odd use case, but I am trying to avoid having to write a script or insert data from a app, or in the SQL shell within the image to try things.

If there is a alternate way for me to solve this problem, that is cleaner, let me know.

Hmm, I'm not sure I understand the use case completely. Can you run rm -rf $REPLICA_DATA_DIR/dbs to clear out the replica's files? It should resync with the primary automatically then.

If you're trying to insert data from outside the Docker image, we are planning to add an HTTP Query API (#326) that would let you curl a SQL command in (so long as you expose the LiteFS port, 20202).