benbjohnson/litestream

Restore operation leaves behind tmp shm/wal files

gwk opened this issue · 2 comments

gwk commented

I am a new livestream user. I just ran a restore operation for the first time using the -o flag to output to a location of my choice. It succeeded and the last line of the log was … level=INFO msg="renaming database from temporary location" db=…XXX.db replica=YYY. When I look in my output directory I see …XXX.db.tmp-shm and …XXX.db.tmp-wal. This suggests to me that litestream is not properly closing the database connection prior to moving the file from the .tmp to final location.

I typically use SQLite via Python as well as the sqlite3 interactive interface. My experience has been that the shm and wal files are only present when a program has the database open, or when such a program has failed to close down properly. Furthermore, my (admittedly vague) understanding is that if those files are present then it is not obviously safe to move the db file from the tmp to final location.

I recognize that this is probably fairly benign clutter, but it would be nice if litestream could avoid leaving orphaned shm/wal files around. Thanks in advance.

hifi commented

It's closing it correctly but because Litestream sets SQLITE_FCNTL_PERSIST_WAL the empty WAL and shm file aren't removed after restore.

We should just not set it during restore or unset it after restoring so that closing will in fact remove those files.

hifi commented

#529 should fix this.