Web clipper overwrites existing database entries

[m0lentum]

Describe the bug
Occasionally when adding an image with the web clipper add-on, it replaces an existing entry instead of creating a new one. The added image receives the tags and time of import of the old entry, which disappears from the gallery until Allusion is restarted, at which point it reappears as a newly imported entry with no tags. Any subsequent images added with the web clipper will all replace this same entry, and all tags added via the web clipper will appear on this entry. This behavior persists even across restarts.

I've also noticed that the next time a new image is added to the directory by saving it there without using the web clipper, it also replaces the same entry, but the next image after that will create a new entry. Edit: After some debugging it seems that files added with the clipper are interpreted as being moved/renamed when a new file is added after them. Continuing to investigate.

To Reproduce
Steps to reproduce the behavior:

1. Choose a download directory and enable the web clipper in settings
2. Use the web clipper context menu to add some images from a browser
3. Eventually an image will replace an existing entry

Strangely, this only sometimes happens on my main Allusion collection, but on a development build with a newly created database it seems to happen every single time I use the web clipper. This also happens if I add an image by manually sending an http request to the clip server endpoint, so it's definitely an issue with the host process, not the browser extension.

Expected behavior
All added images receive new database entries and old ones are unchanged.

Allusion version

• v1.0.0-rc.10

Desktop OS

• NixOS 23.11

[m0lentum]

After a bunch of debugging I believe I've found the root cause in the way inode numbers are used to detect moved/renamed files:

• when the file added via the clipper is initially downloaded, it gets saved to disk, caught by the file watcher, and its inode number stored in the database
• after this, exiftool is used to add the page url to the image metadata. This causes the image to be rewritten to another inode, but the database still has the inode number it was originally downloaded into
• the inode the first image was downloaded to is now vacant. When the clipper is used again, the same inode is used to store the new image, and again caught by the file watcher and stored into the database. This causes the new image to be interpreted as a moved version of the previous one, even though in reality both files are now at new inodes

Relatedly, any operation modifying file metadata with exiftool causes the file's inode number to change, breaking subsequent renames. If you change an image's metadata in Allusion and then rename the file, a new entry is created and the old one is still visible as a missing image.

[m0lentum]

Turns out exiftool has a parameter that makes it write in place, which prevents files being moved. I've made a PR applying it in #624. The same issue can still be caused by replacing files manually, so it may be worth considering alternative solutions that don't involve inode numbers at all, but with this modification Allusion no longer breaks itself (in cases I've tested anyway).