flokno/son

Writes are not strictly atomic if device runs out of space

Opened this issue · 0 comments

If the hard drive runs out of space during the writing of a son entry, the file will be corrupted, at least for large entries. (Corrupted in this case means that the entry will simply terminate in the middle of the JSON, without the closing ---.) The expected behaviour is that the write fails entirely, leaving the previously written entry as the last entry.

I think this is because appends to files on linux are not atomic (in the sense of: "it either succeeds and writes the whole entry, or fails without writing anything") if they are above a certain size limit: https://www.notthewizard.com/2014/06/17/are-files-appends-really-atomic/. I'm not sure there is anything we can do about this, but we should probably at least document the problem somehow.