Append to file, empty contents of file
benpackard opened this issue · 4 comments
Apologies if this is supported already - I have only reads the reader so far.
Is there a command for appending to a file? Or should we read the file first back into model objects, append objects, and rewrite?
Similarly, can we remove the contents of a file without removing the file itself? Or should we delete it and recreate it with an empty array for example?
Is there a command for appending to a file? Or should we read the file first back into model objects, append objects, and rewrite?
No, that would require Disk to have the ability to modify existing files' data, which could lead to the possibility of corrupting the file and not being able to decode to an expected type. This level of modification to the file system should be within your own control, so yes you are right: you must first retrieve your desired objects and manipulate them however you want to that way, then save them again.
Similarly, can we remove the contents of a file without removing the file itself? Or should we delete it and recreate it with an empty array for example?
You should delete the file instead of 'emptying' it.
Thanks, makes sense. I was thinking of a use case where I might want to store objects to disk over time for processing later. Loading a large number to append one more might have a memory impact. But I get the point - what would stop me appending an apple to an array of oranges.
I think one solution I might consider is saving each object to a folder as a separate file. Each can be deleted once processed, and it is the consumers responsibility for saving the right kind of object (and handling an unexpected deserialzed type). I will have to research the impact of reading hundreds or thousands or small files though.
I was thinking of a use case where I might want to store objects to disk over time for processing later. Loading a large number to append one more might have a memory impact.
I've been on the fence about this too. There is a way to append data to a file instead of reading it, decoding it, encoding it, then writing again. But I wasn't sure if it would have been user friendly. I will try to code something up tonight and see what others have to say about it.
Thank you for your input!
Hey @benpackard, I added the ability to append in Disk thanks to your suggestion (I put your name in the README.) Thank you!
Please play around with it and let me know what you think!