Versioning
Opened this issue · 7 comments
Although this feature doesn't currently exist with fakes3, is it possible to implement bucket versioning?
Thanks
You're referencing this: https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html ? I didn't know this feature existed. Awesome!
I agree, it is pretty awesome and it would be quite nice to be able to use it!
I've been thinking about this. I like the idea, but I think it would require a fair bit of work to implement. S3rver does not use a database, it only uses a metadata file to receive information about an object (as you're probably aware). If it were using a database then versioning would be rather simple to implement. Adding a database to s3rver would just add another overhead to to think about for a developer - which I really don't think is necessary
Of course there are other ways of do it....we could store the versions in the metadata file. Each version points to a different file.
Or perhaps if each bucket had a git repo we could use https://github.com/creationix/js-git to manage the object versioning.
To conclude, I like the idea, but I don't think it is necessary for a development server. I don't want to add too much bloat to it.
Let me know your thoughts and ideas! Cheers.
I think your decision is understandable. Although, even for a development server I believe it should still support every feature. The point, as I understand it, is to allow developers to avoid using the online cloud for internal testing and to be able to quickly switch back for production. If it happens that S3's versioning system is an important part of their program, they wouldn't be able to use S3rver in development.
Fortunately, this is not a priority for me right now as I am using my own system for versioning and not S3's system. But perhaps for a future endeavor, I still think it would be good idea.
As for using git to version each file, I think that is way overkill and not really necessary. If anything, I would say keep the original .dummys3_content and .dummys3_metadata files as the current version of a file. Only when a user requests a specific version like 001, it would instead find and pull .dummys3_content_001 and .dummys3_metadata_001 file data. Each time the user uploads a file that already exists in the cloud (assuming versioning mode is enabled for the bucket), the current .dummys3_content/metadata files would be renamed to an incremented version, and the new file would take .dummys3_content/metadata.
would using sqlite do the tricks ? as you would not have yet an other service to configure while still being able to have the full power of sql ?
Each time the user uploads a file that already exists in the cloud (assuming versioning mode is enabled for the bucket), the current .dummys3_content/metadata files would be renamed to an incremented version, and the new file would take .dummys3_content/metadata.
I agree with the general algorithm. However, I would suggest saving the file from the beginning from the path corresponding to the appropriate version. After saving the data and metadata, create or overwrite (UPSERT) a symlink indicating the current version. This slightly reduces the risk that in the event of a failure, the currently saved data will be damaged.
It's worth noting that - looking at the examples in the AWS documentation - that the version identifier is a random string of characters that is opaque for the client nor sequentially increasing counter. Using uuidv4 seems natural to me here as source of random value.
Since this issue is still open, is this something that PRs would be welcome for? @jamhall seemed to push back on if this feature should be implemented, but it's still open which indicates that PRs would be welcome for it.