unixvoid/binder

Cleanup old data

Opened this issue · 1 comments

We should add a feature to clean up old jobs after a set amount of time if chosen.
Logistically we should default to a time of 0 aka no cleanup at all. In the config we should allow the user choice to clean up old files. This means storing some sort of metadata about when the file was uploaded or checking the existing file descriptor. The file descriptor method would be the best case and no additional overhead is needed thus no need to convert existing clients data.

The garbage collection idea is solid. The only takeaway from this is the storage of things we do not want garbage collected.
Lets look at an example of unixvoid's public gpg key. This is something we do not ever want garbage collected as it needs to be public for people to safely access the public rkt images. We also don't want a projects all to be removed. What if a project is stale? If its the only bin left we do not want it removed.
There a couple solutions to this:

  • Project based approach. Every project would have their own subdirectory set up and would all have their own retention policies. The user would be able to set whether or not the project gets garbage collected at all or how many binaries to save.
  • File based approach. Each file gets its own retention date. When a user uploads the file they select the retention period. We could also set a flag to not remove the file if it is the last in its directory.
  • Only save releases, this means the user will have to manually perform any garbage collection.