[RFC] attachment_s3: improvements to avoid s3 backup need

[jjscarafia]

We would like to make some improvements to the module. Any though?
This improvement could work together with this other idea #234

Objetive:

Avoid needing to backup attachments and only need to deal with odoo postgres backup.

How it would work:

The idea is that in production databases, when an attachment is deleted, it would not be actually deleted but It will mark to be deleted later (let's said 6 months later).

Modifications needed:

• new optional parameter: attachment_s3.retention
• on ir.attachment new boolean field to_delete
• when calling ir.attachment.unlink, if s3 file and s3 bucket = actual bucket (actual checks) and attachment_s3.retention = True, then:
  a) a new ir.attachment with same values (linked to same s3 objetc) and with to_delete = True
  b) the attachments that was being deleted is deleted but the actual s3 object is not deleted
• new cron parametrizable that will unlink ir.attachments with to_delete = True older than some period

Use cases

Use case 1: check backup for deleted files

• attachment_s3.retention = True
• Production database creates attachments
• postgres backups are created
• someone delete attachments (they're actually marked to be deleted)
• We can restore a postgres backup and access attachments normally (because they still exists on the bucket), user can do whatever he wants to be restored on production databases

Use case 2: restore as production database a previous backup

• attachment_s3.retention = True
• Production database creates attachments
• postgres backups are created
• someone delete attachments (they're actually marked to be deleted)
• We can restore a postgres backup and use it as production database, deleted attachments will be accesible and are not going to be deleted because on this restored backup they're not marked to be deleted

Use case 4: a production database is deleted (for eg. customer leave us) (actually this already work without needing the proposed change)

• attachment_s3.retention = True
• we make a final backup (only postgres)
• we can schedule to delete s3 bucket in a while
• if needed, we can restore postgres backup and all attachments should be there

Use case 5: a training database, in this case if attachments are deleted we wont to delete them

• attachment_s3.retention = True or not configured
• user creates s3 attachments
• user delete s3 attachments
• the s3 attachments objects are actually deleted (actual behavior without this proposed improvement)

[jjscarafia]

@guewen @yvaucher any thoughs about this idea and this one
I like some feedback before making the PR

Thanks!

[jjscarafia]

@max3903 I appreciate your feedback here too!

[max3903]

@jjscarafia It sounds interesting but Digital Ocean Spaces does not support retention. When the attachment is deleted, it is removed immediately.

[jjscarafia]

Thanks @max3903
The retention would be done by odoo, not the s3. We retain a copy of ir.attachment record and delete it with a cron after some period of time

[jjscarafia]

May be the parameter could be call different to avoid confusion with "s3 retentions", the proposal here is more like a "attachment_s3.mark_to_delete"

[max3903]

@jjscarafia I like the idea but we need to take into account S3 backends that support retention and those that do not (and would need the cron job in Odoo). We could use the domain name in AWS_HOST.