Script to automate backing up ZFS datasets to AWS
The script uses ZFS send on a dataset and the aws cli to transfer the data. The idea is that it'll make periodic full backups, and incremental ones in between.
Metadata is stored with each S3 object that allows the script to decide the course of action, and nothing is known locally about what has already been backed up.
Each dataset has independently configurable options, as can be seen in the supplied example config file
Requirements:
- aws cli installed and configured
- non-standard packages: jq, pv
Recommendations:
- specific aws user
- retention policy to move datasets to glacier
- lifecycle policy to automatically clear incomplete partial uploads
ToDo:
- Delete old backups automatically
- Date based rules for creating new full backups (rather than just absolute number)