/backup-container

Create a snapshot of a BTRFS volume from a Docker container and robustly back it up to multiple BorgBackup repositories on a schedule.

Primary LanguageShellMIT LicenseMIT

My backup container Docker

Create a snapshot of a BTRFS volume from a Docker container and robustly back it up to multiple BorgBackup repositories on a schedule.

Quick start

  1. Review and modify the docker-compose.yml file to set your environment variables as needed
  2. Customise the exclude.conf according to your requirements
  3. Execute the command docker compose --env-file ./config/default.env up to spin up the container

Background

Over the past year, this backup setup has enabled me to successfully restore all my data after incidents. This is all thanks to the excellent backup tool called BorgBackup. The scripts in this repository serve as an opinionated thin wrapper around it. The Docker image provided can be used directly in various self-hosting setups. It's designed to be a simple and effective tool for those looking to establish a reliable backup system, saving time and avoiding common pitfalls.

Features

  • Snapshotting: Takes a snapshot of a BTRFS volume to ensure file consistency during backups.

    I self-host multiple databases and this is the most feasible way of avoiding data corruption.

  • Scheduled Backups: Automates backups according to a defined schedule.
  • Log Rotation: Maintains weekly logs of all backup activities.
  • Multi-Repository Backups: Allows backups to multiple BorgBackup repositories simultaneously.

Multi-target backups

To adhere to the 3-2-1 backup rule without disk-level redundancy, you can configure backups to multiple destinations. For example, backups can be sent simultaneously to rsync.net and a local HDD.

The docker-compose.yml file demonstrates how to set up multiple backup targets using environment variables such as BORG_REPO_0, BORG_REPO_1, BORG_PASSPHRASE_0, BORG_PASSPHRASE_1, and so forth. The backup script sequentially handles each repository defined by the environment variables, ensuring your source volume is backed up across all specified targets.

The backup script first takes BORG_REPO_0 and the corresponding env vars and sets up the BORG_REPO, BORG_REMOTE_PATH, and BORG_PASSPHRASE environment variables for borg. Once the backup finished (successfully or otherwise), the script checks whether BORG_REPO_1 exists, if so, it sets BORG_REPO and the other env vars to their expected values and backs up again. The script keeps going to BORG_REPO_2, BORG_REPO_3 and so on as long as these are set. Otherwise, it unsets the previous BORG_REPO and corresponding env vars and goes to sleep.

Thus, the following sets of environment variables are valid for multi-target backups:

  •   - BORG_PASSPHRASE=$PASSWORD
  - BORG_REPO=/local-backup

    This just backs up to a local repository

  •   - BORG_PASSPHRASE_0=$PASSWORD
  - BORG_REPO_0=/local-backup

    This just backs up to a local repository

  •   - BORG_PASSPHRASE_0=$PASSWORD
  - BORG_REPO_0=/local-backup

  - BORG_PASSPHRASE_1=$PASSWORD2
  - BORG_REPO_1=/local-backup2

    This backs up to two different local repositories

  •   - BORG_PASSPHRASE_0=$PASSWORD
  - BORG_REMOTE_PATH_0=borg1
  - BORG_REPO_0=my-username@my-username.rsync.net:~/backup

  - BORG_PASSPHRASE_1=$PASSWORD
  - BORG_REPO_1=/local-backup

    This first back up to a remote repository, then to a local one

Repository layout

  • src
    • backup.sh: Creates a new BorgBackup repository if none exists, takes a snapshot of the BTRFS volume, performs the backup, and prunes old backups.
    • backup-wrapper.sh: Executes backup.sh for each repository configured via environment variables.
    • schedule.sh: Manages and logs the operation of backup-wrapper.sh and runs it in a continuous loop.
  • config
    • exclude.conf: Exclude list for borg. Files matching these patterns won't be backed up.
    • ssh_config: SSH config for improving remote backup robustness.

Related resources