mithomas/restic-runner

Configure and run Restic more easily

restic-runner

restic-runner is a helper script for using restic. It makes it easy to configure backup repositories and data sets, and to operate on them.

Contents

• Installation
• Configuration
• Usage
• Tips

Installation

1. Put the restic-runner script in your PATH.
2. Make the directory ~/.config/backup/restic (or a path of your choosing after changing the config_dir default variable in the script).

MacOS

On MacOS, you will probably need to install GNU getopt, as well as GNU date (symlinking it to date as needed). You may want to put them earlier in your PATH when running restic-runner. For example, using Homebrew, you could do this:

$ brew install gnu-getopt coreutils
$ ln -s /usr/local/bin/gdate $HOME/.bin/date
$ export PATH="/usr/local/opt/gnu-getopt/bin:$HOME/.bin:$PATH"

Note: The author does not use MacOS, so he cannot directly vouch for this information. See #5, #7, #8, #9.

Configuration

All configuration files are actually Bash scripts, which are sourced by restic-runner. Therefore, the following configuration instructions are actually just a convention. That means that any of the variables mentioned may actually be specified in either repository or set configuration files, or both (e.g. when using Bash arrays, both files could add to an array by using array_var+=(element1 element2) syntax). Obviously, if a variable is set in both files (as opposed to being appended to), the resulting behavior should be considered undefined (i.e. don’t rely on the order in which the files are sourced; keep it simple).

Global

A global configuration file may be put at ~/.config/backup/restic/runner. You may use it to set any variables that you might also set in the following files. This file will be sourced first, so repo- and set-specific files will override settings in this file.

This is especially useful for settings that are Bash arrays, like exclude_patterns, because you can use += syntax to add to the array in each file (or you could use regular assignment to override rather than add to).

Repositories

To configure a backup repository, put a file at ~/.config/backup/restic/repos/REPONAME. It should export these variables:

• RESTIC_REPOSITORY: The path to the repository root
• RESTIC_PASSWORD_FILE (path to a file containing the plain-text password for the repo) or RESTIC_PASSWORD (the password for the repo in plain-text)

It may also define these variables:

• du: Set to true to report the repo size and difference after running a Restic command. This should only be enabled for local repos (e.g. not SFTP ones).
• keep_policy: A Bash array containing a list of --keep-X options.

Example:

export RESTIC_REPOSITORY=/mnt/backup/restic/REPONAME
export RESTIC_PASSWORD_FILE=$RESTIC_REPOSITORY/password_file

du=true

keep_policy=(
    --keep-last 2
    --keep-daily 7
    --keep-weekly 8
    --keep-monthly 12
    --keep-yearly 10
)

Data sets

To configure a backup data set, put a file at ~/.config/backup/restic/sets/SETNAME. It should define these variables:

• tag: The tag to apply to the data set in Restic.
• include_paths: A Bash array of paths to include in the backup. Unquoted paths will be expanded by Bash, so you can use ~ for your home directory. Remember to quote paths (or parts of paths) containing spaces! Unquoted globs will also be expanded by the shell, which you almost certainly never want, so you should always quote glob patterns.
• exclude_patterns: A Bash array of patterns to exclude from the backup. The same rules about quoting and shell expansion apply: paths can be unquoted (except for ones containing spaces), and patterns should be quoted.
• exclude_if_present: A Bash array of filenames to pass to Restic using the --exclude-if-present option.

Example:

tag=main

include_paths=(
    # This comment is ignored by Bash, so you can comment your configuration freely.
    ~/src
    ~/"Important Files"  # Quote the part containing a space, but leave ~ unquoted so Bash will expand it
)

exclude_if_present+=(
    # Using += instead of = so this can also be set by repo config files.
    .nobackup
    .resticignore
)

exclude_patterns=(
    # Backup files
    "*~"
    "*.bak"
    "/**/.backup"

    # Temp files
    "/#*#"
    "/.#*"

    # Misc
    ~/tmp
)

Usage

Run restic-runner with these options:

• --repo REPONAME: Use the name of the repo file you configured.
• --set SETNAME: Use the name of the data set file you configured.

After each command, a log is displayed giving the duration, current repo size, and change in repo size:

LOG (2018-01-06 00:09:57-06:00): backup finished.  Duration: 7m31s  Repo size: 50.341 GB (+1.576 GB)

Combined with the diff command and the --added filter, this makes it easy to find out why your repo suddenly increased in size.

Commands

• backup
• check
• command COMMAND-STRING
• diff {SNAPSHOT1} {SNAPSHOT2}
• expire
• init
• mount PATH
• snapshot-ids
• verify-randomly {N}

backup

Runs a backup. By default it calls Restic with these options:

• --one-file-system
• --exclude-caches

Example:

restic-runner --repo REPONAME --set SETNAME backup

check

Runs restic check on the repo. The --set option may be omitted, since it is meaningless for this command.

Example:

restic-runner --repo REPONAME check

command COMMAND-STRING

Pass the specified command through to restic. This is useful for commands that do not have an equivalent in restic-runner.

Example:

# Forget specific snapshots
restic-runner --repo REPONAME command forget abcd1234 deadbeef

Note: To avoid further processing of options that are valid for both restic and restic-runner, use --, like:

# Prevent "--tag main" from being processed by restic-runner; instead, pass it to restic
restic-runner --repo REPONAME -- command snapshots --tag main

Which results in running restic snapshots --tag main.

If unsure, you can use restic-runner --debug to see how arguments are parsed.

diff [SNAPSHOT1] [SNAPSHOT2]

Shows the diff between two snapshots. One or more snapshot IDs may optionally be specified. If none are given, the latest snapshot is compared with the one before it. If one is given, it is compared with the latest.

These options may be specified (before the command):

• --added Show only added paths
• --modified Show only modified paths
• --removed Show only removed paths

--added and --modified may be used together.

Example:

# Shows the diff between snapshot abcd1234 and the latest one
restic-runner --repo REPONAME --added --modified diff abcd1234

expire

Automatically forget and prune snapshots according to the configured policy.

Example:

restic-runner --repo REPONAME expire

init

Initialize the configured repo.

Example:

restic-runner --repo REPONAME init

mount PATH

Mount the repo to PATH.

Example:

restic-runner --repo REPONAME mount ~/mnt/restic

snapshot-ids

Print a list of snapshot IDs, one per line.

Example:

# Print all snapshot IDs for the repo
restic-runner --repo REPONAME snapshot-ids

# Print snapshot IDs for the tag configured in this set
restic-runner --repo REPONAME --set SETNAME snapshot-ids

# Print snapshot IDs for this tag
restic-runner --repo REPONAME --tag TAG snapshot-ids

verify-randomly [N]

Verify N (default 10) random files from the latest snapshot. If --compare is specified, the restored files are compared with the live versions.

Note that the --set SETNAME option may be specified to e.g. choose the latest snapshot in SETNAME, or omitted to e.g. choose the latest snapshot in the repo.

These options may be specified:

• --compare Compare restored files with live versions, exiting with an error if any differ.
• --snapshot SNAPSHOT-ID Restore from this snapshot.

Examples:

# Verify 10 random files from the latest snapshot in set SETNAME.
restic-runner --repo REPONAME --set SETNAME verify-randomly

# Verify and compare 100 random files from snapshot DEADBEEF with verbose output.
restic-runner -v --repo REPONAME --snapshot deadbeef --compare verify-randomly 100

Tips

• When running in a cron job, use the chronic utility from moreutils, which only sends output if the job exits with non-zero status. (However, this means you’ll only receive the log if an error occurs, so it won’t be as easy to notice if your repo suddenly grows due to unintentionally backing up some files.)
• Repo and set config files can be placed in subdirectories of their respective directories. For example, the repo config file ~/.config/backup/restic/repos/remote/s3 can be referred to like restic-runner --repo remote/s3.

License

GPLv3