rsync

rsync is an open source utility that provides fast incremental file transfer.

If you are into Dockerizing everything or you just want to have a better view over the rsync process (e.g., to see it in your cluster visualizer), this repository provides you with a Docker image to run the rsync process.

The provided image is open-source and built from scratch with the goal to enable you to run a stateless and an immutable container, according to the best practices.

With this image you can:

run a simple one-time rsync job for local drives
run a rsync job for remote drives using ssh (included in the image)
run scheduled rsync jobs using cron (included in the image)

Supported architectures:

the image supports multiple architectures: x86-64 and arm32
the docker manifest is used for multi-platform awareness
by simply pulling or running ogivuk/rsync:latest, the correct image for your architecture will be retreived

Tag	Transmission Version and Architecture
`:latest`	supports both `x64` and `arm32v7` architectures
`:x64`	targeted to the `x64` architecture
`:arm32v7`	targeted to the `arm32v7` architecture

The image is based on alpine image and it includes:

rsync
openssh-client for remote sync over ssh
tzdata for easier setting up of local timezone (for file timestamps)
cron (included in the alpine) for scheduling regular back-ups
rsync.sh script that prepares the cron job and starts the cron daemon

Usage

Quick Start: one time run and sync of local folder

docker run --rm \
    --name=rsync \
    --volume /path/to/source/data:/data/src \
    --volume /path/to/destination/data:/data/dst \
    ogivuk/rsync [OPTIONS] /data/src/ /data/dst/

Replace:

/path/to/source/data with the source folder to be copied or backed-up
/path/to/destination/data with the destination folder
[OPTIONS] with desired rsync optional arguments

Use with cron or ssh

Step 1. Prepare the setup (First time only)

create a folder, for example ~/rsync, that will be later mounted in the container
- this folder is intended to hold supporting files such as log files, crontab file, and any supporting rsync files (e.g., list of files to rsync)
- the subfolder logs, for example ~/rsync/logs is suggested for the log files
- important note: for Docker Swarm, this directory needs to be available on all nodes in Docker swarm, e.g., via network shared storage
```
mkdir -p ~/rsync/logs
```
- you can replace ~/rsync with any other desired location

Step 2. Run

run as a container:

docker run --rm \
    --name=rsync \
    --env TZ="Europe/Zurich" \
    --env RSYNC_CRONTAB="crontab" \
    --volume ~/rsync:/rsync
    --volume /path/to/source/data:/data/src \
    --volume /path/to/destination/data:/data/dst \
    ogivuk/rsync

run as a swarm service:

docker service create \
    --name=rsync \
    --env TZ="Europe/Zurich" \
    --env RSYNC_CRONTAB="crontab" \
    --mount type=bind,src=~/rsync,dst=/rsync \
    --mount type=bind,src=/path/to/source/data,dst=/data/src \
    --mount type=bind,src=/path/to/destination/data,dst=/data/dst \
    ogivuk/rsync

Parameter	Explanation	When to Use
`--env TZ="Europe/Zurich"`	Sets the timezone in the container, which is important for the correct timestamping of logs. Replace `Europe/Zurich` with your own timezone from the list of available timezones.	Always
`--env RSYNC_CRONTAB="crontab"`	Specifies that the rsync is to be run as one or multiple cron jobs, and that the jobs are defined in the `crontab` file located in the mount-binded `~/rsync` folder. The rsync parameters used in the crontab must be mindful of the data locations in the container.	When using cron for regular rsync jobs
`~/rsync`	Specifies the local folder `~/rsync` that is mounted to the container at `/rsync`. Change `~/rsync` if another location is chosen in Step 1.	When using cron or ssh
`/path/to/source/data`	Specifies the source folder for sync and is mounted to the container at `/data/src`. Change to the appropriate folder. Multiple folders can be mounted in this way.	If any source is local
`/path/to/destination/data`	Specifies the destination folder for sync and is mounted to the container at `/data/src`. Change to the appropriate folder. Multiple folders can be mounted in this way.	If any destination is local
`--env RSYNC_UID=$(id -u)`	Provides the UID of the user starting the container so that the ownership of the files that rsync copies belong to that user.	If the rsync option for preserving ownership is not selected
`--env RSYNC_GID=$(id -g)`	Provides the GID of the user starting the container so that the ownership of the files that rsync copies belong to that user group.	If the rsync option for preserving ownership is not selected

Remarks:

rsync will not be run by default, you need to be specify the rsync command with all its arguments in the crontab, or in a script called in the crontab
any later changes to the crontab file require the service to be restarted, and that's why consider to define the rsync job in a script that is called in the crontab
when defining the rsync arguments, including source and destination, do that from the perspective of the container (/data/src, /data/dst)
more volumes can be mount binded if needed
the ssh client is included in the image in case your source or destination is a remote host
- ssh required files (private key, known_hosts, ssh_config) needs to be stored in a folder mounted to the container, for example in ~/rsync/ssh/
- you can define the ssh connection in a ssh_config file
- rsync option -e "ssh -F /rsync/ssh/ssh_config" instructs rsync to use the ssh with the ssh_config for the remote sync