soba: backup hosted git repositories

about
configuration
run using command line
run using docker
run on Synology NAS
restore

about

soba is tool for backing up private and public git repositories hosted on the most popular hosting providers. It generates a git bundle that stores a backup of each repository as a single file.

An unchanged git repository will create an identical bundle file so bundles will only be stored if a change has been made and will not produce duplicates.

latest updates

1.1.3 released 2022-10-12
Fixes issues that resulted in only a subset of GitLab Projects being backed up.
All Projects across GitLab will now be returned, based on the user's minimum access level. The default level is 'Reporter' and can be overriden by setting environment variable: GITLAB_PROJECT_MIN_ACCESS_LEVEL to the numeric value associated with the level shown here.
Thanks to @drummingdemon for all their help in testing this release.

1.1.2 released 2022-06-03
Add feature to enable backup of project repos in GitLab groups

1.1.1 released 2022-03-13
Add feature to enable backup of GitHub organisations' repositories

1.1.0 released 2021-10-27
Resolve exit on backup failure issue

Supported OSes

Tested on Windows 10, MacOS, and Linux (amd64).
Not tested, but should also work on builds for: Linux (386, arm386 and arm64), FreeBSD, NetBSD, and OpenBSD.

supported providers

BitBucket
GitHub (including organisations)
GitLab

configuration

soba can be run from the command line or as docker container. In both cases the only configuration required is an environment variable with the directory in which to create backups, and additional to define credentials for each the providers.

On Windows 10:

search for 'environment variables' and choose 'Edit environment variables for your account'
choose 'New...' under the top pane and enter the name/key and value for each of the settings

On Linux and MacOS you would set these using:

export GIT_BACKUP_DIR="/repo-backups/"

To set provider tokens see below.

run using command line

Download the latest release here and then install:

install <soba binary> /usr/local/bin/soba

After setting GIT_BACKUP_DIR, set your provider token(s) as detailed here.

and then run:

soba

run using docker

Using docker enables you to run soba without anything else installed.

Docker requires you pass environment variables to the container using the '-e' option and that you mount your preferred backup directory. For example:

docker run --rm -t \
             -v <your backup dir>:/backup \
             -e GIT_BACKUP_DIR='/backup' \
             -e GITHUB_TOKEN='MYGITHUBTOKEN' \
             -e GITLAB_TOKEN='MYGITLABTOKEN' \
             ghcr.io/jonhadfield/soba

To hide credentials, you can instead use exported environment variables and specify using this syntax:

docker run --rm -t \
             -v <your backup dir>:/backup \
             -e GIT_BACKUP_DIR='/backup' \
             -e GITHUB_TOKEN=$GITHUB_TOKEN \
             -e GITLAB_TOKEN=$GITLAB_TOKEN \
             ghcr.io/jonhadfield/soba

scheduling backups

Backups can be scheduled to run by setting an additional environment variable: GIT_BACKUP_INTERVAL. The value is the number of hours between backups. For example, this will run the backup daily:

export GIT_BACKUP_INTERVAL=24

if using docker then add:

-e GIT_BACKUP_INTERVAL=24

Note: the interval is added to the start of the last backup and not the time it finished. Therefore, ensure the interval is greater than the duration of a backup.

rotating backups

A new bundle is created every time a change is detected in the repository. To keep only the x most recent, use the following provider specific environment variables:

GITHUB_BACKUPS=x
GITLAB_BACKUPS=x
BITBUCKET_BACKUPS=x

setting provider tokens

On Linux and MacOS you can set environment variables manually before each time you run soba:

export NAME='VALUE'

or by defining in a startup file for your shell so they are automatically set and available when you need them. For example, if using the bash shell and running soba as your user, add the relevant export statements to the following file:

/home/<your-user-id>/.bashrc

and run:

source /home/<your-user-id>/.bashrc

Provider	Environment Variable(s)	Generating token
BitBucket	BITBUCKET_USER	instructions
	BITBUCKET_KEY
	BITBUCKET_SECRET
GitHub	GITHUB_TOKEN	instructions
GitLab	GITLAB_TOKEN	instructions
	GITLAB_PROJECT_MIN_ACCESS_LEVEL	instructions

additional options

GitHub - Returning Organisations' repositories

Repositories in GitHub organisations are not backed up by default. To back these up, specify a comma separated list of organisations in the environment variable: GITHUB_ORGS.

GitLab - filtering Projects by access level

Release 1.1.3 modified the way in which a user's GitLab Projects are returned. By default, every Project a user has at least Reporter access to will be returned. New environment variable GITLAB_PROJECT_MIN_ACCESS_LEVEL can be set to override this, by specifying the number matching the desired access level shown here and here:

Access Level	Value
Guest	10
Reporter	20
Developer	30
Maintainer	40
Owner	50

run on Synology NAS

The following was tested on DS916+

Create a directory on your NAS for backing up Git repositories to
Install Docker from the Synology Package Center
Open Docker and select 'Image'
Select 'Add' from the top menu and choose 'Add From URL'
In 'Repository URL' enter 'jonhadfield/soba', leave other options as default and click 'Add'
When it asks to 'Choose Tag' accept the default 'latest' by pressing 'Select'
Select image 'jonhadfield/soba:latest' from the list and click 'Launch' from the top menu
Set 'Container Name' to 'soba' and select 'Advanced Settings'
Check 'Enable auto-restart'
Under 'Volume' select 'Add folder' and choose the directory created in step 1. Set the 'Mount Path' to '/backup'
Under 'Network' check 'Use the same network as Docker Host'
Under 'Environment' click '+' to add the common configuration:
- variable GIT_BACKUP_DIR Value /backup
- variable GIT_BACKUP_INTERVAL Value (hours between backups)
Also under 'Environment' click '+' to add the relevant provider specific configuration:
- variable BITBUCKET_USER Value
- variable BITBUCKET_KEY Value
- variable BITBUCKET_SECRET Value
- variable BITBUCKET_BACKUPS Value (Number of backups to keep for each repo)
- variable GITHUB_TOKEN Value
- variable GITHUB_ORGS Value (Optional - comma separated list of organisations)
- variable GITHUB_BACKUPS Value (Number of backups to keep for each repo)
- variable GITLAB_TOKEN Value
- variable GITLAB_BACKUPS Value (Number of backups to keep for each repo)
Click 'Apply'
Leave settings as default and select 'Next'
Check 'Run this container after the wizard is finished' and click 'Apply'

The container should launch in a few seconds. You can view progress by choosing 'Container' in the left-hand menu, select 'soba', choose 'details' and then click on 'Log'

restore

A Git bundle is an archive of a Git repository. The simplest way to restore is to clone it like a remote repository.

git clone soba.20180708153107.bundle soba.

StanBarrows/synology-github-backups