hungrybluedev/klonol

klonol

A phonetic play on the phrase "clone all". It is a CLI tool written in V to clone all repositories belonging to you (or the authenticated user).

Features

Supports GitHub and Gitea.
Retrieves information about both public and private repositories belonging to the authenticated user.
You can list all available repositories, clone them, or run git pull on existing clones. It uses the user's SSH key to clone.
Cross-platform! We've switched to using TOML for storing credentials and have tested this project extensively on Windows as well.

Motivation

I self-host my Gitea instance. It contains several private repositories. There are some instances where I need to upgrade the server or perform some maintenance. Although I use Docker Compose with mounted volumes to manage the Gitea instance, there may be times when data retention is not possible. I need to restart my service from scratch. In order to help with these "from-scratch" scenarios, I wrote this tool.

It allows me to automatically retrieve all of my repositories and clone them locally. I can stash away a password-protected local copy while I upgrade my git server in peace.

Prerequisites

You must have Git installed. Use your package manager or navigate to Git's Website to download the latest version.
You must have a GitHub or Gitea account.
Add an SSH key to your appropriate account.
Generate a personal access token with the minimum scope of repo (to allow viewing private repositories) and set an expiration of 7 days or the lowest possible. Regenerate this key when it expires.

Installation

From Source using Git

Step 1: Install V and symlink it.

If you already have V installed, update your installation by doing v up.

Step 2: Clone and compile klonol

# Move into a convenient place to install klonol
cd ~/oss/apps

# Clone this repository and move into it
git clone https://github.com/hungrybluedev/klonol.git --depth 1
cd klonol

# Build the executable
v build.vsh

The optimized klonol binary is saved in the bin subdirectory.

Note: If you are working on klonol locally and want faster builds for quicker prototyping, use v build.vsh -fast to use TCC and build an unoptimised executable quickly.

Step 3: Add to PATH

Add the bin subdirectory to your PATH. You can edit your ~/.bashrc file in Unix-line systems to do this. On Windows, use the system dialog.

Step 4: Test installation

Running the following command from any directory should provide a detailed description on the tool including its name, version, description, and available options.

klonol -h

Usage

Setting the variables

The following variables need to be set in a file called credentials.toml.

Name	Description	Compulsory
`username`	The GitHub or Gitea username whose repositories are to be queried	Yes
`access_token`	The personal access token generated previously	Yes
`base_url`	The base domain to be used to test SSH and make API calls from. Defaults to `github.com`	For Gitea

A sample credentials.toml file will look like this:

[github]
username = "your_username"
access_token = "XYZXYZ"

[gitea]
base_url = "git.yourdomain.com"
username = "your_username"
access_token = "XYZXYZ"

Running klonol

Once the variables have been set, klonol will retrieve the relevant information automatically.

Help Information

# Get the version
klonol --version
# output:
# klonol 0.6.x


# Get detailed usage information
klonol -h
# OR
klonol --help
# output:
# klonol 0.6.x
# -----------------------------------------------
# Usage: klonol [options] [ARGS]
#
# Description: A CLI tool to "clone all" repositories belonging to you.
#
# klonol requires Access Tokens to work properly. Refer to README for more
# information. It retrieves information about ALL available repositories
# belonging to the authenticated user. Both public and private.
#
# Please follow safety precautions when storing access tokens, and read
# the instructions in README carefully.
#
#
# Options:
#   -p, --provider <string>   git provider to use (default is github)
#   -c, --credentials <string>
#                             path to credentials.toml file (default is ./credentials.toml)
#   -a, --action <string>     action to perform [list, clone, pull]
#   -v, --verbose             enable verbose output
#   -h, --help                display this help and exit
#   --version                 output version information and exit

Sample usage flow for GitHub

# Navigate to a directory to store all the repositories in
cd ~/Backups/GitHub

# Make sure you have the proper 'credentials.toml' file in the directory
nano credentials.toml

# List all available repositories
klonol

# Clone all available repositories
klonol --action clone
# OR
klonol -a clone

# ... After some time has passed ...
# Pull all changes from GitHub
klonol -a pull

Sample usage flow for Gitea

# Navigate to a directory to store all the repositories in
cd ~/Backups/Gitea

# Make sure you have the proper 'credentials.toml' file in the directory
nano credentials.toml

# List all available repositories
klonol --provider gitea
# OR
klonol -p gitea

# Clone all available repositories
klonol --action clone --provider gitea
# OR
klonol -a clone -p gitea

# ... After some time has passed ...
# Pull all changes from GitHub
klonol -a pull -p gitea

Updating klonol

If you've installed it from source, navigate to the folder where you cloned the repository. Then run git pull. After all the changes have been downloaded, run v build.vsh.

You don't need to change the PATH variable if klonol is already added to PATH.

License

This project is distributed under the MIT License.

Acknowledgements

Thanks to A1ex-N for contributing the idea and initial code for supporting cross-platform TOML in favour of unix-specific ENV!