The system-configs tool is a series of scripts for managing and tracking of system configuration files. This allows reproducibility of systems, replication of similar systems via the template funcationality, and the traceability of system state configurations. Example files that are tracked may be:
- Network files relating to static IP's
- Network settings pertaining to ROS
- Executable scripts that are essential on said system
If you are adding an additional device to an existing system ("main" PC and "RTOS" system pre-existing, with a Raspberry Pi 4 to be added for example), there is an order of operations required to setup the RPi4 local repository to allow for syncing tracked files for the RPi4 given that there will be files already present on the remote repository. Tracking locally then syncing without first "installing" the existing configuration will result in a Git error: "Updates were rejected because the remote contains work that you do not have locally...".
See "Syncing Files" header for an explanation of the naming convention required to maintain the intended file structure on the remote repository. The order of operations should be as follows:
Note: If the host device does not exist remotely, this will change nothing within its file-system other than the creation of a local repository at /opt/<user>/system-configs-<system name>.
#Install existing config for the entire system.
qcr system-configs install --owner <github_owner>The above will cause errors if the directory already exists, delete it if prior attempts have been made to track files locally. Correct execution should finish with two lines:
"Your branch is up to date with 'origin/main'
"[Error] The directory '/opt/<user>/system-configs-<system name>/<host device>/default' does not exist.This is expected as we are about to sync files to this location from the host device. You can now follow the following instructions for tracking local files and syncing to the remote repository.
- Run the following command to track a file. Currently only individual files are supported (wildcards are not supported but are being looked into):
# Enter the directory containing the file(s) to track
cd <required directory>
# Track the file (replace <FILES>)
qcr system-configs track <FILES>Once tracked, these files can be synced up to a dedicated remote repository within a specified user or Github organisation. These repositories will have the following naming structure: <user-or-organisation>/system-configs-<system-name>; where system-name is the hostname of the machine up until the first hyphen. A machine's hostname after the first hypen, is considered to be the component-name and identifies a component within the system. If the machine's hostname doesn't have a hyphen, the component name defaults to main.
This separation of the hostname after the first hyphen, allows for components within a system to be tracked within the same repository. For example, if a system was comprised of a main GPU machine with the hostname athena, and an Intel NUC running an RTOS with the hostname athena-rtos. The tracked files for athena will be stored at <user-or-organisation>/system-configs-athena/main/default (as the hostname doesn't contain a hyphen). The tracked files for athena-rtos will be stored at <user-or-organisation>/system-configs-athena/rtos/default (rtos as it is the portion contained after the first hyphen).
To sync up files, run the following:
# Run the sync command supplying your GitHub User Email, the GitHub User or Organisation that you wish to own the remote repository, and a GitHub Public Access Token (PAT) that has the ability to create repositories within the supplied GitHub Owner.
qcr system-configs sync <github_user_email> --owner <github_owner> --pat <github_pat>
# You can also set user defaults so you don't need to supply the github owner and github_pat every time. Using the config command.
# Run the command again if the defaults need to be updated.
qcr system-configs config --owner <github_owner> --pat <github_pat>To install a system configuration stored on a remote repository, use the install command.
# This will install the repository stored at https://github.com/<github_owner>/<NAME>, where NAME is the hostname of the machine up until the first hyphen.
qcr system-configs install --owner <github_owner> This command will copy the files in the component machine's directory to the system. Remember, the component is based on the machine's hostname. Additionally, as part of the installation process, the tool will attempt to run any executable scripts contained within an install directory within the components folder. For example, building on the athena and athena-rtos from the earlier sync example:
If the above command was run on a machine called athena. It would
- Clone the remote stored at https://github.com/<github_owner>/athena;
- Install the files stored in https://github.com/<github_owner>/athena/main/default to the required system locations; and
- Run any executable scripts stored in https://github.com/<github_owner>/athena/main/default/install.
If the above command was run on a machine called athena-rtos. It would
- Clone the remote stored at https://github.com/<github_owner>/athena;
- Install the files stored in https://github.com/<github_owner>/athena/rtos/default to the required system locations; and
- Run any executable scripts stored in https://github.com/<github_owner>/athena/rtos/default/install.
You may have noticed in the example above the inclusion of a default folder. For example, in:
- https://github.com/<github_owner>/athena/main/default; and
- https://github.com/<github_owner>/athena/rtos/default
The tool also allows what is known as variants. A variant occurs when a system has competing requirements within a file. For example, a file may need to specify a different configuration depending if it is been used for Robot A or Robot B. This is where variants come in.
Note: The variant feature is limited in its utility. Further development is required to do things such as easily swapping between variants.
A configuration template can be created allowing for the reproduction of similar system types.
Use the sync command to create a template from the currently tracked system configuration files.
# Use the sync command and supply a template name to create a local and remote repo.
# The name will be prefixed with 'template-' to allow for easy identification of templates within the owner's github account
qcr system-configs sync --template <NAME>Use the above command to update a template as well. It will add any files that exist within the tracked system configuration files and notify the user of any differences between files in the template and system configuration file.
When creating a template, it is important to understand what files might require additional setup for usability. For example, any files containing instance specific information such as IP or MAC addresses.
It is important you keep template configurations as general as possible to avoid additional configuration, or stipulate any updates (such as changing IP or MAC addresses) in the template configuration repository README.
To install a configuration template use the install command with the template argument.
qcr system-configs install --template <NAME>This will clone the template configuration files, and create a local and remote repository for tracking this specific system's configuration. You can rerun this command on a system that is already tracking its own system configuration, and only files within the template that aren't already within the system configuration will be installed.
The order of operations when syncing for a machine:
- Create a remote machine repo, if required
- Initialise the local machine repo, if required
- Pull down the latest from the remote machine repo
- Add, commit, and push to the remote machine repo
The order of operations when syncing for a template:
- Create a remote template repo, if required
- Initialise the local template repoe, if required
- Copy files from the local machine repo to the local template repo
- only files that don't already exist in the template will be copied
- for all files that already exist, it will report if any files differ
- Add, commit, and push to the remote template repo
The order of operations when installing are:
- Check remote and local repository for this machine exist, create if required
- Get the remote and local into the same state
- Clone, or pull, the latest template repo (if installing from a template)
- Copy any files from the local template to the local machine repo
- will only copy files that don't exist in the local machine repo
- any files that already exist (based on filepath) will be checked for differences, and files that differ will be reported
- Commit local machine repository to the remote
- Install files from local repo to the machine
- Run any installation scripts for the machine