/mirage-ci

Continuous Integration for MirageOS using DataKit's git workflow

Primary LanguageOCamlISC LicenseISC

Mirage-CI — Continuous Integration for MirageOS using DataKit

%%VERSION%%

Mirage-CI is a continuous integration system for MirageOS libraries, using the DataKit Git-based workflow engine.

Mirage-CI is distributed under the ISC license.

Installation

This project actually contains two CIs: mirage-ci for testing Mirage projects, and opam-repo-ci for testing opam-repository.

Start by generating a GitHub API token. If you're setting up a local test deployment, I suggest using a token for a GitHub user who does not have write permission on opam-repository. This is to avoid your test CI overwriting the status messages from the real CI.

opam depext -i github-unix tls
git-jar make -s repo $USER datakit-github-cookie

Replace $USER with your GitHub user name. Note: repo indicates the token's scope, not a repository name.

  • For docker-compose: Move the resulting ~/.github/jar/datakit-github-cookie file to mirage-ci/data/bridge/datakit-github-cookie. Note: this file MUST NOT have any "other" permissions set in its Unix permissions. Otherwise, the bridge will refuse to start, saying that the file doesn't exist.

  • For docker-stack: docker secret create datakit-github-cookie-bactrian - < ~/.github/jar/datakit-github-cookie

For local testing, you might also want to put the bridge into read-only mode, by changing its -c option to -c "*:r" in docker-compose.yml.

To build and run opam-repo-ci locally (for testing) with docker-compose:

env CI_BINARY=opam-repo-ci CI_DOMAIN_NAME=localhost docker-compose up

For production use, use Docker stack: docker stack deploy opam-repo-ci -c opam-repo-ci.yml

You should see a message like this in the logs:

ci_1       | 2019-05-20 09:30.34 APP [datakit-ci] >>> Configure the CI by visiting
ci_1       |                                      https://localhost/auth/intro/47Q5fErg3aYhi0FjDWXDeX5cWMIJ1g5g

Visit the URL shown and set an admin password for yourself.

The CI wants to write the opam2-archive Docker volume as UID 1000, but it gets created owned by root, so change that:

sudo chown 1000 /var/lib/docker/volumes/opam2-archive/_data

Disk space

Docker caches the builds, which uses up a lot of disk space. It is a good idea to prune Docker's cache daily with a crontab entry:

@daily docker system prune -f > /var/log/docker-prune.log 2>&1

Also, the datakit process writes data to a Git repository in a Docker volume. It never deletes anything, and never runs git gc either. Therefore, this will grow steadily (at roughly 6 GB / month). The data is not hugely valuable, so if you run out of disk space then:

  1. Shutdown the three services: 
    docker service update --replicas=0 opam-repo-ci_bridge
docker service update --replicas=0 opam-repo-ci_ci
docker service update --replicas=0 opam-repo-ci_datakit
  2. Delete (or move) /var/lib/docker/volumes/opam-repo-ci_datakit-data/_data/.git.
  3. Restart the three services: 
    docker service update --replicas=1 opam-repo-ci_datakit
docker service update --replicas=1 opam-repo-ci_ci
docker service update --replicas=1 opam-repo-ci_bridge

The CI will rebuild each active PR at this point.

Documentation

The documentation and API reference is automatically generated by ocamldoc from the interfaces. It can be consulted online and there is a generated version in the doc directory of the distribution.

Usage

Work In Progress.

Some examples live at: