Poor Man's CI (PMCI - Poor Man's Continuous Integration) is a collection of scripts that taken together work as a simple CI solution that runs on Google Cloud. While there are many advanced hosted CI systems today, and many of them are free for open source projects, none of them seem to offer a solution for the BSD operating systems (FreeBSD, NetBSD, OpenBSD, etc.)
The architecture of Poor Man's CI is system agnostic. However in the implementation provided in this repository the only supported systems are FreeBSD and NetBSD. Support for additional systems is possible.
Poor Man's CI runs on the Google Cloud. It is possible to set it up so that the service fits within the Google Cloud "Always Free" limits. In doing so the provided CI is not only hosted, but is also free! (Disclaimer: I am not affiliated with Google and do not otherwise endorse their products.)
A CI solution listens for "commit" (or more usually "push") events, builds the associated repository at the appropriate place in its history and reports the results. Poor Man's CI implements this very basic CI scenario using a simple architecture, which we present in this section.
Poor Man's CI consists of the following components and their interactions:
Controller
: Controls the overall process of accepting GitHubpush
events and starting builds. TheController
runs in the Cloud Functions environment and is implemented by the files in thecontroller
source directory. It consists of the following components:Listener
: Listens for GitHubpush
events and posts them aswork
messages to theworkq
PubSub.Dispatcher
: Receiveswork
messages from theworkq
PubSub and a free instancename
from theBuilder Pool
. It instantiates abuilder
instance namedname
in the Compute Engine environment and passes it the link of a repository to build.Collector
: Receivesdone
messages from thedoneq
PubSub and posts the freed instancename
back to theBuilder Pool
.
PubSub Topics
:workq
: Transportswork
messages that contain the link of the repository to build.poolq
: Implements theBuilder Pool
, which contains thename
's of availablebuilder
instances. To acquire abuilder
name, pull a message from thepoolq
. To release abuilder
name, post it back into thepoolq
.doneq
: Transportsdone
messages (builder
instance terminate and delete events). These message contain thename
of freedbuilder
instances.
builder
: Abuilder
is a Compute Engine instance that performs a build of a repository and shuts down when the build is complete. Abuilder
is instantiated from a VMimage
and astartx
(startup-exit) script.Build Logs
: A Storage bucket that contains the logs of builds performed bybuilder
instances.Logging Sink
: ALogging Sink
capturesbuilder
instance terminate and delete events and posts them into thedoneq
.
A structural view of the system is presented below:
A behavioral view of the system follows:
Prerequisites:
-
Empty project in Google Cloud.
-
Google Cloud SDK installed.
-
gcloud init
command has been run.
Instructions:
-
Obtain a
SECRET
that will guard access to your PMCI deployment.$ openssl rand -hex 16 SECRET
-
Deploy PMCI to your project:
$ ./pmci deploy SECRET
-
Obtain your personal access
TOKEN
by visiting github.com > Account > Settings > Developer settings > Personal access tokens. -
On every project you want to use PMCI go to github.com > Project > Settings > Webhooks > Add Webhook.
- URL:
https://REGION-PROJECT.cloudfunctions.net/listener?secret=SECRET&image=IMAGE&token=TOKEN
- Set
REGION
andPROJECT
accordingly. - Set
IMAGE
tofreebsd
ornetbsd
.
- Set
- Content-type:
application/json
- "Just the
push
event."
- URL:
-
Add a shell script named
.pmci/IMAGE.sh
(whereIMAGE
isfreebsd
ornetbsd
) to your project. This script will be run by PMCI on every push. For example, here is my cgofuse script for FreeBSD:set -ex # FUSE kldload fuse pkg install -y fusefs-libs # cgofuse: build and test export GOPATH=/tmp/go mkdir -p /tmp/go/src/github.com/billziss-gh cp -R /tmp/repo/cgofuse /tmp/go/src/github.com/billziss-gh cd /tmp/go/src/github.com/billziss-gh/cgofuse go build ./examples/memfs go build ./examples/passthrough go test -v ./fuse
-
You should now have working FreeBSD and/or NetBSD builds! Try pushing something into your GitHub project.
-
To undeploy PMCI:
$ ./pmci undeploy
NOTE: The default deployment uses a single builder instance of f1-micro
with a 30GB HDD. This fits within the "Always Free" tier and is therefore free. However it is also extremely slow and can even run out of memory when compiling bigger projects (e.g. it runs out of memory 1 out of 5 times when compiling Go in June 2018). Here are some ways to improve the performance:
-
Use a machine type that is faster and has more memory, such as
n1-standard-1
. -
Use a larger HDD or an SSD.
-
FreeBSD only: Use a custom image that has already performed
firstboot
. The default FreeBSD imagefreebsd-11-1-release-amd64
performs a system update and other expensive work when booted for the first time (i.e. the/firstboot
file exists). An image that has already done this work boots much faster.$ ./pmci freebsd_builder_create builder0 # wait until builder has fully booted; it will do so twice; # when the login prompt is presented in the serial console proceed $ ./pmci builder_stop builder0 $ ./pmci builder_image_create builder0 freebsd-builder $ ./pmci builder_delete builder0 # now modify your controller/index.js file to point to your custom freebsd-builder image $ ./pmci deploy SECRET
PMCI supports status badges that show the last status of your build. Use them as follows in Markdown:
Badge:
![PMCI](http://storage.googleapis.com/PROJECT-logs/github.com/USER/REPO/IMAGE/badge.svg)
Badge that links to the build log:
[![PMCI](http://storage.googleapis.com/PROJECT-logs/github.com/USER/REPO/IMAGE/badge.svg)](http://storage.googleapis.com/PROJECT-logs/github.com/USER/REPO/IMAGE/build.html)
-
The
Builder Pool
is currently implemented as a PubSub; messages in the PubSub contain the names of availablebuilder
instances. Unfortunately a PubSub retains its messages for a maximum of 7 days. It is therefore possible that messages will be discarded and that your PMCI deployment will suddenly find itself out of builder instances. If this happens you can reseed theBuilder Pool
by running the commands below. However this is a serious BUG that should be fixed. For a related discussion see https://tinyurl.com/ybkycuub.$ ./pmci queue_post poolq builder0 # ./pmci queue_post poolq builder1 # ... repeat for as many builders as you want
-
The
Dispatcher
is implemented as a Retry Background Cloud Function. It acceptswork
messages from theworkq
and attempts to pull a freename
from thepoolq
. If that fails it returns an error, which instructs the infrastructure to retry. Because the infrastructure does not provide any retry controls, this currently happens immediately and theDispatcher
spins unproductively. This is currently mitigated by a "sleep" (setTimeout
), but the Cloud Functions system still counts the Function as running and charges it accordingly. While this fits within the "Always Free" limits, it is something that should eventually be fixed (perhaps by the PubSub team). For a related discussion see https://tinyurl.com/yb2vbwfd.