buildbox-agent

There are three commands included in the buildbox-agent package:

buildbox-agent - the main job runner, which polls Buildbox for build steps to execute
buildbox-data - reads and writes to your build-wide key/value store
buildbox-artifact - uploads and downloads files to your build-wide file store

Installing

Simply run the following command (see the source), which will automatically download the correct binaries for your platform (or if you'd prefer not to run this install script see the manual installation guide):

bash -c "`curl -sL https://raw.githubusercontent.com/buildboxhq/buildbox-agent/master/install.sh`"

Copy the access token from the agent settings page on Buildbox:

And then start the agent with the token:

~/.buildbox/buildbox-agent start --access-token b9c784528b92d7e904cfa238e68701f1

Now you're all set to run your first build!

If you're using Windows, you'll want to use the Ruby agent.

Launching on system startup

We've some templates for the default process manageers for various platforms. Using a process manager will allow to you ensure buildbox-agent is running on system boot, and will allow for easy upgrades and restarts because you can simply kill the process and the process manager to start it up again.

If you have another to contribute, or need a hand, let us know! (pull requests also welcome)

Upgrading

Upgrading the agent is simply a matter of re-running the install script and then restarting the agent (with a USR2).

Run the install script again. This will download new copies of the binaries. Dont' worry, it won't override your bootstrap.sh file.
```
bash -c "`curl -sL https://raw.githubusercontent.com/buildboxhq/buildbox-agent/master/install.sh`"
```
Tell the buildbox-agent process to restart by sending it an USR2 signal
```
killall -USR2 buildbox-agent
```
This will tell it to finish off any current job, and then shut itself down.
If you use a process monitor such as upstart or launchd it will startup again automatically, with no more work required. If you don't, just start the buildbox-agent like you did initially.
Check your buildbox-agent logs to make sure it has started up the agent again.

If you're running a really long job, and just want to kill it, send the USR2 signal twice. That'll cause the buildbox-agent process to cancel the current job, and then shutdown.

How it works

After starting, buildbox-agent polls Buildbox over HTTPS looking for work.

When a new job is found it executes bootstrap.sh with the standard Buildbox environment variables and any extra environment variables configured in your build pipeline's steps.

As the build is running the output stream is continously sent to buildbox, and when the build finishes it reports the exit status and then returns to looking for new jobs to execute.

Using a bootsrap.sh ensures that buildbox web can't be configured to run arbitrary commands on your server, and it also allows you to configure bootsrap.sh to do anything you wish (although it works out-of-the-box with git-based projects).

Artifact uploads

Uploading artifacts is handled by a seperate tool buildbox-artifact which is bundled with the agent. You can see it's general usage in templates/bootstrap.sh.

Uploading Artifacts to your own S3

If you'd like to upload artifacts to your own Amazon S3 bucket, edit your bootstrap.sh file, and replace the buildbox-artifact call with something like this:

export AWS_SECRET_ACCESS_KEY=yyy
export AWS_ACCESS_KEY_ID=xxx
# export AWS_DEFAULT_REGION=ap-southeast-2 (Optionally set the region. Defaults to us-east-1)
buildbox-artifact upload "$BUILDBOX_ARTIFACT_PATHS" "s3://name-of-your-s3-bucket/$BUILDBOX_JOB_ID" --url $BUILDBOX_AGENT_API_URL

We'll recognise the s3:// prefix and upload the artifacts to S3 to the bucket name name-of-your-s3-bucket (obviously you'll want to change it to the name of your own S3 bucket)

If you upload artifacts to your own S3 Bucket, you can further secure your artifacts by Restricting Access to Specific IP Addresses

Customizing bootstrap.sh

You can think of the bootstrap.sh file as a pre-script for your build. The template that we provide will checkout a Git project, and run your build script, but that's just the default. You can make it do what ever you like. If you don't use Git, you can edit the script to use what ever version control software you like.

You can also use bootstrap.sh to control what repositores are checked out on your CI server. For example, you could add something like this to the top of your bootstrap.sh file:

if [ "$BUILDBOX_REPO" != *github.com/keithpitt/my-app* ]
then
  echo "Unrecognised repo: $BUILDBOX_REPO"
  exit 1
fi

The benefit of the bootstrap.sh is that it's written in bash. You can change it how ever you like and customize how builds get run on your servers.

Manual Installation

Here we'll show you how to manually install the buildbox agent.

Create a folder at ~/.buildbox
```
mkdir -p ~/.buildbox
```
Download the correct binaries for your platform. See: https://github.com/buildboxhq/buildbox-agent/releases/tag/v0.2 for a list for binaries.
```
wget https://github.com/buildboxhq/buildbox-agent/releases/download/v0.2/buildbox-agent-linux-amd64.tar.gz
```
Extract the tar. This should extract buildbox-agent and buildbox-artifact to the ~/.buildbox folder.
```
tar -C ~/.buildbox -zvxf buildbox-agent-linux-amd64.tar.gz
```

Download our example bootstrap.sh and put it in ~/.buildbox

wget -q https://raw.githubusercontent.com/buildboxhq/buildbox-agent/master/templates/bootstrap.sh -O ~/.buildbox/bootstrap.sh

(Optional) Add ~/.buildbox to your $PATH so you can access the binaries eaiser.
```
echo 'export PATH="$HOME/.buildbox:$PATH"' >> ~/.bash_profile
```

Development

Some basic instructions on setting up your Go environment and the codebase for running.

# Make sure you have go installed.
brew install go --cross-compile-common

# Setup your GOPATH
export GOPATH="$HOME/Code/go"
export PATH="$HOME/Code/go/bin:$PATH"

# Install godep
go get github.com/kr/godep

# Checkout the code
mkdir -p $GOPATH/src/github.com/buildboxhq/buildbox-agent
git clone git@github.com:buildboxhq/buildbox-agent.git $GOPATH/src/github.com/buildboxhq/buildbox-agent
cd $GOPATH/src/github.com/buildboxhq/buildbox-agent
godep get
go run agent.go

To test the commands locally:

go run cmd/artifact/artifact.go upload "buildbox/*.go" --agent-access-token=[..] --job [...] --debug
go run cmd/artifact/artifact.go download "buildbox/*.go" . --agent-access-token=[..] --job [...] --debug

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request

kouky/buildkite-agent