Ad-hoc Jobs as a Service (JaaS)

This project provides a simple Golang CLI tool that binds to the Docker Swarm API to create an ad-hoc/one-shot Service and then poll until it exits. Service logs can also be retrieved if the experimental feature is enabled on the Docker daemon.

Motivation and context

For a blog post covering use-cases for JaaS and more on the portions of the Docker API used see below:

Blog: One-shot containers on Docker Swarm

See also: Serverless

If you would like to build Serverless applications with Docker Swarm or Kubernetes checkout my write-up on OpenFaaS:

OpenFaaS.com

The OpenFaaS project has dozens of contributors and thousands of GitHub stars - if you're here because you want to run short-lived functions then I highly recommend checking out OpenFaaS now.

Contributions are welcome

See the contributing guide and do not raise a PR unless you've read it all.

Get started

Build and install the code

Pre-requisites:

Docker 1.13 or newer (experimental mode must be enabled if accessing service logs)
Go 1.9.2 (or Golang container)
Enable Swarm Mode (docker swarm init)

Run these commands

# export GOPATH=$HOME/go
# go get -d -v github.com/alexellis/jaas
# cd $GOPATH/src/github.com/alexellis/jaas
# go install
# export PATH=$PATH:$GOPATH/bin

Now test jaas with jaas --help

Running a task / batch job / one-shot container

Run your first one-shot container:

# jaas alexellis2/cows:latest

The exit code from your container will also be available, you can check it with echo $?

Hiding logs

If you aren't interested in the output logs then run it with the --showlogs=false override:

# jaas --showlogs=false alexellis2/cows:latest

Removing service after completion

To remove the service after it completes, run with the --rm flag:

# jaas --env url=http://blog.alexellis.io/ alexellis2/href-counter:latest

Service created: peaceful_shirley (uva6bcqyubm1b4c80dghjhb44)
ID:  uva6bcqyubm1b4c80dghjhb44  Update at:  2017-03-14 22:19:54.381973142 +0000 UTC
...

Exit code: 0
State: complete


Printing service logs
?2017-03-14T22:19:55.660902727Z com.docker.swarm.node.id=b2dqydhfavwezorhkqi11f962,com.docker.swarm.service.id=uva6bcqyubm1b4c80dghjhb44,com.docker.swarm.task.id=yruxuawdipz2v5n0wvvm8ib0r {"internal":42,"external":2}

Removing service...

Docker authentication for registries

You can use jaas with Docker images in private registries or registries which require authentication.

Just run docker login then pass the --registryAuth parameter and the encoded string you find in ~/.docker/config.json.

If you want to encode a string manually then do the following:

$ export auth='{
    "username" : "myUserName",
    "password" : "secret",
    "email" : "my@email",
    "serveraddress" : "my.reg.domain"
  }'
$ jaas --registryAuth="`echo $auth | base64`" my.reg.domain/hello-world:latest

Notes on images

You can have a multi-node swarm but make sure whatever image you choose is available in an accessible registry.

A local image will not need to be pushed to a registry.

Running jaas in a container

You can also run jaas in a container, but the syntax becomes slightly more verbose:

# docker run -ti -v /var/run/docker.sock:/var/run/docker.sock \
  alexellis2/jaas alexellis2/cows:latest

Roadmap:

Here are several features / enhancements on the roadmap, please make additional suggestions through Github issues.

Optionally delete service after fetching exit code/logs
Support passing environmental variables
Support private registry auth via -registryAuth flag

Todo:

Support constraints on where to run tasks
Support optional secrets through CLI flag
Validation around images which are not in local library
Extract stdout/stderr etc from logs in human readable format similar to docker logs

Future:

When task logs are available in the API this will be used instead of service logs.
When event streams are released they will prevent the need to poll continually