Lattice: Run Containerized Workloads

Website: http://lattice.cf
Mailing List: Google Groups

Lattice is an open source project for running containerized workloads on a cluster. Lattice bundles up http load-balancing, a cluster scheduler, log aggregation/streaming and health management into an easy-to-deploy and easy-to-use package.

Lattice is based on a number of open source Cloud Foundry components:

Diego schedules and monitors containerized workloads
Loggregator aggregates and streams application logs
Gorouter provides http load-balancing

Deploy Lattice

A local deployment of Lattice can be launched with Vagrant.

A scalable cluster deployment of Lattice can be launched with Terraform. We currently support AWS, DigitalOcean, and Google Cloud

Use Lattice

The Lattice CLI ltc provides a command line interface for launching docker-based applications.

More complex workloads can be constructed and submitted directly to Lattice's Receptor API which is fully documented here.

Local Deployment

Launching with Vagrant

Make sure you have Vagrant installed, then:

git clone git@github.com:cloudfoundry-incubator/lattice.git
cd lattice
git checkout <VERSION>
vagrant up

This spins up a virtual environment that is accessible at 192.168.11.11. Here, VERSION refers to the tagged version you wish to deploy. These tagged versions are known to be stable.

Use the Lattice CLI to target Lattice:

ltc target 192.168.11.11.xip.io

Using Different Providers

You can do this with either VirtualBox or VMware Fusion (version 7 or later):

Virtualbox:

vagrant up --provider virtualbox

VMware Fusion:

vagrant up --provider vmware_fusion

Networking Conflicts

If you are trying to run both the VirtualBox and VMWare providers on the same machine, you'll need to run them on different private networks (subnets) that do not conflict.

Set the System IP to an address that does not conflict with the host networking configuration by passing the LATTICE_SYSTEM_IP environment variable to the vagrant up command:

LATTICE_SYSTEM_IP=192.168.80.100 vagrant up
ltc target 192.168.80.100.xip.io

Updating

Currently, Lattice does not support updating via provision. So to update, you have to destroy the box and bring it back up:

 vagrant destroy --force
 git pull
 vagrant up

Troubleshooting

No such host errors

DNS resolution for xip.io addresses can sometimes be flaky, resulting in errors such as the following:

 ltc target 192.168.11.11.xip.io
 Error verifying target: Get http://receptor.192.168.11.11.xip.io/v1/desired_lrps:
 dial tcp: lookup receptor.192.168.11.11.xip.io: no such host

First, check your networking DNS settings. Local "forwarding DNS" servers provided by some home routers can have trouble resolving xip.io addresses. Try setting your DNS to point to your real upstream DNS servers, or alternatively try using Google DNS by using 8.8.8.8 and/or 8.8.4.4.

Unfortunately xip.io itself also sometimes returns similar "no such host" errors. The recommended alternative is to follow the dnsmasq instructions, pass the LATTICE_SYSTEM_DOMAIN environment variable to the vagrant up command, and target using lattice.dev instead of 192.168.11.11.xip.io to point to the cluster, as follows:

LATTICE_SYSTEM_DOMAIN=lattice.dev vagrant up
ltc target lattice.dev

Miscellaneous

If you have trouble running vagrant up --provider virtualbox with the error

default: Warning: Remote connection disconnect. Retrying...
default: Warning: Authentication failure. Retrying...
...

try upgrading to the latest VirtualBox.

Running Vagrant with a custom Lattice tar

By default, vagrant up will fetch the latest Lattice binary tarball. To use a particular tarball:

VAGRANT_LATTICE_TAR_PATH=/path/to/lattice.tgz vagrant up

Clustered Deployment

This repository contains several Terraform templates to help you deploy on your choice of IaaS. To deploy Lattice in this way you will need:

Terraform >= 0.4.2 installed on your machine
Credentials for your choice of IaaS

Deploying

Here are some step-by-step instructions for deploying a Lattice cluster via Terraform:

Visit the Lattice GitHub Releases page
Select the Lattice version you wish to deploy and download the Terraform example file for your target platform. The filename will be lattice.<platform>.tf
Create an empty folder and place the lattice.<platform>.tf file in that folder.
Update the lattice.<platform>.tf by filling in the values for the variables. Instructions for each supported platform are here:

Run the following commands in the folder containing the lattice.<platform>.tf file

terraform get -update
terraform apply

This will deploy the cluster.

Upon success, terraform will print the Lattice target:

Outputs:

  lattice_target = x.x.x.x.xip.io
  lattice_username = xxxxxxxx
  lattice_password = xxxxxxxx

which you can use with the Lattice CLI to ltc target x.x.x.x.xip.io.

Terraform will generate a terraform.tfstate file. This file describes the cluster that was built - keep it around in order to modify/tear down the cluster.

Destroying

To destroy the cluster go to the folder containing the terraform.tfstate file and run:

terraform destroy

Contributing

In the spirit of free software, everyone is encouraged to help improve this project.

Please submit pull requests against the develop branch. Only the continuous integration system commits to master.

Here are some ways you can contribute:

by using alpha, beta, and prerelease versions
by reporting bugs
by suggesting new features
by writing or editing documentation
by writing specifications
by writing code (no patch is too small: fix typos, add comments, clean up inconsistent whitespace)
by refactoring code
by closing issues
by reviewing patches

Also see the Development Readme

Submitting an Issue

We use the GitHub issue tracker to track bugs and features. Before submitting a bug report or feature request, check to make sure it hasn't already been submitted. You can indicate support for an existing issue by voting it up. When submitting a bug report, please include a Gist that includes a stack trace and any details that may be necessary to reproduce the bug including the Lattice version.

Submitting a Pull Request

Propose a change by opening an issue.
Fork the project.
Create a topic branch.
Implement your feature or bug fix.
Commit and push your changes.
Submit a pull request.

MarkKropf/lattice