Laminar

Don't let your deployments be turbulent!

What is this?

Laminar is a reasonably simple GitOps tool, inspired by things like flux but with a very different architecture.

The TL;DR of what it does is this:

Step 1, you tell it about your docker registry(s):

dockerRegistries:
- reg: 1122334455.dkr.ecr.eu-west-2.amazonaws.com/acmecorp
  name: acmecorp-aws

Step 2, you give it (git+ssh) access to your git repo:

git:
- name: manifests
  url: gitoperations@github.com:acmecorp/k8s-manifests.gitoperations
  branch: master
  key: ~/example_ssh_id_rsa  # path to the SSH key (needed for gitoperations)
  pollFreq: 120              # How often to sync..
  updates: []                # list of updates (see next step)

Step 3, you tell laminar which files you want it to operate on (in the same git repo)

git:
- name: manifests
  updates:

  - pattern: "glob:develop-*" # will match docker tags such as "develop-1.2" or "develop-short_sha"
    # where to look in your gitoperations repo
    files:
    - dev/  # laminar will search in directory "dev"

Step 4, run laminar

AWS_PROFILE=myprofile ./laminar

Laminar will then do the following:

clone the git repo k8s-manifests and checkout branch master
search for files under dev/ inside your git repo
inside the files search for strings such as: 1122334455.dkr.ecr.eu-west-2.amazonaws.com/acmecorp/app-name:develop-52af76b8 (any tag that matches develop-*)
index the tags available from 1122334455.dkr.ecr.eu-west-2.amazonaws.com/acmecorp/app-name
check if there are any tags (matching develop-*) which are more recent than develop-52af76b8
a. if there is a more recent tag in ECR (eg: develop-b2ee56fd) update the file from develop-52af76b8 -> develop-b2ee56fd
b: git commit the updated file(s), then git push
sleep until the end of pollFreq
repeat

Main loop sequence

sequenceDiagram
participant laminar as Laminar
participant git as Git
participant dr as Docker Registry

loop every interval
    loop for each git repo
        laminar->>git: clone "k8s-manifest"
        laminar->>laminar: checkout "branch"
        laminar->>laminar: search files for string matching "pattern"
    end
    
    loop for each docker registry
        laminar->>dr: get available images tags
    end
    
    loop for each git repo
        alt new tag found
            laminar->>laminar: replace tag
            laminar->>laminar: git commit
            laminar->>git: push
        end
    end
end

Reasoning

We love weave flux.. but it makes working with templated manifests challenging. If you're running 10x kubernetes clusters it also makes very little sense to have each one polling your docker registries.

With this pattern you only need one tool to automate your GitOps.
This is also compatible with docker-compose or practically any text file really.
easier to co-ordinate many changes at the same time.
Lets assume my manifests are templated in my git repo.. flux would want to patch the file that is the output of my templating. With laminar you can update both, or just one. This opens the door to using kapitan, tanka, cuelang or any templating really.

Features

checkout your git repo(s)
search yaml files looking for obvious docker images
check if there are more recent docker tags are in your registry and ready to be deployed
update your git repo with the more recent tags
dynamically load a list of files and image:tag patterns from the remote git repos (.laminar.yaml)
add exec action so commands can be run after modifying git (and before the git commit)
docker image + deployment manifest
prometheus metrics

issues/todo

exclude list (users can blacklist promoting specific image+tag patterns).. feature untested
after initialCheckout(), if the (remote) git repo is reverted with a --force push we should handle that and re-clone
more tests, do this when refactoring the logic
the main loop is currently (MVP) and simply just a time.Sleep(). There is no concurrnecy/time.Tick() yet.
occasional errors from registry polling not surfacing correctly. probably needs some attention.
tidy up (specifically the business logic around change requests and add maybe add some concurrency)
quick start/tutorial/example docs!
example: PrometheusAlerts
example: grafana dashboard

digtux/laminar