/gitops-patterns

Collection of patterns, examples and resources for GitOps process design, GitOps repository structures, etc

MIT LicenseMIT

cloudogu/gitops-patterns

Intro

This repo collects some patterns (AKA strategies, models, approaches, best practices, etc.) about GitOps process design (AKA GitOps repository structures, etc.) as well as links to talks, articles and example repos.

It originated in schnatterer's talk about the "perfect" GitOps process, our experience in consulting, and our GitOps trainings for both Flux and ArgoCD.

You can read more about the patterns in this series of articles.

For questions or suggestions, you are welcome to join us at our community forum.

Discuss it on myCloudogu

PRs welcome!

Contents

Articles

article icon Part 1️⃣: GitOps Chasm + categories
article icon Part 2️⃣: Deployment patterns
article icon Part 3️⃣: Repo patterns
article icon Part 4️⃣: Promotion patterns
article icon Part 5️⃣: Wiring patterns + conclusion

Pattern categories

Let's group GitOps patterns into separate categories to make them easier to grasp.

  • Operator deployment: GitOps operators ↔ Clusters/Namespaces
  • Repository: How many repos?
  • Promotion:
    • Environments: How to model environments/stages?
    • Implementation: How to implement the actual process of promotion?
  • Wiring: Bootstrapping operator, linking repos and folders

GitOps Patterns

Operator deployment

  • Hub and Spoke 12 1 Operator : n Clusters
    Synonyms: Management Cluster 3 - sounds familiar, e.g. from Cluster API, but it requires a cluster. In SaaS/PaaS constellations, the central instance may not even be operated in a cluster. Therefore, the term "Hub and Spoke" seems more fitting to us, especially since it is also well known from computer networks or logistics.
    Hube and spoke
  • Instance Per Cluster 3 : 1 Operator : 1 Cluster
    Synonyms: Standalone 1 - could apply to both "Instance per Cluster" and "Instance per Namespace"; does not define the relationship to clusters and namespaces precisely enough, as not every operator supports both deployment patterns.
    Instance Per Cluster
  • Instance per Namespace: n Operators : 1 Cluster
    Synonyms: Namespaced 4
    Instance per Namespace

More Patterns:

  • Split-Instance1: 1 Operator : n Clusters; components split between management and target clusters
  • Hybrids of Hub and Spoke and Instance per Cluster patterns: "Instance per Logical Group" and "Argo Managing more Argos".3

Repository structure

  • Monorepo 25
    Opposite: polyrepo 2
  • Repo per Team 5
    More generalized: Team ➡️ Tenant5
  • Repo per Application 25
    Implementations:
    • Config-Code-Separation 6
    • Config replication 4
    • Repo pointer 74
  • Repo per environment 5
    Synonym: Environment per repository8, Repo per Stage

Promotion

We understand the term "promotion" as the process of deploying applications to different environments.
Sometimes, the term "promotion" is prefixed with other words: Release/Application/Environment/Workload/Change.

For promotion, we see different sets of patterns:

  • one regarding the modelling of environments and
  • one regarding the implementation of the actual process of promotion.

Environments

  • Folder/Directory per environment
    Synonym: Environment per folder 8
  • Repo per environment 5
    Synonym: Environment per repo 8
  • Branch per environment
    Synonym: Environment per branch 8
    Often discouraged 7 or declared an anti-pattern910, but can work 11.
  • Preview environments 12 13 14
    Synonyms: ephemeral, dynamic, pull request15, test, temporary12
    Beyond the GitOps world also known as "Preview Deployments"16 and "Deploy Previews"17

Implementation

  • Configuration Management
    Synonyms: Templating, Patching, Overlay, Rendering
    • Plain kustomize (kustomization.yaml) - "operator-agnostic" (works for Argo CD and Flux)
    • Helm
      • via CRD such as HelmRelease (Flux) or Application (ArgoCD)
      • via Umbrella Chart18
      • via helm template on CI server
  • Global Environments vs Environment per App4
    Global Envs Env per app
  • Config update
    Who updates image (version) in GitOps repo, creates branch and PR?
    • Manual: Human pushes branch and create PR 🥵
    • CI Server: Build job pushes branch, creates PR
    • Image Updater: Operator pushes branch, create PR manually
    • Dependency Bot: Bot pushes branch, creates PR

Wiring

  • Bootstrapping , e.g. using kubectl or operator-specific CLI such as flux or argocd-autopilot
  • Linking e.g. using Operator-specific CRDs such as Kustomization (Flux) or Application (ArgoCD)
    • Nesting, e.g. App of Apps pattern 19 (ArgoCD)
    • Templating, e.g. implemented using ApplicationSets

Public GitOps Repo Examples

See also 4.

GitOps Playground

cloudogu/gitops-playground

  • Repo pattern: Per team mixed with per app
  • Operator pattern: Instance per Cluster (Hub and Spoke also possible)
  • Operator: ArgoCD (Flux)
  • Boostrapping: Helm, kubectl
  • Linking: ArgoCD Application
  • Features:
    • Env per app Pattern
    • Operate ArgoCD with GitOps
    • Config Update via CI server
    • Mixed repo patterns
    • ArgoCD and Flux examples

ArgoCD autopilot

argoproj-labs/argocd-autopilot

  • Repo pattern: Monorepo
  • Operator pattern: Instance per Cluster / Hub and Spoke
  • Operator: ArgoCD
  • Boostrapping: argocd-autopilot
  • Linking: kustomization.yaml, ArgoCD Application, ApplicationSet
  • Features:
    • Operate ArgoCD with GitOps
    • In the future: a lot more automation and YAML creation

Flux Monorepo

fluxcd/flux2-kustomize-helm-example

  • Repo pattern: Monorepo
  • Operator pattern: Instance per Cluster
  • Operator: Flux
  • Boostrapping: flux CLI
  • Linking: kustomization.yaml, Flux Kustomization
  • Features: cross-cutting infra

Flux repo per team/tenant

fluxcd/flux2-multi-tenancy

  • Repo pattern: Repo per team/tenant
  • Operator pattern: Instance per Cluster
  • Operator: Flux
  • Boostrapping: flux CLI
  • Linking: kustomization.yaml, Flux Kustomization
  • Features: cross-cutting infra

📕 Path to GitOps examples

christianh814/example-kubernetes-go-repo

christianh814/example-kubernetes-goflux-repo

christianh814/example-openshift-go-repo

  • Repo pattern: Monorepo
  • Operator pattern: Instance per Cluster
  • Operator: [ArgoCD] [flux]
  • Boostrapping: kubectl
  • Linking: kustomization.yaml,
    • ArgoCD Application, ApplicationSet /
    • Flux Kustomization
  • Features:
    • Cross-cutting infra and app(s)
    • ArgoCD and Flux examples

Environment variations

kostis-codefresh/gitops-environment-promotion

  • Operator: ArgoCD (Flux)
  • Features:
    • Env variants for a single app
    • Promotion "via cp"

Synonyms

  • Patterns ≈ strategies, models, approaches, best practices, standards
  • GitOps process design ≈ GitOps repository structures,
  • GitOps Operator ≈ GitOps controller ≈ GitOps agent
  • Config Repo = GitOps repo, Infra repo, Payload repo
    Config repo example
  • App repo = Source code repo, Source repo
    App repo example
  • Environment = Stage
  • Folder = Directory
  • Templating ≈ Patching, Overlay, Rendering, Bundling, Packaging?

References

Footnotes

  1. Article A Comprehensive Overview of Argo CD Architectures – 2023 by Dan Garfield 2 3

  2. Article/Book How to set up your GitOps directory structure by Christian Hernandez 2 3 4

  3. Talk Control Plane, Service, or Both? – Argo CD Multi-Cluster Architectures - Nicholas Morey, Akuity, Article How many do you need? - Argo CD Architectures Explained by Nicholas Morey 2 3

  4. Slides The perfect GitOps process: repos, folders, stages, patterns by Johannes Schnatterer 2 3 4 5

  5. Documentation Flux | Ways of structuring your repositories 2 3 4 5 6

  6. Documentation Argo CD: Best Practices

  7. Talk GitOps: Core Concepts & Ways of Structuring Your Repos by Pinky Ravi and Scott Rigby 2

  8. Lesson GitOps at Scale Lesson series - Git repository strategies by Codefresh (paywalled) 2 3 4

  9. Article Stop Using Branches for Deploying to Different GitOps Environments by Kostis Kapelonis

  10. Article Git best practices: Workflows for GitOps deployments by Christian Hernandez

  11. Article Monitoring and Hardening the GitOps Delivery Pipeline with Flux by Florian Heubeck

  12. Article Creating Temporary Preview Environments Based On Pull Requests With Argo CD And Codefresh by Codefresh 2

  13. Talk GitOps Con Europe - Implementing Preview Environments with GitOps in Kubernetes by François Le Pape, Remazing

  14. Talk Preview Environments with ArgoCD by Brandon Phillips

  15. Video Environments Based On Pull Requests (PRs): Using Argo CD To Apply GitOps Principles On Previews by Viktor Farcic

  16. Documentation Vercel: Preview Deployments

  17. Documentation Netlify: Deploy Previews

  18. Documentation Helm | Chart Development Tips and Tricks

  19. Documentation ArgoCD: Cluster Bootstrapping - App Of Apps Pattern