Argo CD environment handling POC

This repository contains a concept suggestion for handling environments with GitOps process and Argo CD.

Contents:

Local setup
Concept
References

Local setup

In this section you can find how to test out this concept yourself on your local machine. In order for this to work smoothly, I recommend you to fork this repository. The start script will push to the repository as the sealed secrets example won't work otherwise for your local Kubernetes cluster. Check out what you need to do after forking.

Tools you'll need

kubectl (https://kubernetes.io/docs/tasks/tools/)
helm (https://helm.sh/docs/intro/install/)
docker (https://docs.docker.com/engine/install/)
k3d (https://k3d.io/v5.3.0/#installation)
kubeseal (https://github.com/bitnami-labs/sealed-secrets/releases/tag/v0.17.3) (take more recent version if available)

Once you've installed all those tools, run

./start

This will set up a local kubernetes cluster, install Argo CD on it and set up the Argo CD applications for "dev" and for "prod".

Next, enter

kubectl port-forward service/argocd-server -n argocd 18080:443

to start port-forwarding to your Argo UI.

In your browser open http://localhost:18080. The credentials are admin and 1234.

⚠️ NEVER run this setup on any public or exposed environment without changing the setup of credentials!

When you're done and want to clean up, simply run

k3d cluster delete argo-cd-poc

to remove the whole local cluster.

For forked repositories:

If you've forked this repository, did some changes to the code and now want to see the changes in your local Argo CD, you'll have to change the following files:

infra/helm/values.yaml => apps.a.repoUrl and apps.b.repoUrl
infra/application-dev.yaml => spec.source.repoURL
infra/application-prod.yaml => spec.source.repoURL

Change the repoUrls to your own git-repository.

Concept

Background

I noticed that the repository handling the desired state for the application is getting to big in my usual setup. The usual setup is to store all the deployment related files inside one repository. This was my first take on GitOps with Argo CD.

New way

In order to keep the desired state repository (here the directory "infra") minimal, keep the Helm Chart definitions with your application code.

How it works

For better understanding of the following section, please assume that app-A , app-B and infra are on separate repositories. For simplicity, I used the same source code for app-A and app-B which is why the application source code is outside those directories. Assume, that the source code would normally be inside app-A and app-B.

The application repositories

The application repositories contain their own Helm Chart definitions (or whatever technology you use for deployment) next to the source code. Inside the Helm Chart, only the default values are defined (values.yaml). There are no environment specific values anywhere.

Example outline:

|
+-- src
|   +-- ...
+-- helm
|   +-- templates
|   +-- Chart.yaml
|   +-- values.yaml
|

The desired state repository

The desired state repository now actually only defines the state of the deployment that you want.

Inside this repository you have a Helm Chart (or what ever you like to deploy with). The Helm Chart templates contain the Application files for your inner applications. They reference Helm Charts on application repositories. Inside the Application files, you can define that Helm should overwrite some values in the referenced Helm Chart.

Assuming your (repo of app) helm/values.yaml looks something like this:

app:
  tag: latest
  env:
    port: 8080

You can overwrite that from within your Application file (repo of state) helm/templates/app.yaml:

# ...
spec:
  source:
    helm:
      values: |
        app:
          env:
            port: 80

Now your application will be deployed by Argo CD with the port 80 instead of 8080.

Unfortunately, you cannot use the spec.source.helm.valueFiles as those are relative to the path defined in spec.source.path. As we are on a completely different repository than the path, we cannot use it. To make this hurdle a bit smaller, we can do the following setup.

(state repo) templates/app.yaml

# ...
spec:
  source:
    path: app/helm
    repoURL: { { .Values.app.repoUrl } }
    targetRevision: HEAD
    { { - with .Values.overrideValues } }
    helm:
      values: |{{ toYaml . | nindent 8 }}
    { { - end } }
  destination:
# ...

(state repo) environment-values/prod.yaml

app:
  repoUrl: http://some-example.git
  overrideValues:
    app:
      env:
        port: 80

This way it becomes a bit easier to handle the overwritten values as they're bundled up inside the environment specific values file.