`registry` parameter ignored when using `manifests_path`

[kcighon]

Describe the bug

We would like to use flux_bootstrap_git in an airgapped environment. Apart from #590 being an issue (our terraform runs in a unpriveleged docker container), setting manifests_path to a location on the container disregards the registry parameter and bootstrap fails as images are set to default (ghcr.io/fluxcd)

Steps to reproduce

example.tf

terraform {
  required_providers {
    flux = {
      source = "fluxcd/flux"
      version = "1.2.3"
    }
  }
}

provider "flux" {
  kubernetes = {
    config_path = "~/.kube/config"
  }
  git = {
    url = "<your_url>"
    branch = "<your_branch>"
    
    # configure credentials
  }
}

resource "flux_bootstrap_git" "this" {
  path = "<your_cluster_path>"
  
  manifests_path = "${path.cwd}/manifests"
  registry             = "<your registry url>"
  version             = "v2.2.3"
}

Expected behavior

Manifests honor the registry parameter when being applied.

Screenshots and recordings

No response

Terraform and provider versions

Terraform v1.7.3
fluxcd/flux Provider v1.2.3

Terraform provider configurations

terraform {
  required_providers {
    flux = {
      source = "fluxcd/flux"
      version = "1.2.3"
    }
  }
}

flux_bootstrap_git resource

resource "flux_bootstrap_git" "this" {
  path = "<your_cluster_path>"
  
  manifests_path = "${path.cwd}/manifests"
  registry             = "<your registry url>"
  version             = "v2.2.3"
}

Flux version

v2.2.3

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[stefanprodan]

Yes this is by design. When you pass the raw manifests they are applied as they are. You need to patch the images inside the overlay.

[kcighon]

Thanks @stefanprodan - is there an alternative solution to manifests_path for airgapped scenarios?

[stefanprodan]

You can set the image patches in Git and bootstrap will use them along with manifests_path

[kcighon]

Any chance you have an example git repo?

[stefanprodan]

You can use a kustomization.yaml like so:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - gotk-components.yaml
  - gotk-sync.yaml
images:
  - name: ghcr.io/fluxcd/source-controller
    newName: my.reg/fluxcd/source-controller
  - name:  ghcr.io/fluxcd/kustomize-controller
    newName: my.reg/fluxcd/kustomize-controller
  - name:  ghcr.io/fluxcd/helm-controller
    newName: my.reg/fluxcd/helm-controller
  - name:  ghcr.io/fluxcd/notification-controller
    newName: my.reg/fluxcd/notification-controller
  - name:  ghcr.io/fluxcd/image-reflector-controller
    newName: my.reg/fluxcd/image-reflector-controller
  - name:  ghcr.io/fluxcd/image-automation-controller
    newName: my.reg/fluxcd/image-automation-controller

And pass this to kustomization_override

[kcighon]

Many thanks. That works well.

As an alternative, do you see any issues with the below to avoid downloading the manifests.tar.gz?:

resource "local_file" "flux_install" { 
  content = <<-EOT 
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources: 
- flux.yaml
EOT
 filename = "${path.module}/files/kustomization.yaml"
 provisioner "local-exec" {
   command = <<-EOT
flux install --export --registry ${var.registry} > ${path.module}/files/flux.yaml 
EOT 
  } 
}
resource "flux_bootstrap_git" "this" {
  depends_on = [local_file.flux_install]
  path = "<your_cluster_path>"
  
  manifests_path = "${path.cwd}/manifests"
  registry             = "<your registry url>"
  version             = "v2.2.3"
}

[stefanprodan]

If you use the CLI binary, then you really need to make sure the CLI is at same version as the one specified in flux_bootstrap_git.

[kcighon]

Understood - the same would apply to automating the download of the manifests.tar.gz as a step within our docker container build. We end up version locked either way for airgapped usage.

Many thanks for your help.

[swade1987]

@kcighon you happy for this issue to be closed inlight of #635 being added to the next release.

[kcighon]

@kcighon you happy for this issue to be closed inlight of #635 being added to the next release.

That's great - many thanks.