Couldn't find helm repository?
szinn opened this issue · 20 comments
https://github.com/szinn/k8s-homelab/actions/runs/5206160235/jobs/9392423235
It seemed to generate a diff but showed the error in the PR - szinn/k8s-homelab#1905
The helm repository is an OCI repository - could that be it?
It looks like it is having a problem finding the HelmRepository:
flux-local error: Unable to find HelmRepository for flux-system-bjw-s-charts/app-template for HelmRelease lidarr-exporter
$ git clone https://github.com/szinn/k8s-homelab.git
$ cd k8s-homelab/
$ python3 -m venv venv
$ source venv/bin/activate
$ pip3 install flux-local
$ flux-local get cluster --path kubernetes/
NAME PATH KUSTOMIZATIONS
flux-system-add-ons ./kubernetes/apps/flux-system/add-ons 1
media-lidarr ./kubernetes/apps/media/lidarr 1
media-radarr ./kubernetes/apps/media/radarr 1
media-radarr-4k ./kubernetes/apps/media/radarr-4k 1
media-sonarr ./kubernetes/apps/media/sonarr 1
monitoring-vector ./kubernetes/apps/monitoring/vector 1
networking-external-services ./kubernetes/apps/networking/external-services 1
self-hosted-shlink ./kubernetes/apps/self-hosted/shlink 1
system-intel-device-plugin ./kubernetes/apps/system/intel-device-plugin 1
cluster ./kubernetes/cluster 78
This is showing each of these objects above as a separate cluster. For some reason it can't make the linkage between kubernetes/cluster and those other applications. (e.g. cluster has 78 kustomizations, but i'm assuming all those other ones should be in the same cluster too.
Very interesting - so I should be able to play with it locally as well?
Will tinker a bit this week
@szinn Yeah you can, its all based on a command line tool in this repo. You can set --log-level=DEBUG to see a spew of output about how its discovering the repository. Basically, what it does is tries to find all the Kustomization objects and link them together.
Which Fluxtomization is supposed to include the Kustomization ./kubernetes/apps/media/lidarr? I'm having trouble finding this.
./kubernetes/cluster/config/cluster.yaml pulls in ./kubernetes/cluster/apps.yaml which loads ./kubernetes/apps which is processed by directory traversal rather than an explicit kustomization.
OK thanks for confirming that. I suspected that was the case, but wanted to be sure.
As its walking through the paths it seems to have trouble with lidar's directory structure:
DEBUG:flux_local.git_repo:---
DEBUG:flux_local.git_repo:Source path: kubernetes/apps/media/calibre-web/install.yaml
DEBUG:flux_local.git_repo:Path : kubernetes/apps/media/calibre-web/app
DEBUG:flux_local.git_repo:Parent path: kubernetes/apps/media/calibre-web/app
DEBUG:flux_local.git_repo:Found parent kubernetes/apps/media/calibre-web/app => kubernetes/apps (kubernetes/apps/media/calibre-web/install.yaml)
DEBUG:flux_local.git_repo:---
DEBUG:flux_local.git_repo:Source path: kubernetes/apps/media/immich/install.yaml
DEBUG:flux_local.git_repo:Path : kubernetes/apps/media/immich/app
DEBUG:flux_local.git_repo:Parent path: kubernetes/apps/media/immich/app
DEBUG:flux_local.git_repo:Found parent kubernetes/apps/media/immich/app => kubernetes/apps (kubernetes/apps/media/immich/install.yaml)
DEBUG:flux_local.git_repo:---
DEBUG:flux_local.git_repo:Source path: kubernetes/apps/media/lidarr/install.yaml
DEBUG:flux_local.git_repo:Path : kubernetes/apps/media/lidarr
DEBUG:flux_local.git_repo:Parent path: kubernetes/apps/media/lidarr
DEBUG:flux_local.git_repo:No parent for kubernetes/apps/media/lidarr (kubernetes/apps/media/lidarr/install.yaml) (parent_path=kubernetes/apps/media/lidarr)
I'm taking a closer look. I think it may be getting confused because of the explicit install.yaml used here and it happens to work on the others since they aren't in the same directory.
I think this is getting confused because there are two types of kustomizations in a single directory. A Kustomization in kustomization.yaml and a fluxtomization in install.yaml.
The tool assumes that the directory structure has meaning with respect to representing dependencies. Yikes.
Ah, with my layout (and others that follow / riff on onedr0p's) the directory structure has meaning but not necessarily for dependencies.
./kubernetes/cluster/config/cluster.yaml is the root entity and it brings in other fluxtomizations and kustomization entities and those alternate back and forth.
For the directory you mention with both types, the parent directory will bring in the install.yaml fluxtomization and that will bring in the kustomization in the same directory which loads the app.
Yes, though I haven't seen this in onedr0p's repo fwiw:
$ cd home-ops
$ flux-local get cluster --path kubernetes/
NAME PATH KUSTOMIZATIONS
cluster ./kubernetes/flux 86
I think it works in his repo because he uses a separate app subdirectory of the fluxtomization directory that contains the kustomization. It seems like flux-local fails when they're in the same directory. I can see if its possible to adjust, but in the mean time moving to an app subdir can work as a workaround.
I think Devin has it in apps/monitoring/vector
How do I run flux-local from a clone of the repo?
Ah yes, I see what he has - I didn't think the extra sub-directory to hold the two packages was necessary :-) Same pattern for lidarr
I have a test repo where i have been able to reproduce this and can see if its straight forward to fix:
#231
(venv) bash-3.2$ ~/Development/public-repos/flux-local/script/run-in-env.sh flux-local get cluster --path ./kubernetes/cluster/config
NAME PATH KUSTOMIZATIONS
cluster ./kubernetes/cluster 87
Looks like it worked
Great, this has been released in 1.3.1
https://github.com/allenporter/flux-local/releases/tag/1.3.1
Thanks for the report 👍🏼 and let me know if anything else acts up.
Sorry to revisit an old issue, but I'm having a similar issue with my PR flux diff checks.
Run docker://ghcr.io/allenporter/flux-local:main
/usr/bin/docker run --name ghcrioallenporterfluxlocalmain_ed81d7 --label 6ed130 --workdir /github/workspace --rm -e "INPUT_ARGS" -e "HOME" -e "GITHUB_JOB" -e "GITHUB_REF" -e "GITHUB_SHA" -e "GITHUB_REPOSITORY" -e "GITHUB_REPOSITORY_OWNER" -e "GITHUB_REPOSITORY_OWNER_ID" -e "GITHUB_RUN_ID" -e "GITHUB_RUN_NUMBER" -e "GITHUB_RETENTION_DAYS" -e "GITHUB_RUN_ATTEMPT" -e "GITHUB_REPOSITORY_ID" -e "GITHUB_ACTOR_ID" -e "GITHUB_ACTOR" -e "GITHUB_TRIGGERING_ACTOR" -e "GITHUB_WORKFLOW" -e "GITHUB_HEAD_REF" -e "GITHUB_BASE_REF" -e "GITHUB_EVENT_NAME" -e "GITHUB_SERVER_URL" -e "GITHUB_API_URL" -e "GITHUB_GRAPHQL_URL" -e "GITHUB_REF_NAME" -e "GITHUB_REF_PROTECTED" -e "GITHUB_REF_TYPE" -e "GITHUB_WORKFLOW_REF" -e "GITHUB_WORKFLOW_SHA" -e "GITHUB_WORKSPACE" -e "GITHUB_ACTION" -e "GITHUB_EVENT_PATH" -e "GITHUB_ACTION_REPOSITORY" -e "GITHUB_ACTION_REF" -e "GITHUB_PATH" -e "GITHUB_ENV" -e "GITHUB_STEP_SUMMARY" -e "GITHUB_STATE" -e "GITHUB_OUTPUT" -e "RUNNER_OS" -e "RUNNER_ARCH" -e "RUNNER_NAME" -e "RUNNER_ENVIRONMENT" -e "RUNNER_TOOL_CACHE" -e "RUNNER_TEMP" -e "RUNNER_WORKSPACE" -e "ACTIONS_RUNTIME_URL" -e "ACTIONS_RUNTIME_TOKEN" -e "ACTIONS_CACHE_URL" -e "ACTIONS_RESULTS_URL" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/homelab/homelab":"/github/workspace" ghcr.io/allenporter/flux-local:main diff helmrelease --unified 6 --path /github/workspace/pull/kubernetes/apps --path-orig /github/workspace/default/kubernetes/apps --strip-attrs "helm.sh/chart,checksum/config,app.kubernetes.io/version,chart" --limit-bytes 10000 --all-namespaces --sources "home-kubernetes" --output-file diff.patch
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
flux-local error: Unable to find HelmRepository for flux-system-prometheus-community/kube-prometheus-stack for HelmRelease kube-prometheus-stack
I've run your local debugging testing and see results that I believe look correct
/ # flux-local get cluster --path /homelab/kubernetes/
PATH KUSTOMIZATIONS
kubernetes 38
/ # flux-local get helmreleases --path /homelab/kubernetes --all-namespaces
Unable to find Secret monitoring/snmp-secret referenced in HelmRelease monitoring/snmp-exporter
NAMESPACE NAME REVISION CHART SOURCE
services atuin 2.4.0 services-app-template bjw-s
auth authelia 2.4.0 auth-app-template bjw-s
cert-manager cert-manager v1.13.3 cert-manager-cert-manager jetstack
kube-system cilium 1.14.5 kube-system-cilium cilium
networking cloudflared 2.4.0 networking-app-template bjw-s
database cloudnative-pg 0.20.0 database-cloudnative-pg cloudnative-pg
kube-system coredns 1.29.0 kube-system-coredns coredns
networking echo-server 2.4.0 networking-app-template bjw-s
networking external-dns-cloudflare 1.13.1 networking-external-dns external-dns
networking external-dns-pihole 1.13.1 networking-external-dns external-dns
kube-system external-secrets 0.9.11 kube-system-external-secrets external-secrets
kube-system onepassword-connect 2.4.0 kube-system-app-template bjw-s
monitoring gatus 2.4.0 monitoring-app-template bjw-s
monitoring grafana 7.0.19 monitoring-grafana grafana
monitoring kube-prometheus-stack 55.5.1 monitoring-kube-prometheus-stack prometheus-community
monitoring kubernetes-dashboard 6.0.8 monitoring-kubernetes-dashboard kubernetes-dashboard
auth lldap 2.4.0 auth-app-template bjw-s
kube-system local-path-provisioner None kube-system-./deploy/chart/local-path-provisioner local-path-provisioner
kube-system metrics-server 3.11.0 kube-system-metrics-server metrics-server
storage minio 2.4.0 storage-app-template bjw-s
networking nginx-external 4.9.0 networking-ingress-nginx ingress-nginx
networking nginx-internal 4.9.0 networking-ingress-nginx ingress-nginx
database redis 18.6.2 database-redis bitnami
kube-system reloader 1.0.60 kube-system-reloader stakater
default smtp-relay 2.4.0 default-app-template bjw-s
kube-system snapshot-controller 2.0.4 kube-system-snapshot-controller piraeus
monitoring snmp-exporter 2.4.0 monitoring-app-template bjw-s
kube-system synology-csi 0.9.7 kube-system-synology-csi synology-csi
monitoring unpoller 2.4.0 monitoring-app-template bjw-s
volsync volsync 0.8.0 volsync-volsync backube
flux-system weave-gitops 4.0.36 flux-system-weave-gitops weave-gitops
The flux-local get helmreleases even shows the kube-prometheus-stack helmrelease that the PR check is reporting missing.
FWIW, I pulled the latest main docker image and ran this test there.
What is maybe interesting is that I get a missing HelmRepository that seems to relate to the changing package in the PR. In another one of my PRs it was a missing redis HelmRepository
Any ideas?
Would you be up for filing a new separate issue and we can continue discussing there?
One thought I have is that the root cause is #483 but we can dig in more and see.
One thing we could do is have you enable debug: true when running the actions and we can see if something fishy is happening in the intermediate steps that drops the HelmRepository
Sure thing!
I'll go ahead and add the debug flag now and trigger the build again.