helm repo add bitnami gives not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR

Question

helm repo add bitnami gives not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR

chukka opened this issue 3 years ago · 79 comments

Which chart: Not chart but bitnami helm repo - helm repo add bitnami https://charts.bitnami.com/bitnami
The name (and version) of the affected chart

Describe the bug
As part of CI tests, we add the bitnami repo and do some chart-testing. This breaks CI flow. is there any option to retry or is it a known issue

To Reproduce
Steps to reproduce the behavior:

$ helm repo add bitnami https://charts.bitnami.com/bitnami
Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR

Expected behavior
A clear and concise description of what you expected to happen.

Version of Helm and Kubernetes: 3.5.4

Additional context
This doesn't happen always but fails 2 out of 10 times atleast

Answer 1 · 2021-12-16T06:58:28.000Z

I was not able to reproduce the issue, I tried several times and it worked fine for all of them. Maybe it's related to the AWS outage that happened yesterday, see some news https://www.theguardian.com/technology/2021/dec/15/amazon-down-web-services-outage-netflix-slack-ring-doordash-latest

Are you still facing the issue?

Answer 2 · 2021-12-16T07:10:24.000Z

@carrodher This issue happens intermittently , This is not the first time it happened for us yesterday .

Answer 3 · 2021-12-16T08:08:44.000Z

Hi @chukka, could you tell us since when this problem occurred in your case? My 2cts is that it can be related to any connectivity issue produced by a third-party outage like AWS, DNS provider, etc.
We have several automations to release the charts that are constantly adding the repository and I didn't see any issue, or if there were an issue it was transient and solved with the automatic retries.

Answer 4 · 2021-12-17T12:53:45.000Z

I've had this happen 2-3 times on 15 December and 3 times this morning 17 December:

Downloading common from repo https://charts.bitnami.com/bitnami
Save error occurred:  could not find : chart rabbitmq not found in https://charts.bitnami.com/bitnami: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR
Deleting newly downloaded charts, restoring pre-update state
Error: could not find : chart rabbitmq not found in https://charts.bitnami.com/bitnami: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR

Answer 5 · 2021-12-20T11:26:50.000Z

Hi,

In my case I wasn't able to reproduce the issue either.

Answer 6 · 2021-12-21T08:39:47.000Z

I'm seeing this as well, and it is intermittent. I got this from running terraform locally against a k3d cluster via the helm provider:

Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR

Answer 7 · 2021-12-22T10:10:04.000Z

I forwarded this to the engineering team so they can take a look.

Answer 8 · 2021-12-26T23:46:04.000Z

Doesn't work now. Already about 30 minutes

ArgoCD

rpc error: code = Unknown desc = Manifest generation error (cached): helm repo add https://charts.bitnami.com/bitnami https://charts.bitnami.com/bitnami failed exit status 1: Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR

Answer 9 · 2021-12-27T16:02:55.000Z

Thanks for letting us know, we are looking into it

Answer 10 · 2021-12-29T10:36:06.000Z

Thanks to everyone for reporting this issue. We are facing some transient issues with the Helm registry, it should work if you retry it:

$ helm repo add bitnami2 https://charts.bitnami.com/bitnami
"bitnami2" has been added to your repositories

About users trying to access using the browser, please note this URL is designed for work with the Helm CLI, the only path accessible via browser is https://charts.bitnami.com/bitnami/index.yaml where you can download the index.yaml

We will update the thread with more information once properly analyzed the issue on our side. Thanks!

Answer 11 · 2022-01-12T11:30:31.000Z

We have been doing some tests, for example the following script was running during a couple of days:

#!/bin/bash

set -e

export HELM_DEBUG=1

error() {
    echo "See file /tmp/chart.$$.log for debugging info"
}

command -v helm || exit 1

mkdir -p /tmp/charts/

# Do not fail here
helm repo remove bitnami || true

(helm repo add bitnami https://charts.bitnami.com/bitnami 2> /tmp/charts/$$.log) || error

From the original 20063 probes, about 9.1% failed with this message:

Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR; received from peerError: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR; received from peerhelm.go:88: [debug] stream error: stream ID 1; INTERNAL_ERROR; received from peerlooks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reachedmain.(*repoAddOptions).run helm.sh/helm/v3/cmd/helm/repo_add.go:206main.newRepoAddCmd.func1 helm.sh/helm/v3/cmd/helm/repo_add.go:82github.com/spf13/cobra.(*Command).execute github.com/spf13/cobra@v1.2.1/command.go:856github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/cobra@v1.2.1/command.go:974github.com/spf13/cobra.(*Command).Execute github.com/spf13/cobra@v1.2.1/command.go:902main.main helm.sh/helm/v3/cmd/helm/helm.go:87runtime.main runtime/proc.go:255runtime.goexit runtime/asm_amd64.s:1581

This error seems to be caused by a network issue, it can be potentially caused on both sides: client or server: is not a valid chart repository or cannot be reached

Line https://github.com/helm/helm/blob/39ca699ca790e02ba36753dec6ba4177cc68d417/cmd/helm/repo_add.go#L205 points to be a client error, but it's not a determinant clue.

We have performed some tests and we can fetch the file https://charts.bitnami.com/bitnami/index.yaml 100% of the cases, but sometimes helm fails after curl succeeding:

# Curl 
*   Trying 99.84.160.70...*   Trying 99.84.160.70...* TCP_NODELAY set* Connected to charts.bitnami.com (99.84.160.70) port 443 (#0)* ALPN, offering h2* ALPN, offering http/1.1* successfully set certificate verify locations:*   CAfile: /etc/ssl/cert.pem  CApath: none* TLSv1.2 (OUT), TLS handshake, Client hello (1):} [232 bytes data]* TLSv1.2 (IN), TLS handshake, Server hello (2):{ [100 bytes data]* TLSv1.2 (IN), TLS handshake, Certificate (11):{ [4820 bytes data]* TLSv1.2 (IN), TLS handshake, Server key exchange (12):{ [300 bytes data]* TLSv1.2 (IN), TLS handshake, Server finished (14):{ [4 bytes data]* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):} [37 bytes data]* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):} [1 bytes data]* TLSv1.2 (OUT), TLS handshake, Finished (20):} [16 bytes data]* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):{ [1 bytes data]* TLSv1.2 (IN), TLS handshake, Finished (20):{ [16 bytes data]* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256* ALPN, server accepted to use h2* Server certificate:*  subject: CN=charts.bitnami.com*  start date: Feb 14 00:00:00 2021 GMT*  expire date: Mar 15 23:59:59 2022 GMT*  subjectAltName: host "charts.bitnami.com" matched cert's "charts.bitnami.com"*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon*  SSL certificate verify ok.* Using HTTP2, server supports multi-use* Connection state changed (HTTP/2 confirmed)* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0* Using Stream ID: 1 (easy handle 0x7ffbbb00d600)> GET /bitnami/index.yaml HTTP/2> Host: charts.bitnami.com> User-Agent: curl/7.64.1> Accept: */*> * Connection state changed (MAX_CONCURRENT_STREAMS == 128)!< HTTP/2 200 < content-type: binary/octet-stream< content-length: 11287899< last-modified: Tue, 11 Jan 2022 14:36:14 GMT< x-amz-server-side-encryption: AES256< x-amz-version-id: nmDdVr8LQPExrl5WNUdeu5redpWHeUxw< accept-ranges: bytes< server: AmazonS3< date: Tue, 11 Jan 2022 14:52:52 GMT< etag: "04935efc90d38c851fb4009f3b700563"< x-cache: RefreshHit from cloudfront< via: 1.1 7fa75077148a5e108de410272ea8ccc4.cloudfront.net (CloudFront)< x-amz-cf-pop: ORD52-C2< x-amz-cf-id: oWfuxcKqMqT0AP-kpwCfGBX-IA42jtR6mjFoXGazr0-p2GmBIoFwfQ==< { [15983 bytes data]* Connection #0 to host charts.bitnami.com left intact* Closing connection 0


# Helm
Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR; received from peerhelm.go:88: [debug] stream error: stream ID 1; INTERNAL_ERROR; received from peerlooks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reachedmain.(*repoAddOptions).run helm.sh/helm/v3/cmd/helm/repo_add.go:206main.newRepoAddCmd.func1 helm.sh/helm/v3/cmd/helm/repo_add.go:82github.com/spf13/cobra.(*Command).execute github.com/spf13/cobra@v1.2.1/command.go:856github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/cobra@v1.2.1/command.go:974github.com/spf13/cobra.(*Command).Execute github.com/spf13/cobra@v1.2.1/command.go:902main.main helm.sh/helm/v3/cmd/helm/helm.go:87runtime.main runtime/proc.go:255runtime.goexit runtime/asm_amd64.s:1581

Answer 12 · 2022-02-02T08:28:28.000Z

I'm also seeing this issue pop up in a CI build that calls Helm about once per day.

Answer 13 · 2022-02-22T19:40:27.000Z

I'm having the same issue. This is the message I get:

Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: Get "https://charts.bitnami.com/bitnami/index.yaml": read tcp 10.163.207.67:61516->108.138.167.54:443: wsarecv: An existing connection was forcibly closed by the remote host.

Answer 14 · 2022-02-24T22:04:14.000Z

Hi, I'm also facing a similar type of error. Jenkins build fails and I'm using Terraform HEML Provider to deploy Prometheus HEML chat from bitnami. Tried to get around with it by creating a docker image as well; defining commands to install HELM 3 and adding CMD to add repos, and still didn't help. I also tried replacing the TF HELM repository attribute with "/.cache/helm/repository/" but throws a Protocol Handler error.

Below is the error I have been getting:

module.prometheus.helm_release.grafana: Creating...
module.prometheus.helm_release.prometheus: Creating...
module.prometheus.helm_release.grafana: Still creating... [10s elapsed]
module.prometheus.helm_release.prometheus: Still creating... [10s elapsed]

Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: open /.cache/helm/repository/HpOj5uooN5ifybCQee2mfuZUHA4=-index.yaml: no such file or directory

  with module.prometheus.helm_release.prometheus,
  on .terraform/modules/prometheus/main.tf line 16, in resource "helm_release" "prometheus":
  16: resource "helm_release" "prometheus" {


Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: open /.cache/helm/repository/ON43pZ61a8pJuVDiqDkXwhDeE0I=-index.yaml: no such file or directory

  with module.prometheus.helm_release.grafana,
  on .terraform/modules/prometheus/main.tf line 25, in resource "helm_release" "grafana":
  25: resource "helm_release" "grafana" {

My Terraform HEML module below, tried with chart name without bitnami/ path too and still throws same error as above:

resource "helm_release" "prometheus" {
  name             = "prometheus"
  repository       = "https://charts.bitnami.com/bitnami"
  chart            = "bitnami/prometheus"
  namespace        = "monitoring"
  create_namespace = true
}

// Grafana HEML chart for Visualisation of telemetry data
resource "helm_release" "grafana" {
  name             = "grafana"
  repository       = "https://charts.bitnami.com/bitnami"
  chart            = "bitnami/grafana"
  namespace        = "monitoring"
  create_namespace = true
}

Any info and advice would be grateful, please. Thank you.

Answer 15 · 2022-03-10T16:37:26.000Z

Hi, monitoring didn't show bad connections or alike, on our side, are you still facing this intermittent issue?

Answer 16 · 2022-03-22T19:54:28.000Z

We are also using Terraform to deploy the external DNS Helm chart and have experienced it several times today.
It did not however occur with the metrics-server Helm Chart.

Answer 17 · 2022-03-23T12:58:35.000Z

Yes, we are still seeing this issue

Answer 18 · 2022-03-25T12:59:07.000Z

https://charts.bitnami.com/bitnami

AccessDenied Access Denied SN5Q4NA1V71A1ZKZ /KusJDXgVbJl8HSUg3Q/U45KyzXNVDntRtCPoB34yPbvV4MWBWIXtBe8mQyMNtOHvRVcDfRkQHQ=

Answer 19 · 2022-03-26T10:25:04.000Z

FYI- To workaround this issue, we moved all used (by our team) bitnami charts under https://charts.jfrog.io (powered by artifactory)

if you are using artifactory , you can try Remote Repositories to cache all bitnami charts

Answer 20 · 2022-03-28T09:21:16.000Z

I can't reproduce the issue on my side:

$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

Answer 21 · 2022-03-28T13:44:09.000Z

I reproduced the issue now, but it worked immediately after:

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "bitnami" chart repository (https://charts.bitnami.com/bitnami):
	stream error: stream ID 1; INTERNAL_ERROR; received from peer
Update Complete. ⎈Happy Helming!⎈

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈

Our team is looking into this.

Answer 22 · 2022-03-31T19:12:12.000Z

Howdy all, I would just like to report that I am seeing this issue to, in southeast Texas.

Clone failed: Cloning into 'management-state/catalog-cache/a68a354dfd35755d14bebd02d825feb24ecbbd0496438ceb36691e1eabaa4947'... fatal: Remote branch master not found in upstream origin : exit status 128

Adding via command line still works - but in the above, this is through the Rancher Global Catalog UI.

Answer 23 · 2022-04-01T06:05:33.000Z

Please note this is a totally different and unrelated issue. Helm client doesn't work with git not cloning any Git repository. It seems something totally different and unrelated to Bitnami.

Regarding the original issue related to the Helm registry, we are doing more research on our side in order to find the root cause of those transient errors and mitigate them.

Answer 24 · 2022-04-01T14:49:12.000Z

Ah, well the original message I had was: Update failed: fatal: not a git repository (or any parent up to mount point /var/lib) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). : exit status 128

When I totally removed the repo and added it back, the error I got was what I posted here in my previous comment. Nothing in my environments has changed (I see the same issue in Test/Dev/Pre/Prod K8s cluster envs) - that just popped up this week. The setup had been running fine for a couple of years now.

Answer 25 · 2022-04-01T15:21:33.000Z

Please, note that Helm charts it's not a Git repository but a CDN distribution. It doesn't understand any git API call, and this was causing a lot of errors from the CDN perspective. We set a few rules to prevent these calls from polluting the CDN statistics and thanks to this we're able to find the root issue, that was related to caching settings. We're actively monitoring the issue.

Answer 26 · 2022-04-01T15:40:12.000Z

I am not sure what Rancher UI is doing then, I'm not specifically telling it that this is a Git repo. Rancher has these requrements for adding custom Helm repos:

Custom Git Repository:

The Git URL needs to be one that git clone [can handle](https://git-scm.com/docs/git-clone#_git_urls_a_id_urls_a) and must end in .git. The branch name must be a branch that is in your catalog URL. If no branch name is provided, it will default to use the master branch. Whenever you add a catalog to Rancher, it will be available almost immediately.

Custom Helm Chart Repository:

A Helm chart repository is an HTTP server that contains one or more packaged charts. Any HTTP server that can serve YAML files and tar files and can answer GET requests can be used as a repository server.

Helm comes with a built-in package server for developer testing (helm serve). The Helm team has tested other servers, including Google Cloud Storage with website mode enabled, S3 with website mode enabled or hosting custom chart repository server using open-source projects like [ChartMuseum](https://github.com/helm/chartmuseum).

In Rancher, you can add the custom Helm chart repository with only a catalog name and the URL address of the chart repository.

So perhaps the new filtering that is being done now is preventing the clone? Or preventing Rancher from doing some check against the Helm repo, so it cannot continue? I will see if Rancher supports CDN distros specifically - I suspect they do already however.

Answer 27 · 2022-04-05T16:36:25.000Z

@ArgonV Hi,
I have the same issue here, i've been using bitnami charts in Rancher and everything was fine until last week.

The message error is the same: Clone failed: Cloning into 'management-state/catalog-cache/a68a354dfd35755d14bebd02d825feb24ecbbd0496438ceb36691e1eabaa4947'... fatal: Remote branch master not found in upstream origin : exit status 128

Did you have some new about it?

Answer 28 · 2022-04-05T16:40:57.000Z

Hello @dboc , sadly I do not. It is still broken for us (after adding, deleting, refreshing, removing the branch name from the Catalog add UI, etc...) - and I'm pretty sure the new filtering in-place on the CDN distro is the cause. Other Helm 3 CDN repos add just fine in the Rancher UI.

Answer 29 · 2022-04-05T17:08:36.000Z

@ArgonV , thanks.
There is an open Issue in Ranchers Github's, Maybe its related to the Rancher and not related to bitnami.
Link: rancher/rancher#37136

Answer 30 · 2022-04-11T08:16:03.000Z

Apart from the Rancher UI issue, are you still facing this issue once applied on our side the mitigation described by @jbianquetti-nami at #8433 (comment)?

Please, note that Helm charts it's not a Git repository but a CDN distribution. It doesn't understand any git API call, and this was causing a lot of errors from the CDN perspective. We set a few rules to prevent these calls from polluting the CDN statistics and thanks to this we're able to find the root issue, that was related to caching settings. We're actively monitoring the issue.

Answer 31 · 2022-04-11T20:18:09.000Z

@carrodher I changed the URL to https://raw.githubusercontent.com/bitnami/charts/index/bitnami as was suggested in rancher/rancher#37136 (comment) , refreshed and that worked for me for now.

Answer 32 · 2022-04-12T06:50:06.000Z

Thanks for letting us know, let's see if this workaround fixes the issue in Rancher. We are monitoring any issue using the Helm client

Answer 33 · 2022-04-15T13:45:58.000Z

@carrodher I run into the same problem, I added the url https://github.com/bitnami/charts to get it to refresh in rancher UI. But then I run into the dependency problem of the mariadb chart I wanted to upgrade. That's pointing to the url: https://charts.bitnami.com/bitnami

dependencies:
- name: common
  repository: https://charts.bitnami.com/bitnami
  version: 1.13.0

Answer 34 · 2022-04-16T23:50:50.000Z

About the Rancher issue it was already reported in the Rancher repo, not sure how we can help with this specific topic.

Answer 35 · 2022-05-03T15:36:44.000Z

@carrodher I changed the URL to https://raw.githubusercontent.com/bitnami/charts/index/bitnami as was suggested in rancher/rancher#37136 (comment) , refreshed and that worked for me for now.

I've the same issue. The https://raw.githubusercontent.com/bitnami/charts/index/bitnami worked for me!! Thanks

Answer 36 · 2022-05-04T09:34:54.000Z

Hi,
In our case the error is:

Update failed: fatal: not a git repository (or any parent up to mount point /var/lib) Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set). : exit status 128

this one however worked for us:
https://raw.githubusercontent.com/bitnami/charts/index/bitnami

Answer 37 · 2022-05-04T09:54:54.000Z

This seems a specific error related to rancher, git is not involved in any action related to the Helm registry when using the helm CLI

Answer 38 · 2022-05-04T18:31:37.000Z

This seems a specific error related to rancher, git is not involved in any action related to the Helm registry when using the helm CLI

Yeah, thought the same, but it only fails at Bitnami repository! Also, the other link also works, which makes it really suspicious....

Answer 39 · 2022-05-26T20:21:59.000Z

Looking at comments here #8433 (comment) it looks like 9.1% failure rate is observed and indeed that is similar to what we're seeing with our CI pipelines. Aside from caching charts locally or several retries - any other suggestions?

Along the same lines: is it possible that outages happen when charts are being published and indexes being re-generated?

Answer 40 · 2022-05-30T15:34:39.000Z

Since this afternoon this is a continous issue for us on a gitlab ci/cd env. Locally this worked fine though. Is there any quota system in place with regards to shared ip ranges for the ci providers that now suddenly kicks in?

Answer 41 · 2022-05-31T05:19:15.000Z

It looks like some CloudFront frontends are running terribly slowly - for my locally resolved CF frontends, 2 out of 4 run at ~50KiB/s, and the other 2 run at multi-megabytes per sec.

So 50% of the time based on DNS round-robin, index downloads will take ~5 minutes, and hence time out for any automated process (or interactive process with a sane timeout, like helm repo update).

Answer 42 · 2022-05-31T05:30:19.000Z

Thanks for letting us know, we are looking into it. In fact, we are also facing this same issue in our internal test & release pipelines

Answer 43 · 2022-05-31T08:38:02.000Z

Hello
I confirm the issue on my side. Since Thursday 26th, we have multiple alerts on helmCharts reconciliation with Flux v2. The common point of these charts? They use dependencies charts from https://charts.bitnami.com/bitnami.
The error we have on kubernetes (k describe helmcharts -n <namespace> <chart name>):

dependency build error: failed to add remote dependency 'common': failed to load index for 'common': failed to strategically load index: failed to cache index to temporary file: stream error: stream ID 1; INTERNAL_ERROR; received from peer
dependency build error: failed to add remote dependency 'common': failed to load index for 'common': failed to strategically load index: failed to cache index to temporary file: context deadline exceeded (Client.Timeout or context cancellation while reading body)

Answer 44 · 2022-06-01T07:02:00.000Z

I'm having the same issue.

Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR; received from peer

Answer 45 · 2022-06-01T14:41:24.000Z

We've also started seeing these errors since a few days ago. Please look into it!
Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR; received from peer

Fortunately retrying usually fixes the problem.

Answer 46 · 2022-06-01T15:02:54.000Z

We are experiencing the same issue. Any help we can offer?

Answer 47 · 2022-06-01T15:25:49.000Z

I just got this error minutes ago.

Answer 48 · 2022-06-01T16:25:24.000Z

we are also facing the same issue

Answer 49 · 2022-06-01T16:32:51.000Z

we are as well. our cicd can't get through a workflow because we pull down 2-3 bitnami charts + we're hitting this error

Answer 50 · 2022-06-01T16:47:21.000Z

Also facing this issue

Answer 51 · 2022-06-01T17:40:19.000Z

After some investigation, it seems the root cause is related to CloudFront serving the index.yaml (a pretty fat 14MBs index.yaml with a ton of monthly pulls). Because of that, we're probably reaching CF limits with this volume of traffic.

We are evaluating what is the best way to put the index.yaml on a diet, probably by removing some old versions from the file and keeping just the latest one for a period of time (i.e 6 months). We will post a pinned GH issue with the final decision to mitigate this issue and any workaround for anyone interested in a previous version of the index.yaml

Answer 52 · 2022-06-02T09:29:46.000Z

Hi, we just created a new index.yaml version containing the data from the latest 6 months (see associated PR #10530).
Now the size was significantly reduced so those issues should be gone:

$ curl -SsL https://charts.bitnami.com/bitnami/index.yaml -o index-downloaded.yaml
$ ls -lah index-downloaded.yaml
-rw-rw-r-- 1 bitnami bitnami 3.6M Jun  2 09:27 index-downloaded.yaml

In the same way, we created this pinned issue (#10539) notifying this new change and providing some context.

Don't hesitate to let us know if you are still facing issues.

Answer 53 · 2022-06-02T14:04:13.000Z

This was horribly misguided - seriously...older than less than 6 months ago? Do you have any idea how many charts from you are used in production systems that just broke?

Answer 54 · 2022-06-02T14:14:40.000Z

This is a huge breaking change. Would it not instead be sensible to look at alternative CDNs? I suspect any number of them would be happy to help. In fact, even AWS may be willing to help if you reach out.

Answer 55 · 2022-06-02T14:30:03.000Z

I feel keeping last 6 months version is not a good solution as it will put pressure on developers to update their chart every 6 month. Further I feel that this was breaking change and should not have done without proper announcement and warning.

Answer 56 · 2022-06-02T14:57:22.000Z

Any ideas on how to retrieve helm charts older than the recent 6 months?

Answer 57 · 2022-06-02T14:58:54.000Z

@kulinskyvs you can use index.yaml from commit history. like use https://raw.githubusercontent.com/bitnami/charts/defb094c658024e4aa8245622dab202874880cbc/bitnami/ as repository in Chart.yaml

Answer 58 · 2022-06-02T15:05:31.000Z

Any ideas on how to retrieve helm charts older than the recent 6 months?

helm repo remove bitnami
helm repo add bitnami  https://raw.githubusercontent.com/bitnami/charts/index/bitnami

Answer 59 · 2022-06-02T15:15:11.000Z

All the previous Chart versions still exist and they are available as usual, please check this post. This change only affects the index.yaml listing.

This change can potentially affect users or CI/CDs trying to package charts (helm package or helm dep update). In any case regular installations or deployments.

Apart from that, in the above-mentioned issue, you can see a workaround to continue using older versions as part of your Helm charts:

$ helm repo add bitnami-pre-2022 https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
"bitnami-pre-2022" has been added to your repositories

or

-  repository: https://charts.bitnami.com/bitnami
+  repository: https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami

Being said that, the recommendation is to use recent versions of the Helm charts in order to keep the deployments secure and up-to-date.

Answer 60 · 2022-06-02T15:20:34.000Z

Any ideas on how to retrieve helm charts older than the recent 6 months?
helm repo remove bitnami
helm repo add bitnami  https://raw.githubusercontent.com/bitnami/charts/index/bitnami

Please note this can solve some of the previously mentioned CloudFront issues but this is the latest version of the index.yaml that only contains the latest 6 months. In order to list previous versions you need to pin a specific commit, for example:

$ helm repo add bitnami-pre-2022 https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
"bitnami-pre-2022" has been added to your repositories

Replacing this pre-2022 snapshot with the regular one is not recommended since you won't receive new version updates.

Answer 61 · 2022-06-02T15:27:53.000Z

@carrodher This broke our internal builds. Overnight changing this without sufficient notice would have caused production outage for many companies. You should have implemented the fix in other way without causing the breakage to existing customers. Give a 1 month notice period, Let the users who want performance benefit use a specific commit with that slim index file.

Answer 62 · 2022-06-02T16:01:03.000Z

Being said that, the recommendation is to use recent versions of the Helm charts in order to keep the deployments secure and up-to-date.

If the deployment interface (values.yaml, etc.) hasn't changed, what's the motivation for changing the chart version instead of just bumping the image tag of the software being installed? If nothing about a chart changed for 6 months, will it just be dropped off the index even though it's still the latest?

Answer 63 · 2022-06-02T20:51:07.000Z

i realize that this is a challenging problem + bitnami is offering a dope service here (mostly for free) so i'm very appreciative of all the effort

...that said, i just hit this pull error once again, even after the pruning of old chart versions 😭

    Exception: invoke of kubernetes:helm:template failed: invocation of kubernetes:helm:template returned an error: failed to generate YAML for specified Helm chart: failed to pull chart: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: stream error: stream ID 1; INTERNAL_ERROR; received from peer

Answer 64 · 2022-06-03T08:16:17.000Z

i cannot pull any chart from bitnami it keep giving me this error:
Save error occurred: could not find : no matching version
Error: could not find : no matching version

Answer 65 · 2022-06-03T13:00:38.000Z

The charts we are using didn't make the 6 month cut off period which immediately broke our ci/cd and prevents update to production systems. As a quick work around we had to move back to the old repo. If these charts are expected to be used in production I don't think you can expect 6 months to be a sufficient time. Product dependencies are not updated at that kind of frequency unless there is a something not working or a serious security issue.

Answer 66 · 2022-06-03T19:40:11.000Z

The Helm charts were not deleted but the index.yaml was updated to keep Helm charts from the latest 6 months. Please, take a look a this pinned issue #10539 to have more info about this change and some ways to use previous versions as usual by adding a previous index.yaml version to the helm client.

The index.yaml is stored in this repository under the index branch, users should be able to use any commit in that branch to add a previous version of the index.yaml.

Manually using helm repo add

$ helm repo add bitnami-pre-2022 https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
"bitnami-pre-2022" has been added to your repositories

When used as a dependency in Chart.yaml:

  - name: postgresql
    version: 8.1.0
-   repository: https://charts.bitnami.com/bitnami
+   repository: https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
    condition: postgresql.enabled

Sine the original issue reported here should be solved and in order to consolidate the conversations and unify the effort, we will answer questions as well as any other communication in the above mentioned GH issue, closing the other ones.

Answer 67 · 2022-08-24T16:34:31.000Z

$ helm repo add bitnami https://charts.bitnami.com/bitnami
Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: Get "https://charts.bitnami.com/bitnami/index.yaml": EOF

that was just now. it doesn't seem like this issue is resolved.

Answer 68 · 2022-11-02T15:21:49.000Z

$ helm repo add bitnami https://charts.bitnami.com/bitnami
Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: Get "https://charts.bitnami.com/bitnami/index.yaml": dial tcp: lookup charts.bitnami.com on 8.8.4.4:53: no such host

This issue is still happening

Answer 69 · 2022-11-02T16:48:57.000Z

It's fixed: #13313

Answer 70 · 2022-12-20T15:34:23.000Z

Still getting certificate related issues:

Answer 71 · 2022-12-20T15:37:02.000Z

@tal-ayalon You are missing bitnami at the end of the url. It should be:

helm repo add bitnami https://charts.bitnami.com/bitnami

Answer 72 · 2022-12-21T10:20:47.000Z

@basaveswar-kureti Still getting the error:

If I'm using the --insecure-skip-tls-verify flag than it works

Answer 73 · 2022-12-21T11:01:36.000Z

@tal-ayalon this error appears to be macos-specific and not directly related to bitnami:

golang/go#51991

Answer 74 · 2022-12-21T20:45:57.000Z

Works on my machine. 😛 That's not super helpful though.

$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories
$ helm version
version.BuildInfo{Version:"v3.7.0", GitCommit:"eeac83883cb4014fe60267ec6373570374ce770b", GitTreeState:"clean", GoVersion:"go1.17"}
$ sw_vers
ProductName:		macOS
ProductVersion:		13.1
BuildVersion:		22C65

Answer 75 · 2023-01-11T11:25:00.000Z

I'm seeing this error at least 5 time a day in argocd-repo-server:

level=error msg="`helm dependency build` failed exit status 1: Error: error loading /helm-working-dir/repository/https:--charts.bitnami.com-bitnami-index.yaml: empty index.yaml file\nError: plugin \"secrets\

Answer 76 · 2023-09-14T20:10:30.000Z

helm repo add bitnami https://charts.bitnami.com/bitnami
Error: looks like "https://charts.bitnami.com/bitnami" is not a valid chart repository or cannot be reached: Get "https://charts.bitnami.com/bitnami/index.yaml"

Answer 77 · 2023-09-15T07:27:23.000Z

Hi,

We changed the bitnami/charts helm repository to use OCI. Check the new instructions here https://github.com/bitnami/charts/

Answer 78 · 2023-10-14T13:48:35.000Z

Hello friends, I have a little issue I am facing using Helm charts;
I am using the helm dependency update [folder_name] command to update a chart, the update was successful and all but helm is experiencing issues saving the chart.

edit: I used this same chart repository few minutes ago and it worked.

Getting updates for unmanaged Helm repositories...
...Unable to get an update from the "https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami" chart repository:
	context deadline exceeded (Client.Timeout or context cancellation while reading body)
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Save error occurred:  could not find : chart mariadb not found in https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami: looks like "https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami" is not a valid chart repository or cannot be reached: context deadline exceeded (Client.Timeout or context cancellation while reading body)
Error: could not find : chart mariadb not found in https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami: looks like "https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami" is not a valid chart repository or cannot be reached: context deadline exceeded (Client.Timeout or context cancellation while reading body)

Answer 79 · 2023-10-14T14:36:00.000Z

I retried the command, after deleting the Charts folder, and it ran successfully.

Getting updates for unmanaged Helm repositories...
...Successfully got an update from the "https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami" chart repository
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "bitnami" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading mariadb from repo https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
Deleting outdated charts

Hello friends, I have a little issue I am facing using Helm charts; I am using the helm dependency update [folder_name] command to update a chart, the update was successful and all but helm is experiencing issues saving the chart.

edit: I used this same chart repository few minutes ago and it worked.

Getting updates for unmanaged Helm repositories... ...Unable to get an update from the "https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami" chart repository: context deadline exceeded (Client.Timeout or context cancellation while reading body) Hang tight while we grab the latest from your chart repositories... ...Successfully got an update from the "bitnami" chart repository Update Complete. ⎈Happy Helming!⎈ Saving 1 charts Save error occurred: could not find : chart mariadb not found in https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami: looks like "https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami" is not a valid chart repository or cannot be reached: context deadline exceeded (Client.Timeout or context cancellation while reading body) Error: could not find : chart mariadb not found in https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami: looks like "https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami" is not a valid chart repository or cannot be reached: context deadline exceeded (Client.Timeout or context cancellation while reading body)