GoogleCloudPlatform/gcs-fuse-csi-driver

wrong architecture in v1.5.0 image

Closed this issue · 12 comments

nikolaydubina commented
  • GKE cluster is running on linux/amd64
  • other images are linux/amd64 (e.g. v1.4.3)

however, v1.5.0 image is build for linux/arm64. which can be verified in Docker

image

this causes problems on startup of container, with error like this

image

please fix. thx.

hime commented

Hi @nikolaydubina, could you provide more details for replicating this issue? Could you share more details about your setup and workload deployment?

Some other helpful information to know would be:

  • Are you using non-managed driver?
  • Are you using a privately hosted sidecar image?
nikolaydubina commented

Are you using non-managed driver?

I am following https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/cloud-storage-fuse-csi-driver#private-image

I am running in private nodes, and have to use private image.

I guess this is "non-managed" driver.

Are you using a privately hosted sidecar image?

yes

hime commented

Thanks! Could you confirm the image you're using and how you got it? Do you have the image SHA that's placed in your workload yaml? The links in the description point to the same place.

hime commented

That's odd. I'll try to recreate this week... I see the manifests link to the right sha's too.

I'm curious if there's an external factor at play, could you share the cluster ID with me? You can get the ID by running gcloud container clusters describe <cluster-name> --location <cluster-location> | grep id:, and share the id with me? Thanks!

  • 👍1
nikolaydubina commented

hey, sorry will have to back off here. can't share cluster id (other ids from my GCP) for security reasons.

I can share details of my cluster / nodes tho. hope that helps:

1.30.5-gke.1014001
asia-southeast
IPv4
Cloud Storage Fuse CSI driver: Enabled
Container-Optimized OS with containerd (cos_containerd)

as for sidecar spec, I am closely following: https://github.com/GoogleCloudPlatform/gcs-fuse-csi-driver/blob/main/docs/sidecar-manual-injection.md

as for the rest, I think this should be possible to repro in brand new cluster.

hime commented

Could you share the exact steps you are doing to:

  1. Copy the image from public artifact registry to your local machine
  2. Steps you're doing to prepare the image locally (if any, i don't think there should be any).
  3. Push image to your private registry

Our original image manifests have not changed. For example, the v1.5 image manifest still handles both arm and amd.

nikolaydubina commented 
docker pull --platform=linux/amd64 gcr.io/gke-release/gcs-fuse-csi-driver-sidecar-mounter@sha256:a527a083127fb456c96a6e4a478639222065dc0c2d485729e63605035d624f8f

now produces amd64

image

maybe I did not use --platform flag before? but I used same command to pull all images last time, how come other images were pulled correctly and this was not? anyways, odd!

hime commented

ARG TARGETPLATFORM can be set externally and can affect the selection of the gcsfuse binary architecture. I'm not entirely sure how this can happen when pulling the image locally or pushing it to private registry.

I'm not familiar with the interface that you're using to view the image architecture. Could you share what that is? If you could share the steps you took to get there would be helpful too (to document it)!!

  • 👍1
nikolaydubina commented

right. this is layers for image as seen in Docker desktop

image
nikolaydubina commented

thanks for investigating this @hime.

probably good idea to have test that hits image registry for all platforms you expect. 🤔

hime commented

Created item to track documentation guide improvements. Closing as issue is now resolved.