Azure/Bridge-To-Kubernetes

Failed to pull image "bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4"

mobius7777 opened this issue ยท 27 comments

I've been struggling for hours to get Bridge To Kubernetes to work in VS2022 for a kubernetes project. It had been working perfectly fine, but earlier today, running the service I was debugging via B2K simply failed to load the service. According to k9s, the service is bouncing between "ErrImagePull" and "ImagePullBackOff".

Checking kubectl describe pod for the service reveals the following error:

Failed to pull image "bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4": Error response from daemon: manifest for bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4 not found: manifest unknown: manifest tagged by "1.3.4" is not found

If I docker pull this myself, I get the same error.
docker pull bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4
Error response from daemon: manifest for bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4 not found: manifest unknown: manifest tagged by "1.3.4" is not found

It also happens if I try to get by latest tag.

I have no idea what to do.

Have you solved your problem? I have the same problem

@xuexiaoxuan Sadly, I have not. It continues to fail for me the same way. I am basically stuck. I have no idea how or why this suddenly started, but the problem started late Friday afternoon.

I guess it's good to know I'm not the only one having the issue. We can commiserate together...

Maybe someone will have an answer on Monday.

Just tried to use b2k this morning and I am also getting this issue :(

I am getting the same issue if anyone finds a fix please share it

I'm having the same issue

Looks like the issue is caused by this PR: #602
_"Old ACR will be deleted and will be announced to users that they need to update to latest version before releasing this to marketplace."

Why would they remove the old ARC before updating and releasing a new version? I can't make B2K pull the container from somewhere else, can I? I don't even know where the new ACR is.

With vscode you can point it at other ACRs using these 3 environment variables, i am not sure if they will work in VS2022
BRIDGE_DEVHOSTIMAGENAME
BRIDGE_RESTORATIONJOBIMAGENAME
BRIDGE_ROUTINGMANAGERIMAGENAME

However it appears we can't point it at the new repo yet anyway.

Error response from daemon: Head "https://bridgetok8s.azurecr.io/v2/lpkremoteagent/manifests/1.3.4": unauthorized: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required, visit https://aka.ms/acr/authorization for more information."}]}

So we are just out of luck and dead in the water for now?

Issue is affecting me too... any chance of looking at this @hsubramanianaks?

Acknowledged, I will look into this and get back soon. Thanks for your patience.

@pkoelbl can you try now with the new repo? I have fixed the unauthorized issue. ex: docker pull bridgetok8s.azurecr.io/lpkremoteagent:1.3.4 works for me.

Hi All,
I am also getting the same,If anyone solved, please do describe here.

"Failed to pull image "bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4": Error response from daemon: manifest for bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4 not found: manifest unknown: manifest tagged by "1.3.4" is not found"

@hsubramanianaks - Adding the 3 env variables pointing to the new ACR seems to work (it downloads the images now), but I do notice there seems to be a problem with the actual images

You must install or update .NET to run this application.

App: /app/restorationjob
Architecture: x64
Framework: 'Microsoft.NETCore.App', version '7.0.15' (x64)
.NET location: /usr/share/dotnet

The following frameworks were found:
  8.0.1 at [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Learn more:
https://aka.ms/dotnet/app-launch-failed

To install missing framework, download:
https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=7.0.15&arch=x64&rid=linux-x64&os=mariner.2.0

fyi - these are the 3 I added (the image names are guesswork, so please correct if I'm wrong)

BRIDGE_DEVHOSTIMAGENAME = bridgetok8s.azurecr.io/lpkremoteagent:1.3.4
BRIDGE_RESTORATIONJOBIMAGENAME = bridgetok8s.azurecr.io/lpkrestorationjob:1.3.4
BRIDGE_ROUTINGMANAGERIMAGENAME = bridgetok8s.azurecr.io/routingmanager:stable

those above env vars (BRIDGE_DEVHOSTIMAGENAME BRIDGE_RESTORATIONJOBIMAGENAME BRIDGE_ROUTINGMANAGERIMAGENAME) did NOT work for me... got the same stuck connecting error message.

mee too... no fix... how can I set this env vars??

I had to restart VS before they took effect. In fact to be safe the first time I rebooted the PC (VS seemed to be a bit stuck).

I just set the env vars in the system environment variables control panel (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.core/about/about_environment_variables?view=powershell-7.4#set-environment-variables-in-the-system-control-panel).

@hsubramanianaks I was able to use the environment variables to get the images from bridgetok8s.azurecr.io, but I got the same error as @steverash . Can we please have the images back on bridgetokubernetes.azurecr.io as they were before so we can continue our jobs?

The images are deployed back to old ACR repositories (bridgetokubernetes.azurecr.io), can you all please retest and confirm here?

I have the same problem with the bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.2 image

Hi Everyone,
Step 1:
$env:BRIDGE_DEVHOSTIMAGENAME="bridgetok8s.azurecr.io/lpkremoteagent:1.3.4"
$env:BRIDGE_RESTORATIONJOBIMAGENAME="bridgetok8s.azurecr.io/lpkrestorationjob:1.3.4"
$env:BRIDGE_ROUTINGMANAGERIMAGENAME="bridgetok8s.azurecr.io/routingmanager:stable"

I have created Environmetal Variables and I have pulled all the 3 images

Now I can debug using BridgetoK8S.
Its perfectly works for me.
Thanks

Yes,it is back "bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4"
docker pull bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4
1.3.4: Pulling from lpkremoteagent
Digest: sha256:82c0039c90d4cdc6bdb8614d58be2e9ddb4a06ec8156c33031e5bb87dbec28bd
Status: Image is up to date for bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4
bridgetokubernetes.azurecr.io/lpkremoteagent:1.3.4
image

@hsubramanianaks I can debug again, thank you!
How is this going to work in the future? Obviously you are wanting to migrate to bridgetok8s.azurecr.io, so what can we expect? Will the images there be fixed, since they are giving the "You must install or update .NET to run this application" error?

@mobius7777 Thanks for testing it. this switch was not intended to be turned live yet, but some internal security issues caused the repos to be deleted which is reverted now. When we switch to new acr repo, users will not see any issues like this is our best hope. We will keep you posted on this. For now can we close this issue?

works now.. i did not even have to set env vars. Thanks

@hsubramanianaks I am not having any more issues since you reactivated the bridgetokubernetes.azurecr.io ARC. Thank you. I guess you can close the issue since that resolved it.

Closing this issue as it works for everyone. Please feel free to reopen if anyone face issues. Thank you all.