Timeout error on using B2K
DavidAtBede opened this issue · 13 comments
Describe the bug
I am not able to use B2K because a timeout exception occurs.
4 other developers in our organization have the same issue. Though two do not.
Mention the platform you are using
Windows 10 Pro , WSL2, Microsoft Visual Studio Enterprise 2022 (64-bit) - Current Version 17.6.4
Azure/AKS
To Reproduce
Steps to reproduce the behavior:
- Deploy our large complex microservice software to aks cluster :)
- Use Visual Studio to debug on of the services, launch it under the vs debugger with B2K (see screenshots below)
- Chose a cluster
- Wait for some minutes - observe error message (See screenshots).
Expected behavior
Not to see the timeout exception and to be able to debug :)
Screenshots
If applicable, add screenshots to help explain your problem.
Logs:
bridge-mindarocli-2023-09-22-09-58-12-33048.txt
Desktop (please complete the following information):
- OS: Windows 10 Pro
- Browser - not relevant
- Version 19044.3086
Additional context
Add any other context about the problem here.
I have also run the Kubernetes-Client lib (that B2K is using) under the debugger and believe I can recreate the timeout in that scenario. Basically this call process.WaitForExit((int)(ExecTimeout.TotalMilliseconds) hangs:
Inspecting that process I see that it's running KubeLogin
FileName="kubelogin", Arguments="get-token --environment AzurePublicCloud --server-id 6dae42f8-4368-4678-94ff-3960e28e3630 --client-id 80faf920-1908-4b52-b5ef-a8e7bedfc67a --tenant-id 1e68b3b0-f976-4cd2-8073-4f6d8a304537 --login devicecode", WorkingDirectory=""
when I actually do that on the command line (cut paste, edit a tiny bit) ...
it works fine.
I am aware there is at least one report similar which seemed to fizzle out inconclusively to this one however I believe I have provided more information, and that this issue is widespread in my org. This issue is important to me and my organization and we'll be very responsive to requests for more info etc. So we might actually get to a resolution :)
@DavidAtBede thanks for bringing this up and more details. Can you provide me bridge-library-* log's please? I can take a look and let you know how to fix this.Also using kubelogin with bridge is tricky needs some env variables to be set.
@hsubramanianaks I'm not totally sure what you mean when you say "bridge-library-* log's" - do you mean the diagnostic logs written by the Bridge-To-Kubernetes VS plugin? That seems a reasonable guess - but I already have provided one of those above "bridge-mindarocli-2023-09-22-09-58-12-33048.txt" B2K writes a small handful of log for each run - here are all of them for the run above in a zip.
bridge-mindarocli-2023-09-22-09-56-04-29548.zip
If I've miss-understood what you wanted do forgive me and ask again :)
@hsubramanianaks also - when you say "using kubelogin with bridge is tricky needs some env variables to be set." could you be explicit and say which variables and set to what?
@DavidAtBede Thanks for the logs. No problem, yes in the same directory where you found bridge-mindarocli-.txt logs there would be other files with name bridge-library-.txt if you don't find that is fine. This is similar like #309 and refer this link for workaround - #309 (comment). Basically you need to use kubectl proxy until k8s client for csharp fixes or handles this issue. Please let me know how it goes. Since more people are facing this issue, I can do couple of things
- Follow up with maintainer of k8s client for csharp to see if there are plans to fix this or enhance this experience.
- I will update readme/documentation and pin this issue in bridge's repo so that people who face this issue in future can easily try with this workaround.
Kubernetes client for csharp issue here
@hsubramanianaks SO I've tried your work around - it exhibits a different error ... which I suppose is hopeful :) See screenshot.
Also re you request for bridge-library-* logs ... these are they from the run associated with attempting your suggested workaround
bridge-library-2023-09-22-13-32-23-4868.zip
The new error is Invalid URI: The URI scheme is not valid
@DavidAtBede Thanks for trying out and the prompt response. I will check the logs and get back to you.
@DavidAtBede sorry I just noticed it has to be $env:KUBECTL_PROXY="http://127.0.0.1:8001", http protocol was missing. Thats why this error I guess, please try and let me know. Thanks.
@hsubramanianaks actually that appears to work! Which is great.
There may be other issues subsequent to this but the proxying work around seems to work now. \o/
@DavidAtBede Thanks for confirmation. what are the other issues, are those related to bridge or something else? Also, to support this feature bridge needs some changes along with Azure AAD and Kubernetes RBAC combination. I might play around with this and let you know if this can work without kubectl proxy.
Ah right now I just observed a timeout - but it's late on a Friday night here in England :) I think my connectivity issue is solved for now. Thank you so much for your help on this matter.
I might play around with this and let you know if this can work without kubectl proxy.
That would be great Hsubramanianaks. the work around is messy and kinda fiddly.
We're also investigating why some people here seem to have no issue - it seems that they have azure subscription owner role (which seems a bit crazy). I'll let you know if we bottom this aspect out.
@DavidAtBede FYIP - I tried this myself with AAD enabled AKS cluster with kubelogin (which is enabled by default for AAD enabled clusters). I didn't face any error or timeout; I was able to use b2k without any issues. Please see below. I think azure subscription owner role plays a vital role with this, since I had owner access to this AKS cluster, I didn't face any issues. I think people without owner access need to access this AKS cluster either via joining AD group which has access to these clusters or via service principals. More details here
kubeconfig:
@DavidAtBede any update on this ? Please let us know. Thanks.
@DavidAtBede Please feel free to reopen this issue or create a new issue if it happens again with all details/logs etc... Thank you.