Error from server (NotFound): pods "nsenter-qylbtm" not found
dshakey opened this issue · 22 comments
new install on a clean AKS cluster
/usr/local/bin/kubectl-node_shell aks-commpool-35862059-vmss000000
spawning "nsenter-qylbtm" on "aks-commpool-35862059-vmss000000"
error: Invalid JSON Patch
Error from server (NotFound): pods "nsenter-qylbtm" not found
yes have the latest version, error still the same. even running help get same error
bash-3.2$ kubectl node-shell --help
spawning "nsenter-ih3lqx" on "--help"
error: Invalid JSON Patch
Error from server (NotFound): pods "nsenter-ih3lqx" not found
++ kubectl version --client -o yaml
++ awk '-F[ :"]+' '$2 == "minor" {print $3+0}'
+ m=21
+ '[' 21 -lt 18 ']'
+ trap 'EC=$?; kubectl --context=aks-u2-prod-cu-01-admin --namespace= delete pod --wait=false nsenter-264z5u >&2 || true; exit $EC' EXIT INT TERM
+ echo 'spawning "nsenter-264z5u" on "aks-default-22362725-vmss000000"'
spawning "nsenter-264z5u" on "aks-default-22362725-vmss000000"
++ '[' -t 0 ']'
++ echo -t
+ kubectl --context=aks-u2-prod-cu-01-admin --namespace= run --image docker.io/library/alpine --restart=Never '--overrides={
"spec": {
"nodeName": "aks-default-22362725-vmss000000",
"hostPID": true,
"hostNetwork": true,
"containers": [
{
"securityContext": {
"privileged": true
},
"image": "docker.io/library/alpine",
"name": "nsenter",
"stdin": true,
"stdinOnce": true,
"tty": true,
"command": ["nsenter\\", \\"--target\\", \\"1\\", \\"--mount\\", \\"--uts\\", \\"--ipc\\", \\"--net\\", \\"--pid\\", \\"--\\", \\"bash\\", \\"-l\\", \\""]
}
],
"tolerations": [
{
"key": "CriticalAddonsOnly",
"operator": "Exists"
},
{
"effect": "NoExecute",
"operator": "Exists"
}
]
}
}' -t -i nsenter-264z5u
error: Invalid JSON Patch
+ EC=1
+ kubectl --context=aks-u2-prod-cu-01-admin --namespace= delete pod --wait=false nsenter-264z5u
Error from server (NotFound): pods "nsenter-264z5u" not found
+ true
+ exit 1
if i remove the escape characters and add in the shell it works for me
"command": ["/bin/sh", "-c", "nsenter", "--target", "1", "--mount", "--uts", "--ipc", "--net", "--pid", "--", "bash", "-l", " "]
why the need for escape chars?
yes have the latest version, error still the same. even running help get same error
We don't provide any information about the plugin version inside the code, that's not good and I have to correct it.
why the need for escape chars?
I guess there is some imperfection in this regexp
kubectl-node-shell/kubectl-node_shell
Lines 80 to 82 in 6da560b
Sorry, can't reproduce. What bash / sed versions are you using? And is your sed GNU sed?
I have GNU sed v4.7 and bash 5.0.17 on Ubuntu 20.04. Dunno where the difference lies, maybe in your bash version?
@dshakey what OS do you use, could you provide output of:
bash --version
sed --version
my output
$ bash --version GNU bash, version 5.1.8(1)-release (x86_64-pc-linux-gnu) Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or laterThis is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.$ sed --version
sed (GNU sed) 4.8
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.Written by Jay Fenlason, Tom Lord, Ken Pizzini,
Paolo Bonzini, Jim Meyering, and Assaf Gordon.This sed program was built without SELinux support.
GNU sed home page: https://www.gnu.org/software/sed/.
General help using GNU software: https://www.gnu.org/gethelp/.
E-mail bug reports to: bug-sed@gnu.org.
And I can't reproduce as well
Getting this same issue on a bone stock macbook, switching to gnu-sed instead of the mac provided one fixes it.
That's what I was talking about, the new dependencies are adding new problems.
Could someone test kubectl-node_shell from #31, please?
Okay now we have two concurrent PRs #31 and #32.
Good! Let's find it out which one is better 🙂
@joesuf4 I would also be glad if you could review my changes as well and say objectively if they are better or worse than yours.
I still trying to preserve the compatibility, and believe that pure shell will be better to support wide range of systems.
New shell is also causing issues for me. Please don't try to fix what ain't broke. This is a very useful plugin that I use very regularly, and I'd like to avoid tracking down breaking changes. For anyone else running into this and installed via krew, you can downgrade via: kc krew install --manifest-url https://raw.githubusercontent.com/kubernetes-sigs/krew-index/12c74b3b813797bc6cdc36603b671b129a74cc38/plugins/node-shell.yaml to revert this commit: https://github.com/kubernetes-sigs/krew-index/pull/1332/files until the bikeshedding is done. @kvaps thanks for this plugin!
@kvaps I'm still having issues with node-shell version v1.5.0, fyi:
> kc node-shell ip-10-23-60-23.us-east-2.compute.internal
spawning "nsenter-1vijo9" on "ip-10-23-60-23.us-east-2.compute.internal"
error: Invalid JSON Patch
Error from server (NotFound): pods "nsenter-1vijo9" not found
> kc krew info node-shell
NAME: node-shell
INDEX: default
URI: https://github.com/kvaps/kubectl-node-shell/archive/v1.5.0.tar.gz
SHA256: ad0def95b4b581aafc646f76d03f4ff717e33a708bcd75905a6c99c373846ac3
VERSION: v1.5.0
HOMEPAGE: https://github.com/kvaps/kubectl-node-shell
No issues with version v1.3.0 😅
@ybstaked please provide me debug output:
sh -x kubectl-node_shell node-shell ip-10-23-60-23.us-east-2.compute.internal
and your versions information:
sh --version
awk --version
@kvaps here you go:
> kc node-shell --version
kubectl-node-shell 1.5.0
> sh -x kubectl-node_shell ip-10-23-60-23.us-east-2.compute.internal
+ set -e
+ kubectl=kubectl
+ version=1.5.0
+ generator=
+ node=
+ nodefaultctx=0
+ nodefaultns=0
+ cmd='[ "nsenter", "--target", "1", "--mount", "--uts", "--ipc", "--net", "--pid", "--"'
+ '[' -p /dev/stdin ']'
+ '[' -p /dev/stdout ']'
+ tty=true
+ '[' 1 -gt 0 ']'
+ key=ip-10-23-60-23.us-east-2.compute.internal
+ case $key in
+ '[' -z '' ']'
+ node=ip-10-23-60-23.us-east-2.compute.internal
+ shift
+ '[' 0 -gt 0 ']'
+ '[' 0 = 1 ']'
++ kubectl config current-context
+ kubectl='kubectl --context=stg-e2'
+ '[' 0 = 1 ']'
++ kubectl --context=stg-e2 config view --minify --output 'jsonpath={.contexts..namespace}'
+ kubectl='kubectl --context=stg-e2 --namespace=celer'
+ '[' 0 -gt 0 ']'
+ cmd='[ "nsenter", "--target", "1", "--mount", "--uts", "--ipc", "--net", "--pid", "--""bash", "-l" ]'
+ '[' -z ip-10-23-60-23.us-east-2.compute.internal ']'
+ image=docker.io/library/alpine
++ env LC_ALL=C tr -dc a-z0-9
++ head -c 6
+ pod=nsenter-fdrct5
+ kubectl --context=stg-e2 --namespace=celer get node ip-10-23-60-23.us-east-2.compute.internal
++ cat
+ overrides='{
"spec": {
"nodeName": "ip-10-23-60-23.us-east-2.compute.internal",
"hostPID": true,
"hostNetwork": true,
"containers": [
{
"securityContext": {
"privileged": true
},
"image": "docker.io/library/alpine",
"name": "nsenter",
"stdin": true,
"stdinOnce": true,
"tty": true,
"command": [ "nsenter", "--target", "1", "--mount", "--uts", "--ipc", "--net", "--pid", "--""bash", "-l" ]
}
],
"tolerations": [
{
"key": "CriticalAddonsOnly",
"operator": "Exists"
},
{
"effect": "NoExecute",
"operator": "Exists"
}
]
}
}'
++ kubectl version --client -o yaml
++ awk '-F[ :"]+' '$2 == "minor" {print $3+0}'
+ m=17
+ '[' 17 -lt 18 ']'
+ generator=--generator=run-pod/v1
+ trap 'EC=$?; kubectl --context=stg-e2 --namespace=celer delete pod --wait=false nsenter-fdrct5 >&2 || true; exit $EC' EXIT INT TERM
+ echo 'spawning "nsenter-fdrct5" on "ip-10-23-60-23.us-east-2.compute.internal"'
spawning "nsenter-fdrct5" on "ip-10-23-60-23.us-east-2.compute.internal"
++ '[' true = true ']'
++ echo -t
+ kubectl --context=stg-e2 --namespace=celer run --image docker.io/library/alpine --restart=Never '--overrides={
"spec": {
"nodeName": "ip-10-23-60-23.us-east-2.compute.internal",
"hostPID": true,
"hostNetwork": true,
"containers": [
{
"securityContext": {
"privileged": true
},
"image": "docker.io/library/alpine",
"name": "nsenter",
"stdin": true,
"stdinOnce": true,
"tty": true,
"command": [ "nsenter", "--target", "1", "--mount", "--uts", "--ipc", "--net", "--pid", "--""bash", "-l" ]
}
],
"tolerations": [
{
"key": "CriticalAddonsOnly",
"operator": "Exists"
},
{
"effect": "NoExecute",
"operator": "Exists"
}
]
}
}' -t -i nsenter-fdrct5 --generator=run-pod/v1
error: Invalid JSON Patch
+ EC=1
+ kubectl --context=stg-e2 --namespace=celer delete pod --wait=false nsenter-fdrct5
Error from server (NotFound): pods "nsenter-fdrct5" not found
+ true
+ exit 1
> sh --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin20)
Copyright (C) 2007 Free Software Foundation, Inc.
> awk --version
awk version 20200816
p.s. please do keep the dependencies as minimal as possible 😃
ah got it, there is just a stupid typo, one mmnt
fixed in v1.5.1, thank you for timely report
Works well once again! Thank you for your timely fix :)