Unable to spin shell on a node with taints
amnk opened this issue · 6 comments
amnk commented
First of all - thank you very much for the project. It is elegant and useful!
It saved my day today, but only partially - because it does not support nodes with taints. Would be great to see it being able to spin shells on any type of node.
kvaps commented
Hi, are you sure that this is not working with taints?
When you creating pod with the spec.nodeName specified it will always be run on node regardless of it's taints.
amnk commented
@kvaps I have two nodepools at the moment. Tool works perfectly fine on a nodepool without taints, and fails on a pool with taints (at least that is the only different that I can spot). It fails like this:
❯ k node-shell xxxxx
spawning "nsenter-p4kc5e" on "xxxx"
No resources found
Error from server (NotFound): pods "nsenter-p4kc5e" not found
If I can provide any other info - let me know.
kvaps commented
OK, please provide me your node yaml:
k get node <node_name> -o yaml
and
k version
I'll try to reproduce this
amnk commented
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-18T02:59:13Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
and
apiVersion: v1
kind: Node
metadata:
annotations:
alpha.kubernetes.io/provided-node-ip: 10.136.206.175
csi.volume.kubernetes.io/nodeid: '{"dobs.csi.digitalocean.com":"201815603"}'
io.cilium.network.ipv4-cilium-host: 10.244.4.161
io.cilium.network.ipv4-health-ip: 10.244.4.194
io.cilium.network.ipv4-pod-cidr: 10.244.4.128/25
node.alpha.kubernetes.io/ttl: "0"
volumes.kubernetes.io/controller-managed-attach-detach: "true"
creationTimestamp: "2020-07-28T20:59:46Z"
labels:
beta.kubernetes.io/arch: amd64
beta.kubernetes.io/instance-type: s-4vcpu-8gb
beta.kubernetes.io/os: linux
cluster: python-apps
doks.digitalocean.com/node-id: a8fe8fea-e6c6-4cd8-a35c-15f836448016
doks.digitalocean.com/node-pool: python-apps
doks.digitalocean.com/node-pool-id: 698d5eca-8ecd-4538-8167-bfc193711765
doks.digitalocean.com/version: 1.18.6-do.0
failure-domain.beta.kubernetes.io/region: nyc1
kubernetes.io/arch: amd64
kubernetes.io/hostname: python-apps-3re6j
kubernetes.io/os: linux
node.kubernetes.io/instance-type: s-4vcpu-8gb
region: nyc1
topology.kubernetes.io/region: nyc1
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:labels:
f:beta.kubernetes.io/instance-type: {}
f:failure-domain.beta.kubernetes.io/region: {}
f:node.kubernetes.io/instance-type: {}
f:topology.kubernetes.io/region: {}
f:status:
f:addresses:
k:{"type":"ExternalIP"}:
.: {}
f:address: {}
f:type: {}
manager: digitalocean-cloud-controller-manager
operation: Update
time: "2020-07-28T20:59:48Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:node.alpha.kubernetes.io/ttl: {}
f:spec:
f:podCIDR: {}
f:podCIDRs:
.: {}
v:"10.244.4.128/25": {}
manager: kube-controller-manager
operation: Update
time: "2020-07-28T20:59:56Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
f:io.cilium.network.ipv4-cilium-host: {}
f:io.cilium.network.ipv4-health-ip: {}
f:io.cilium.network.ipv4-pod-cidr: {}
f:status:
f:conditions:
k:{"type":"NetworkUnavailable"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
manager: cilium-agent
operation: Update
time: "2020-07-28T21:00:04Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:labels:
f:cluster: {}
f:spec:
f:taints: {}
manager: kubectl
operation: Update
time: "2020-07-28T21:06:05Z"
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:metadata:
f:annotations:
.: {}
f:alpha.kubernetes.io/provided-node-ip: {}
f:csi.volume.kubernetes.io/nodeid: {}
f:volumes.kubernetes.io/controller-managed-attach-detach: {}
f:labels:
.: {}
f:beta.kubernetes.io/arch: {}
f:beta.kubernetes.io/os: {}
f:doks.digitalocean.com/node-id: {}
f:doks.digitalocean.com/node-pool: {}
f:doks.digitalocean.com/node-pool-id: {}
f:doks.digitalocean.com/version: {}
f:kubernetes.io/arch: {}
f:kubernetes.io/hostname: {}
f:kubernetes.io/os: {}
f:region: {}
f:spec:
f:providerID: {}
f:status:
f:addresses:
.: {}
k:{"type":"Hostname"}:
.: {}
f:address: {}
f:type: {}
k:{"type":"InternalIP"}:
.: {}
f:address: {}
f:type: {}
f:allocatable:
.: {}
f:cpu: {}
f:ephemeral-storage: {}
f:hugepages-2Mi: {}
f:memory: {}
f:pods: {}
f:capacity:
.: {}
f:cpu: {}
f:ephemeral-storage: {}
f:hugepages-2Mi: {}
f:memory: {}
f:pods: {}
f:conditions:
.: {}
k:{"type":"DiskPressure"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"MemoryPressure"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"PIDPressure"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Ready"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
f:daemonEndpoints:
f:kubeletEndpoint:
.: {}
k:{"type":"Hostname"}:
.: {}
f:address: {}
f:type: {}
k:{"type":"InternalIP"}:
.: {}
f:address: {}
f:type: {}
f:allocatable:
.: {}
f:cpu: {}
f:ephemeral-storage: {}
f:hugepages-2Mi: {}
f:memory: {}
f:pods: {}
f:capacity:
.: {}
f:cpu: {}
f:ephemeral-storage: {}
f:hugepages-2Mi: {}
f:memory: {}
f:pods: {}
f:conditions:
.: {}
k:{"type":"DiskPressure"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"MemoryPressure"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"PIDPressure"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
k:{"type":"Ready"}:
.: {}
f:lastHeartbeatTime: {}
f:lastTransitionTime: {}
f:message: {}
f:reason: {}
f:status: {}
f:type: {}
f:daemonEndpoints:
f:kubeletEndpoint:
f:Port: {}
f:images: {}
f:nodeInfo:
f:architecture: {}
f:bootID: {}
f:containerRuntimeVersion: {}
f:kernelVersion: {}
f:kubeProxyVersion: {}
f:kubeletVersion: {}
f:machineID: {}
f:operatingSystem: {}
f:osImage: {}
f:systemUUID: {}
manager: kubelet
operation: Update
time: "2020-07-29T19:24:06Z"
name: python-apps-3re6j
resourceVersion: "2462715"
selfLink: /api/v1/nodes/python-apps-3re6j
uid: 6a72b4b6-38d7-4a27-bc32-220044c8004e
spec:
podCIDR: 10.244.4.128/25
podCIDRs:
- 10.244.4.128/25
providerID: digitalocean://201815603
taints:
- effect: NoSchedule
key: node-type
value: python-apps
- effect: NoExecute
key: node-type
value: python-apps
status:
addresses:
- address: python-apps-3re6j
type: Hostname
- address: 10.136.206.175
type: InternalIP
- address: 104.248.120.49
type: ExternalIP
allocatable:
cpu: "4"
ephemeral-storage: "152161143761"
hugepages-2Mi: "0"
memory: 6694Mi
pods: "110"
capacity:
cpu: "4"
ephemeral-storage: 165105408Ki
hugepages-2Mi: "0"
memory: 8170048Ki
pods: "110"
conditions:
- lastHeartbeatTime: "2020-07-28T21:00:04Z"
lastTransitionTime: "2020-07-28T21:00:04Z"
message: Cilium is running on this node
reason: CiliumIsUp
status: "False"
type: NetworkUnavailable
- lastHeartbeatTime: "2020-07-29T19:24:06Z"
lastTransitionTime: "2020-07-28T20:59:46Z"
message: kubelet has sufficient memory available
reason: KubeletHasSufficientMemory
status: "False"
type: MemoryPressure
- lastHeartbeatTime: "2020-07-29T19:24:06Z"
lastTransitionTime: "2020-07-28T20:59:46Z"
message: kubelet has no disk pressure
reason: KubeletHasNoDiskPressure
status: "False"
type: DiskPressure
- lastHeartbeatTime: "2020-07-29T19:24:06Z"
lastTransitionTime: "2020-07-28T20:59:46Z"
message: kubelet has sufficient PID available
reason: KubeletHasSufficientPID
status: "False"
type: PIDPressure
- lastHeartbeatTime: "2020-07-29T19:24:06Z"
lastTransitionTime: "2020-07-28T20:59:56Z"
message: kubelet is posting ready status
reason: KubeletReady
status: "True"
type: Ready
daemonEndpoints:
kubeletEndpoint:
Port: 10250
nodeInfo:
architecture: amd64
bootID: 0df05998-4b67-490a-936e-961ec8156310
containerRuntimeVersion: docker://18.9.2
kernelVersion: 4.19.0-0.bpo.6-amd64
kubeProxyVersion: v1.18.6
kubeletVersion: v1.18.6
machineID: fa1b1d9578c74b8583d7a73feb857047
operatingSystem: linux
osImage: Debian GNU/Linux 10 (buster)
systemUUID: fa1b1d95-78c7-4b85-83d7-a73feb857047
node is a droplet on DO. I removed .images[] block
cccs-cat001 commented
Looks like this PR fixes it: #13