panic: runtime error: invalid memory address or nil pointer dereference
Anto74 opened this issue · 7 comments
Hi,
I've a CSI driver, implemented according to CSI standard. My node server includes liveness probe (v2.8.0) as sidecar. After normal execution for some hours I got (twice) the following:
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x857ceb]goroutine 130244 [running]:
google.golang.org/grpc.(*ClientConn).Close(0x0)
/home/esepadm/jenkins-exclusive-executor/workspace/SEPReleaseLivenessProbe/3pp/vendor/google.golang.org/grpc/clientconn.go:1017 +0x4b
main.acquireConnection.func1()
/home/esepadm/jenkins-exclusive-executor/workspace/SEPReleaseLivenessProbe/3pp/cmd/livenessprobe/main.go:103 +0x16a
created by main.acquireConnection
/home/esepadm/jenkins-exclusive-executor/workspace/SEPReleaseLivenessProbe/3pp/cmd/livenessprobe/main.go:97 +0x1a5
This causes node server daemonset pod restart.
Can you help?
Best regards,
Antonio Vitiello
The Kubernetes project currently lacks enough contributors to adequately respond to all issues.
This bot triages un-triaged issues according to the following rules:
- After 90d of inactivity,
lifecycle/staleis applied - After 30d of inactivity since
lifecycle/stalewas applied,lifecycle/rottenis applied - After 30d of inactivity since
lifecycle/rottenwas applied, the issue is closed
You can:
- Mark this issue as fresh with
/remove-lifecycle stale - Close this issue with
/close - Offer to help out with Issue Triage
Please send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle stale
/remove-lifecycle stale
Hi,
we're often getting this error in our environment and we would like to have some analysis on it.
Best regards,
Antonio Vitiello
Hi,
we moved to Liveness-Probe version 1.9 and we're getting more and more occurrence of this fault. Anyone that can give and suggestion how to deal with it?
{"message":"panic: runtime error: invalid memory address or nil pointer dereference","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.204+00:00","version":"1.2.0"}
{"message":"[signal SIGSEGV: segmentation violation code=0x1 addr=0x10 pc=0x858cab]","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.205+00:00","version":"1.2.0"}
{"message":"","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.205+00:00","version":"1.2.0"}
{"message":"goroutine 4045171 [running]:","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.205+00:00","version":"1.2.0"}
{"message":"google.golang.org/grpc.(*ClientConn).Close(0x0)","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.205+00:00","version":"1.2.0"}
{"message":"\t/home/esepadm/jenkins-exclusive-executor/workspace/SEPReleaseLivenessProbe/3pp/vendor/google.golang.org/grpc/clientconn.go:1016 +0x4b","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.206+00:00","version":"1.2.0"}
{"message":"main.acquireConnection.func1()","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.206+00:00","version":"1.2.0"}
{"message":"\t/home/esepadm/jenkins-exclusive-executor/workspace/SEPReleaseLivenessProbe/3pp/cmd/livenessprobe/main.go:103 +0x16a","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.206+00:00","version":"1.2.0"}
{"message":"created by main.acquireConnection","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.206+00:00","version":"1.2.0"}
{"message":"\t/home/esepadm/jenkins-exclusive-executor/workspace/SEPReleaseLivenessProbe/3pp/cmd/livenessprobe/main.go:97 +0x1a5","metadata":{"container_name":"liveness-probe","namespace":"eric-eea-ns","pod_name":"eric-cs-storage-encryption-provider-node-5k7p6"},"service_id":"eric-cs-storage-encryption-provider","severity":"info","timestamp":"2023-05-22T18:15:27.206+00:00","version":"1.2.0"}
Regards,
///Antonio
I can't reproduce the issue, however, I can see the livenessprobe tries to close a nil connection. I'm fixing it in #192. Anyway, the livenessprobe will only exit with a better error message instead of panic, it won't change much.
/assign