banzaicloud/koperator

Koperator panics when CruiseControl responds with a progress/async response

amuraru opened this issue · 0 comments

Describe the bug

CruiseControl has webserver.request.maxBlockTimeMs configuration that controls the threshold above which a request is converted to async and a handler returned.
koperator is not able to handle this and panics:

{"level":"info","ts":"2022-06-29T05:24:46.382Z","logger":"controller.CruiseControl","msg":"got response for request","reconciler group":"kafka.banzaicloud.io","reconciler kind":"KafkaCluster","name":"pipeline-kafka","namespace":"ns-team-aep-pipeline-kafka-1-int","url":"http://pipeline-kafka-cruisecontrol-svc.ns-team-aep-pipeline-kafka-1-int.svc.cluster.local:8090/kafkacruisecontrol/state?json=true&substates=ANALYZER%2CANOMALY_DETECTOR%2CEXECUTOR%2CMONITOR&verbose=true","status":202}
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x2e8 pc=0x16dc126]

goroutine 1368 [running]:
github.com/banzaicloud/koperator/pkg/scale.(*cruiseControlScaler).Status(0xc002a49bf0)
	/workspace/pkg/scale/scale.go:75 +0x186
github.com/banzaicloud/koperator/controllers.(*CruiseControlTaskReconciler).Reconcile(0xc000304b58, {0x2143e20, 0xc002a48c60}, {{{0xc00068f940?, 0x1d65200?}, {0xc000973ac0?, 0x30?}}})
	/workspace/controllers/cruisecontroltask_controller.go:91 +0x28a
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0xc00056a000, {0x2143e20, 0xc002a48ba0}, {{{0xc00068f940?, 0x1d65200?}, {0xc000973ac0?, 0x4041f4?}}})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:114 +0x27e
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00056a000, {0x2143d78, 0xc000b91800}, {0x1c15100?, 0xc000ab6040?})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:311 +0x349
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00056a000, {0x2143d78, 0xc000b91800})
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:266 +0x1d9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:227 +0x85
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
	/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.0/pkg/internal/controller/controller.go:223 +0x31c

Steps to reproduce the issue:

Hard to reproduce but happens when CruiseControl is loaded (slow)

Expected behavior

koperator should gracefully handle this case.

Screenshots
N/A