trento-project/trento

web pod crashing when receiving unexpected data

abravosuse opened this issue · 0 comments

A ramp-up customer has reported the agent is giving “connection refused to :8081” errors during discovery:

Jan 28 19:53:03 ua00118t trento[9984]: time="2022-01-28 19:53:03" level=info msg="Identifying if the VM is running in a cloud environment..."

Jan 28 19:53:03 ua00118t trento[9984]: time="2022-01-28 19:53:03" level=info msg="VM is running on azure"

Jan 28 19:53:03 ua00118t trento[9984]: time="2022-01-28 19:53:03" level=error msg="Error while running discovery 'cloud_discovery': Post \http://ua00114t.xxxxxxx.com:8081/api/collect\: dial tcp 10.89.199.43:8081: connect: connection refused"

On the server side, they observe the web pod is in status CrashLoopBackOff:

ua00114t:/sapsw/unix/trento/server # k3s kubectl get pods --all-namespaces

NAMESPACE     NAME                                     READY   STATUS             RESTARTS        AGE

...

default       ua00114t-web-558c65c9cc-kt7rq            0/1     CrashLoopBackOff   11 (2m3s ago)   3h1m

And the log file of the web pod shows a panic message:

time="2022-01-28 20:01:28" level=info msg="Projecting event: 33"

time="2022-01-28 20:01:28" level=info msg="Projector: hosts is interested in cloud_discovery. Projecting event: 33"

panic: interface conversion: interface {} is nil, not string