vault-crd sometimes crashes with nullpointer exception

[McSlow]

Hi, we're currently experiencing the following issue from time to time. Doesn't happen too often but sometimes this happens:

2018-08-13 12:09:29.213  INFO 7 --- [pool-1-thread-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
2018-08-13 12:09:29.257  INFO 7 --- [pool-1-thread-1] d.k.v.k.scheduler.ScheduledRefresh       : Finished refresh of secret...
2018-08-13 12:10:29.213  INFO 7 --- [pool-1-thread-1] d.k.v.k.scheduler.ScheduledRefresh       : Start refresh of secret...
2018-08-13 12:10:29.923 ERROR 7 --- [pool-1-thread-1] o.s.s.s.TaskUtils$LoggingErrorHandler    : Unexpected error occurred in scheduled task.

java.lang.NullPointerException: null
        at de.koudingspawn.vault.kubernetes.KubernetesService.modifySecret(KubernetesService.java:58) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.kubernetes.EventHandler.modifyHandler(EventHandler.java:44) ~[classes!/:0.0.1-SNAPSHOT]
        at de.koudingspawn.vault.kubernetes.scheduler.ScheduledRefresh.refreshCertificates(ScheduledRefresh.java:42) ~[classes!/:0.0.1-SNAPSHOT]
        at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_171]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_171]
        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) ~[spring-context-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.0.4.RELEASE.jar!/:5.0.4.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_171]
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_171]
        at java.lang.Thread.run(Thread.java:748) [na:1.8.0_171]

(this continues for every request afterwards...)

Simply killing the pod so that it gets recreated gets things back going again.

Any idea what this could be?

Btw: is there any chance (beside a sidecar constantly requesting a new cert) to implement a healtcheck?
Apart from this vault-crd is running really fine! :)

If you need more logs just drop me a line...

best regards,
Bjoern

[DaspawnW]

Hi Björn,

I've made some better exception handling to prevent such outtakes and also implemented some health check, that will validate the vault communication.

Please see the rbac.yaml file in the deploy folder that contains liveness probes.

Release version 1.0.2 is also build and pushed.

If this doesn't fix your problems please feel free to reopen the issue.

Cheers,
Björn