Regression: vault resources not syncing to k8s secrets
Closed this issue · 1 comments
Expected behavior: vault-crd creates and deletes kubernetes secrets in the cluster when Vault
resources are created or deleted.
Observed behavior:
- vault-crd doesn't sync newly created
Vault
resources with the vault server to create a kubernetes secret. - On deletion of a
Vault
resource, its respective kubernetes secret is not deleted. - Output from the modules
d.koudingspawn.vault.kubernetes.Watcher
andd.k.vault.kubernetes.KubernetesService
no longer appear in logs
Versions: 1.3.2
, or 1.4.0
(crd update was applied during deployment of 1.4.0). Tested with kubernetes clusters on versions 1.12.8
and 1.15.6
. Same behavior occurs in both clusters.
This might be a regression introduced in version 1.3.2, present in 1.4.0 as well.
Logs from vault version 1.4.0 (demonstrates incorrect behavior):
+ vault-crd-5595bf88d-mngb5 › vault-crd
vault-crd-5595bf88d-mngb5 vault-crd
vault-crd-5595bf88d-mngb5 vault-crd . ____ _ __ _ _
vault-crd-5595bf88d-mngb5 vault-crd /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
vault-crd-5595bf88d-mngb5 vault-crd ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
vault-crd-5595bf88d-mngb5 vault-crd \\/ ___)| |_)| | | | | || (_| | ) ) ) )
vault-crd-5595bf88d-mngb5 vault-crd ' |____| .__|_| |_|_| |_\__, | / / / /
vault-crd-5595bf88d-mngb5 vault-crd =========|_|==============|___/=/_/_/_/
vault-crd-5595bf88d-mngb5 vault-crd :: Spring Boot :: (v2.2.4.RELEASE)
vault-crd-5595bf88d-mngb5 vault-crd
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:18.016 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-5595bf88d-mngb5 with PID 1 (/opt/vault-crd.jar started by root in /opt)
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:18.019 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : No active profile set, falling back to default profiles: default
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.181 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.193 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.193 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.30]
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.251 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.252 INFO 1 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 1190 ms
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:19.595 INFO 1 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:20.738 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:20.791 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:20.793 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : Started VaultApplication in 3.104 seconds (JVM running for 3.53)
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:52:30.791 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.413 INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.413 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.418 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 5 ms
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:06.623 INFO 1 --- [nio-8080-exec-1] o.s.v.a.LifecycleAwareSessionManager : Scheduling Token renewal
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:53:30.784 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:54:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:54:30.783 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:55:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:55:30.784 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:56:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:56:30.781 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:57:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:57:30.784 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:58:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:58:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:59:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 14:59:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:00:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:00:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:01:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:01:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:02:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:02:30.781 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:03:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:03:30.788 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:04:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:04:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:05:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:05:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:06:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:06:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:07:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:07:30.781 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:08:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:08:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:09:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:09:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:10:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:10:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:11:30.774 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-5595bf88d-mngb5 vault-crd 2020-01-27 15:11:30.780 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
Logs from vault version 1.3.1 (demonstrates correct behavior):
+ vault-crd-586bb99858-kg9zc › vault-crd
vault-crd-586bb99858-kg9zc vault-crd
vault-crd-586bb99858-kg9zc vault-crd . ____ _ __ _ _
vault-crd-586bb99858-kg9zc vault-crd /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
vault-crd-586bb99858-kg9zc vault-crd ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
vault-crd-586bb99858-kg9zc vault-crd \\/ ___)| |_)| | | | | || (_| | ) ) ) )
vault-crd-586bb99858-kg9zc vault-crd ' |____| .__|_| |_|_| |_\__, | / / / /
vault-crd-586bb99858-kg9zc vault-crd =========|_|==============|___/=/_/_/_/
vault-crd-586bb99858-kg9zc vault-crd :: Spring Boot :: (v2.1.1.RELEASE)
vault-crd-586bb99858-kg9zc vault-crd
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:16.616 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : Starting VaultApplication v0.0.1-SNAPSHOT on vault-crd-586bb99858-kg9zc with PID 1 (/opt/vault-crd.jar started by root in /opt)
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:16.619 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : No active profile set, falling back to default profiles: default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.058 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.086 INFO 1 --- [ main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.087 INFO 1 --- [ main] org.apache.catalina.core.StandardEngine : Starting Servlet Engine: Apache Tomcat/9.0.13
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.099 INFO 1 --- [ main] o.a.catalina.core.AprLifecycleListener : The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: [/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64/server:/usr/lib/jvm/java-1.8-openjdk/jre/lib/amd64:/usr/lib/jvm/java-1.8-openjdk/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib]
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.178 INFO 1 --- [ main] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.178 INFO 1 --- [ main] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 1519 ms
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.559 INFO 1 --- [ main] o.s.s.c.ThreadPoolTaskScheduler : Initializing ExecutorService 'vaultThreadPoolTaskScheduler'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:18.779 WARN 1 --- [ main] i.f.k.client.internal.VersionUsageUtils : The client is using resource type 'customresourcedefinitions' with unstable version 'v1beta1'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.529 INFO 1 --- [ main] o.s.b.a.e.web.EndpointLinksResolver : Exposing 1 endpoint(s) beneath base path '/actuator'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.591 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.593 INFO 1 --- [ main] de.koudingspawn.vault.VaultApplication : Started VaultApplication in 3.304 seconds (JVM running for 3.761)
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.693 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret0 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.702 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret1 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.716 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret2 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.721 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret3 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.726 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret4 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.729 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret5 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.738 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret6 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.744 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret7 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.748 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret8 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.758 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret9 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.762 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret10 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.771 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret11 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.777 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret12 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.782 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret13 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.787 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret14 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.794 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret15 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.797 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret16 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.801 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret17 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.805 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret18 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.809 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret19 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.812 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret20 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.820 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret21 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.824 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret22 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.834 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret23 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.838 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret24 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:19.841 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret25 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:29.566 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:29.695 INFO 1 --- [TaskScheduler-1] o.s.v.a.LifecycleAwareSessionManager : Scheduling Token renewal
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:30.084 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:35.683 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:05:35.730 INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService : Created secret for vault resource secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:02.649 INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:02.650 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:02.658 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 7 ms
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:06:29.925 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:07:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:07:29.902 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:08:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:08:29.902 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:09:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:34:29.869 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:35:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:35:29.872 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:36:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:36:29.879 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:37:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:37:29.867 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:38:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:38:29.953 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:39:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:39:29.868 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:00.305 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: DELETED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:00.317 INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService : Deleted secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:40:29.860 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:18.063 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: ADDED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:18.088 INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService : Created secret for vault resource secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:41:29.913 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:42:29.568 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:42:29.853 INFO 1 --- [TaskScheduler-1] d.k.vault.kubernetes.KubernetesService : Modified secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:42:30.084 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:43:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:43:29.865 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:44:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:44:29.903 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:45:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:45:29.876 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:29.886 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:34.800 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: MODIFIED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:46:34.827 INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService : Modified secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:28.099 INFO 1 --- [/x.x.x.x/...] d.koudingspawn.vault.kubernetes.Watcher : Received action: DELETED for secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:28.107 INFO 1 --- [/x.x.x.x/...] d.k.vault.kubernetes.KubernetesService : Deleted secret secret26 in namespace default
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:29.565 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Start refresh of secret...
vault-crd-586bb99858-kg9zc vault-crd 2020-01-27 15:47:29.897 INFO 1 --- [TaskScheduler-1] d.k.v.k.scheduler.ScheduledRefresh : Finished refresh of secret...
Hi @gaffneyd4,
you are right, it doesn't worked anymore because of a change in the fabric8io kubernetes-client api. I've provided a fix that is released in version 1.4.1. I already tested on one of my clusters and it seems to work again, sorry for the hassle.
Cheers,
Björn