DeriveFromObject slow when using ObservedObjectCollection
haarchri opened this issue · 0 comments
haarchri commented
What happened?
have an issue with ObservedObjectCollection in provider-kubernetes - When I start using them, all other objects with spec.readiness.policy set to DeriveFromObject become extremely slow.
It takes up to 15 minutes to get the status updated, whereas without ObservedObjectCollection, it takes around 5 minutes. The ObservedObjectCollection is not used in combination with these other objects - they are independent of each other
NAME KIND PROVIDERCONFIG SYNCED READY AGE
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-8hsn8 Provider ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True False 5m23s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-9ntqr Provider 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True False 5m23s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-ftw5w Provider ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True False 5m23s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-gmsj2 Provider 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True False 5m23s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-jsdxk Provider 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True False 5m23s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-tdjjf Provider ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True False 5m24s
NAME KIND PROVIDERCONFIG SYNCED READY AGE
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-6bxtc ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 5m24s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-8gcc2 ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 5m23s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-btwcs 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 5m24s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-bxrvn 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 5m24s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-qlqff ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 5m23s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-wgbjq 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 5m23s
Also, the observed objects are not created - it takes around 15-25 minutes.
Interestingly, when I switch spec.readiness.policy to default, all resources are created immediately.
So, something is blocking when using ObservedObjectCollection in combination with objects that have DeriveFromObject set for spec.readiness.policy
NAME KIND PROVIDERCONFIG SYNCED READY AGE
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-6klh7-c167b6f ProviderRevision 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 2s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-8hsn8 Provider ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 7m13s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-9ntqr Provider 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 7m13s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-ftw5w Provider ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 7m13s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-gmsj2 Provider 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 7m13s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-h6gbk-9e50f04 ProviderRevision 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 1s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-jsdxk Provider 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 7m13s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-jv77w-64835f1 ProviderRevision ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 2s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-tdjjf Provider ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 7m14s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-tvh2s-9e50f04 ProviderRevision ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 2s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-vdvzs-64835f1 ProviderRevision 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 3s
object.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-xj22n-c167b6f ProviderRevision ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 1s
NAME KIND PROVIDERCONFIG SYNCED READY AGE
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-6bxtc ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 7m34s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-8gcc2 ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 7m33s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-btwcs 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 7m34s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-bxrvn 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 7m34s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-qlqff ea1e5ffc-8798-42ac-bad1-f69a127fa151-ctp True True 7m33s
observedobjectcollection.kubernetes.crossplane.io/upbound-upbound-aws-us-east-1-wgbjq 8d24ce56-40df-4db9-ae7d-96c823d2f564-ctp True True 7m33s
How can we reproduce it?
What environment did it happen in?
Crossplane version: