kudobuilder/kuttl

Improved error output

jtucci opened this issue · 0 comments

jtucci commented

What would you like to be added:
Improve the way the errors are logged.

  • ability to suppress the diff output. sometimes I would prefer to just have a block of failed assertions ie.
    spec.name mismatch expected "name" != actual "wrong name"
  • reduce what is shown in the diff output. I don't need to see a diff with all the k8s generated fields
  • log all failed assertions. As of now, it only will show the first mismatch found.
  • Add ability to enable colorized output
  • When a resource is not found, the error message is a little bit confusing. The template it uses is gvk but if there is no name or namespace specified it formats the string with Kind:/. I think a better implementation would be to just output the top half off the resource by using a go template like
apiVersion: example.api/v1
Kind:  ExampleKind
metdata:
  name: (if it exists)
  namespace: (if it exists)
  labels: (if they exists)

Here is an example of the output that I think would make it more clear

Error:  Comparison Failed for Object:

--- Account (Expected)
+++ Account (Actual)
@@ -4,22 +4,27 @@
   annotations:
     crossplane.io/composition-resource-name: account
     crossplane.io/external-name: teststageeudemo
   labels:
     crossplane.io/claim-namespace: test-stage-eu
+    crossplane.io/composite: test-storage-account
+    ice.devops.essentials.sap/external-name: teststageeudemo
 spec:
   forProvider:
     accountKind: StorageV2
-    accountReplicationType: Standard
+    accountReplicationType: GRS
     accountTier: Standard
-    allowNestedItemsToBePublic: true
     blobProperties:
     - containerDeleteRetentionPolicy:
-      - days: 60
+      - days: 30
       deleteRetentionPolicy:
       - days: 30
       versioningEnabled: true
     enableHttpsTrafficOnly: true
     location: westeurope
+    minTlsVersion: TLS1_2
     resourceGroupNameSelector:
       matchLabels:
         crossplane.io/claim-namespace: test-stage-eu
@@ -29,7 +34,7 @@
     tags:
       sec-by-def-public-storage-exception: enabled
   providerConfigRef:
-    name: test-stage-eu-azure
+    name: test-stage-us-azure
   writeConnectionSecretToRef:
     name: test-stage-eu-demo-connection-secret
     namespace: test-stage-eu

apiVersion: storage.azure.upbound.io/v1beta1
kind: Account
metadata:
  labels:
    crossplane.io/claim-namespace: test-stage-eu

 - spec.forProvider.accountReplicationType: value mismatch, expected: Standard != actual: GRS
 - spec.forProvider.blobProperties.[0].containerDeleteRetentionPolicy.[0].days: value mismatch, expected: 60 != actual: 30
 - spec.forProvider.allowNestedItemsToBePublic: key is missing from map
 - spec.providerConfigRef.name: value mismatch, expected: test-stage-eu-azure != actual: test-stage-us-azure

Error:  the following resource was expected but no match was found:
apiVersion: storage.azure.upbound.io/v1beta1
kind: Container
metadata:
  labels:
    crossplane.io/claim-name: test-storage-account
    crossplane.io/claim-namespace: test-stage-eu

Error:  the following resource was expected but no match was found:
apiVersion: storage.azure.upbound.io/v1beta1
kind: Container
metadata:
  labels:
    crossplane.io/claim-name: test-storage-account
    crossplane.io/claim-namespace: test-stage-eu

Why is this needed:
The way errors are currently logged make it difficult to parse.