strict mode: missing type "number,string" for keyword "format" at "#" (strictTypes)

Question

strict mode: missing type "number,string" for keyword "format" at "#" (strictTypes)

jeraldsm opened this issue 2 years ago · 13 comments

I get the following error, when I run kubevious guard against a Custom resource external-secrets with --live-k8s

strict mode: missing type "number,string" for keyword "format" at "#/properties/status/properties/refreshTime" (strictTypes)

Here is the full logs of running kubevious guard:

kustomize build deployment/kustomize/overlays/dev/ | kubevious guard --stream --live-k8s --ignore-unknown  --skip-rules container-latest-image
✔ Sources identified.
✔ Manifests loaded.
✔ Received manifests from stdin.
✔ Connected to K8s Cluster.
⠧ Extracting K8s API Schema...[2023-01-06 11:44:02.620] WARN (k8s): [request] Failed. Method: GET. StatusCode: 404-Not Found. 
⠸ Extracting K8s API Schema...[2023-01-06 11:44:03.108] WARN (k8s): [request] Failed. Method: GET. StatusCode: 404-Not Found. 
✔ K8s API Schema Fetched.
✔ Lint complete.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary populated from K8s.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary locally populated.
✔ Rules validation complete.
ℹ️  Linting against Kubernetes Version: v1.22.15-eks-fb459a0

-= SOURCES =-
   ✅ No issues with sources.

-= MANIFESTS =-
   ❌ Namespace: microservice-blueprint, API: external-secrets.io/v1alpha1, Kind: ExternalSecret, Name: app-secret
      ♒ STREAM: stream
      🔴 strict mode: missing type "number,string" for keyword "format" at "#/properties/status/properties/refreshTime" (strictTypes)

-= RULES =-
   ✅ No issues with rules.

-= SUMMARY =-
    📚 Sources: 38
        ✖️  Sources with Errors: 0
    📄 Manifests: 45
        ✅ Valid Manifests: 43
        ❌ Manifests with Errors: 1
        ❕ Manifests with Warnings: 0
        ✅ Manifests Processed for Rules: 3
        ✖️  Manifests with Rule Errors: 0
        ❕ Manifests with Rule Warnings: 0
    📜 Rules: 30
        ✅ Rules Passed: 30
        ✖️  Rules Failed: 0
        🔘 Rules with Errors: 0
        ❕ Rules with Warnings: 0

ℹ️  Run with --detailed to see all sources and manifests

❌ Guard Failed

Following is the external-secret configuration in local. The configuration itself works without any issues.

---
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  name: app-secret
spec:
  dataFrom:
  - key: project-dev-app-secret 
  refreshInterval: 1h
  secretStoreRef:
    name: secrets
    kind: SecretStore
  target:
    name: app-secret

Kindly help me to resolve this issue.

Answer 1 · 2023-01-06T22:02:18.000Z

hi @jeraldsm, Could you post the raw output of kustomize build deployment/kustomize/overlays/dev/ command?

Just a couple of comments regarding the usage. You can pass the kustimoze directory or kustomization.yaml file directly as an input without calling kustomize and piping it into kubevious CLI like this:

kubevious guard deployment/kustomize/overlays/dev/ --live-k8s --ignore-unknown  --skip-rules container-latest-image

Also, this should work without the --ignore-unknown flag, because it is expected that all YAML files to be valid K8s resources.

Answer 2 · 2023-01-09T06:13:12.000Z

Hi @rubenhak

Thanks for your comment.

I get the same error when I pass the kustomize path directly as input:

kubevious guard deployment/kustomize/overlays/dev  --live-k8s  --skip-rules container-latest-image
✔ Sources identified.
✔ Succeeded: kustomize build /Users/jeraldmanakkunnel/Documents/projects/xxx/backend/microservice-blueprint/deployment/kustomize/overlays/dev
✔ Manifests loaded.
✔ Connected to K8s Cluster.
⠼ Extracting K8s API Schema...[2023-01-09 06:02:29.074] WARN (k8s): [request] Failed. Method: GET. StatusCode: 404-Not Found. 
⠋ Extracting K8s API Schema...[2023-01-09 06:02:29.563] WARN (k8s): [request] Failed. Method: GET. StatusCode: 404-Not Found. 
✔ K8s API Schema Fetched.
✔ Lint complete.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary populated from K8s.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary locally populated.
✔ Rules validation complete.
ℹ️  Linting against Kubernetes Version: v1.22.15-eks-fb459a0

-= SOURCES =-
   ✅ No issues with sources.

-= MANIFESTS =-
   ❌ Namespace: microservice-blueprint, API: external-secrets.io/v1alpha1, Kind: ExternalSecret, Name: app-secret
      ☸️  KUSTOMIZE: /Users/jeraldmanakkunnel/Documents/projects/xxx/backend/microservice-blueprint/deployment/kustomize/overlays/dev
      📄 FILE: /Users/jeraldmanakkunnel/Documents/projects/xxx/backend/microservice-blueprint/deployment/kustomize/overlays/dev/kustomization.yaml
      🔴 strict mode: missing type "number,string" for keyword "format" at "#/properties/status/properties/refreshTime" (strictTypes)


-= RULES =-
   ✅ No issues with rules.

-= SUMMARY =-
    📚 Sources: 39
        ✖️  Sources with Errors: 0
    📄 Manifests: 43
        ✅ Valid Manifests: 42
        ❌ Manifests with Errors: 1
        ❕ Manifests with Warnings: 0
        ✅ Manifests Processed for Rules: 3
        ✖️  Manifests with Rule Errors: 0
        ❕ Manifests with Rule Warnings: 0
    📜 Rules: 30
        ✅ Rules Passed: 30
        ✖️  Rules Failed: 0
        🔘 Rules with Errors: 0
        ❕ Rules with Warnings: 0

ℹ️  Run with --detailed to see all sources and manifests

❌ Guard Failed

Please find the output of kustomize build deployment/kustomize/overlays/dev/ command:

apiVersion: v1
data:
  AWS_REGION: eu-central-1
  JAVA_TOOL_OPTIONS: -Dspring.profiles.active=dev -javaagent:/usr/app/dd-java-agent.jar
    -Xms256m -Xmx512m -Xverify:none -XX:TieredStopAtLevel=1 -Dspring.main.lazy-initialization=false
    -Dspringdoc.api-docs.enabled=false
kind: ConfigMap
metadata:
  labels:
    app: microservice-blueprint
  name: app-config
  namespace: microservice-blueprint
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: microservice-blueprint
  name: microservice-blueprint
  namespace: microservice-blueprint
spec:
  ports:
  - name: web
    port: 80
    targetPort: web
  selector:
    app: microservice-blueprint

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: microservice-blueprint
  name: microservice-blueprint
  namespace: microservice-blueprint
spec:
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: microservice-blueprint

  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: microservice-blueprint
      name: microservice-blueprint
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - microservice-blueprint
              topologyKey: topology.kubernetes.io/zone
            weight: 100
      containers:
      - env:
        envFrom:
        - secretRef:
            name: app-secret
        - configMapRef:
            name: app-config
        image: REPLACE_IMAGE
        imagePullPolicy: Always
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /microservice-blueprint/actuator/health/readiness
            port: 8999
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 2
        name: microservice-blueprint
        ports:
        - containerPort: 8999
          name: web
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /microservice-blueprint/actuator/health/readiness
            port: 8999
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 15
          successThreshold: 1
          timeoutSeconds: 2
        resources:
          limits:
            cpu: 250m
            memory: 512Mi
          requests:
            cpu: 250m
            memory: 512Mi
        securityContext:
          readOnlyRootFilesystem: false
          runAsGroup: 10001
          runAsNonRoot: true
          runAsUser: 10001
        startupProbe:
          failureThreshold: 5
          httpGet:
            path: /microservice-blueprint/actuator/health/readiness
            port: 8999
            scheme: HTTP
          initialDelaySeconds: 30
          periodSeconds: 15
          successThreshold: 1
          timeoutSeconds: 2
        volumeMounts:
        - mountPath: /app-config/mount
          name: app-config-volume
      serviceAccountName: microservice-blueprint
      shareProcessNamespace: true
      terminationGracePeriodSeconds: 5
      volumes:
      - configMap:
          name: app-config
        name: app-config-volume
---
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
  labels:
    app: microservice-blueprint

  name: app-secret
  namespace: microservice-blueprint
spec:
  dataFrom:
  - key: project-dev-app-secret
  refreshInterval: 1h
  secretStoreRef:
    kind: SecretStore
    name: secrets
  target:
    name: app-secret
---
apiVersion: external-secrets.io/v1alpha1
kind: SecretStore
metadata:
  labels:
    app: microservice-blueprint

  name: secrets
  namespace: microservice-blueprint
spec:
  provider:
    aws:
      region: eu-central-1
      service: SecretsManager
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/backend-protocol: HTTP
    alb.ingress.kubernetes.io/group.name: backend
    alb.ingress.kubernetes.io/scheme: internal
    alb.ingress.kubernetes.io/success-codes: "200"
    alb.ingress.kubernetes.io/target-type: ip
    kubernetes.io/ingress.class: alb
  labels:
    app: microservice-blueprint

  name: microservice-blueprint
  namespace: microservice-blueprint
spec:
  rules:
  - http:
      paths:
      - backend:
          service:
            name: microservice-blueprint
            port:
              name: web
        path: /microservice-blueprint*
        pathType: ImplementationSpecific****

Answer 3 · 2023-01-09T22:15:11.000Z

Could you try this command locally? I took the output you provided and ran it. Didn't see any errors with the "external-secret" object. I see in your case, it complains about "#/properties/status/properties/refreshTime", but the status object was not even populated in your kustomize output.

$ kubevious guard https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/external-secret.yaml https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/crd.yaml

Here is the output I see:

✔ Sources identified.
✔ Manifests loaded.
✔ Fetched K8s API schema.
✔ Lint complete.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary locally populated.
✔ Rules validation complete.
ℹ️  Linting against Kubernetes Version: 1.25.2

-= SOURCES =-
   ✅ No issues with sources.

-= MANIFESTS =-
   ✅ No issues with manifests.

-= RULES =-
   ✅ No issues with rules.

-= SUMMARY =-
    📚 Sources: 38
        ✖️  Sources with Errors: 0
    📄 Manifests: 38
        ✅ Valid Manifests: 38
        ✖️  Manifests with Errors: 0
        ❕ Manifests with Warnings: 0
        ☑️  Manifests Processed for Rules: 0
        ✖️  Manifests with Rule Errors: 0
        ❕ Manifests with Rule Warnings: 0
    📜 Rules: 31
        ✅ Rules Passed: 31
        ✖️  Rules Failed: 0
        🔘 Rules with Errors: 0
        ❕ Rules with Warnings: 0

ℹ️  Run with --detailed to see all sources and manifests

✅ Guard Succeeded.

Answer 4 · 2023-01-10T06:27:11.000Z

Hi @rubenhak,

The kubevious guard succeeds with the command you have provided locally and also with "--live-k8s". Please find the output below:

kubevious guard https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/external-secret.yaml https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/crd.yaml
✔ Sources identified.
✔ Manifests loaded.
✔ Fetched K8s API schema.
✔ Lint complete.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary locally populated.
✔ Rules validation complete.
ℹ️  Linting against Kubernetes Version: 1.25.2

-= SOURCES =-
   ✅ No issues with sources.

-= MANIFESTS =-
   ✅ No issues with manifests.

-= RULES =-
   ✅ No issues with rules.

-= SUMMARY =-
    📚 Sources: 38
        ✖️  Sources with Errors: 0
    📄 Manifests: 38
        ✅ Valid Manifests: 38
        ✖️  Manifests with Errors: 0
        ❕ Manifests with Warnings: 0
        ☑️  Manifests Processed for Rules: 0
        ✖️  Manifests with Rule Errors: 0
        ❕ Manifests with Rule Warnings: 0
    📜 Rules: 31
        ✅ Rules Passed: 31
        ✖️  Rules Failed: 0
        🔘 Rules with Errors: 0
        ❕ Rules with Warnings: 0

ℹ️  Run with --detailed to see all sources and manifests

✅ Guard Succeeded.

But, When running the kubevious guard locally without specifying "live-k8s" argument and crd.yaml file, I get the "Unknown API Resource" error in the manifest validation part as shown below:

kubevious guard https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/external-secret.yaml           
✔ Sources identified.
✔ Manifests loaded.
✔ Fetched K8s API schema.
✔ Lint complete.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary locally populated.
✔ Rules validation complete.
ℹ️  Linting against Kubernetes Version: 1.25.2

-= SOURCES =-
   ✅ No issues with sources.

-= MANIFESTS =-
   ❌ Namespace: microservice-blueprint, API: external-secrets.io/v1alpha1, Kind: ExternalSecret, Name: app-secret
      🌐 WEB: https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/external-secret.yaml
      🔴 Unknown API Resource. apiVersion: external-secrets.io/v1alpha1, kind: ExternalSecret.


-= RULES =-
   ✅ No issues with rules.

-= SUMMARY =-
    📚 Sources: 37
        ✖️  Sources with Errors: 0
    📄 Manifests: 37
        ✅ Valid Manifests: 36
        ❌ Manifests with Errors: 1
        ❕ Manifests with Warnings: 0
        ☑️  Manifests Processed for Rules: 0
        ✖️  Manifests with Rule Errors: 0
        ❕ Manifests with Rule Warnings: 0
    📜 Rules: 31
        ✅ Rules Passed: 31
        ✖️  Rules Failed: 0
        🔘 Rules with Errors: 0
        ❕ Rules with Warnings: 0

ℹ️  Run with --detailed to see all sources and manifests

❌ Guard Failed

Also, When running the kubevious guard locally by specifying "live-k8s" argument and without the CRD.yaml file , I get the inital error strict mode: missing type "number,string" for keyword "format" at "#/properties/status/properties/refreshTime" (strictTypes) in the manifest validation part as shown below:

 kubevious guard https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/external-secret.yaml  --live-k8s 
✔ Sources identified.
✔ Manifests loaded.
✔ Connected to K8s Cluster.
⠏ Extracting K8s API Schema...[2023-01-10 06:20:26.053] WARN (k8s): [request] Failed. Method: GET. StatusCode: 404-Not Found. 
⠦ Extracting K8s API Schema...[2023-01-10 06:20:26.584] WARN (k8s): [request] Failed. Method: GET. StatusCode: 404-Not Found. 
✔ K8s API Schema Fetched.
✔ Lint complete.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary populated from K8s.
✔   Libraries loaded.
✔   ClusterRules loaded.
✔   Rules loaded.
✔   RuleApplicators loaded
✔ RulesLibrary locally populated.
✔ Rules validation complete.
ℹ️  Linting against Kubernetes Version: v1.22.15-eks-fb459a0

-= SOURCES =-
   ✅ No issues with sources.

-= MANIFESTS =-
   ❌ Namespace: microservice-blueprint, API: external-secrets.io/v1alpha1, Kind: ExternalSecret, Name: app-secret
      🌐 WEB: https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/external-secret.yaml
      🔴 strict mode: missing type "number,string" for keyword "format" at "#/properties/status/properties/refreshTime" (strictTypes)


-= RULES =-
   ✅ No issues with rules.

-= SUMMARY =-
    📚 Sources: 38
        ✖️  Sources with Errors: 0
    📄 Manifests: 37
        ✅ Valid Manifests: 36
        ❌ Manifests with Errors: 1
        ❕ Manifests with Warnings: 0
        ☑️  Manifests Processed for Rules: 0
        ✖️  Manifests with Rule Errors: 0
        ❕ Manifests with Rule Warnings: 0
    📜 Rules: 31
        ✅ Rules Passed: 31
        ✖️  Rules Failed: 0
        🔘 Rules with Errors: 0
        ❕ Rules with Warnings: 0

ℹ️  Run with --detailed to see all sources and manifests

❌ Guard Failed

Passing the file: https://raw.githubusercontent.com/kubevious/demos/main/bug-fixes/guard/external-secret/crd.yaml does resolve this issue. But do I always need to pass the crd.yaml link to fix this ?

Note: I'm using kubevious version v1.0.50 in Mac OS.

Answer 5 · 2023-01-10T07:30:31.000Z

@jeraldsm, can I see your CRD definition for external-secrets.io/v1alpha1 ExternalSecret?

No, it is supposed to work with --live-k8s. For the time being, you can try passing the crd.yaml as an input. But, lets try to get this resolved.

Answer 6 · 2023-01-10T16:37:06.000Z

@rubenhak the CRD definition I use for external-secrets was downloaded using helm and the version is 0.3.11. Please find the link below:
https://github.com/external-secrets/external-secrets/releases/tag/helm-chart-0.3.11

Answer 7 · 2023-01-10T19:35:49.000Z

I've tested it on a local KIND cluster, and it looks file. I may need a few more inputs from you. I want to look at the OpenAPI Schema. Please run:

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

Then query the URL http://localhost:8001/openapi/v2.

The result would be pretty large. This is just the API schema, it doesn't contain your k8s objects. Please feel free to email me at r@kubevious.io

Answer 8 · 2023-01-11T05:34:05.000Z

@rubenhak I have emailed you the API -Schema. I have also sent you the CRD definition we use, just incase if there's any additional change from the vendor manifest.

Answer 9 · 2023-01-11T07:04:55.000Z

@jeraldsm, I got your email. There is something wrong with the "refreshTime" field. See the line 88653.
The refreshTime has the "format": "date-time" set, but not the "type". It is supposed to have "type": "string".

            "refreshTime": {
              "description": "refreshTime is the time and date the external secret was fetched and the target secret updated",
              "format": "date-time"
            },

The manifest CRD contains the "type" field:

refreshTime:
  description: refreshTime is the time and date the external secret was fetched and the target secret updated
  format: date-time
  nullable: true
  type: string

Maybe it is some weird EKS behavior. I will look into this more, and might add a check to assume "string" type if no type is set explicitly.

Answer 10 · 2023-01-11T08:47:20.000Z

@jeraldsm, could you try the latest version v.1.0.52? Home will take bit longer time to release, so can you use the npm installer or binaries.

Answer 11 · 2023-01-17T20:47:54.000Z

Looks like the issue was resolved. Closing for inactivity.

Answer 12 · 2023-01-18T05:51:01.000Z

@rubenhak Apologies for the late response.
I tried the latest version v1.0.52 and indeed the issue was fixed. Thanks a lot for the support!

Answer 13 · 2023-01-18T06:24:57.000Z

Thanks for testing it out!