Ability to check correctness of values against Chart JSON schema

[MathieuAvila]

Hello,

Is there a way to check that the JSON Schema is applied correctly by Helm and the charts ?

The idea is to check valid and invalid values, and check they are accepted or rejected by the Helm chart. For now, I understand it is only possible to check for correctness, as incorrectness would fail upfront. Would that be a good idea to add helm-unittest the possibility to test failure ?
For now, we're running a python script that checks for assertions, but we'd prefer to have only one source of UT (helm-unittest).

Thanks,

(note: same question asked to https://github.com/quintush/helm-unittest , not sure which is best :) )

[PrivatePuffin]

Helm lint does that ;-)

[mac2000]

indeed that can be done with helm lint, something like:

#!/usr/bin/env bash

set -e

echo 'should pass for valid numeric service port'
helm lint demo --set service.port=80

echo 'should fail for non numeric service port'
! helm lint demo --set service.port=hello

but it becomes messy especially when there are many required but not provided variables in chart also that is one more step/script/tool/etc

I was wondering if we could add one more property to TestJob struct, aka - ShouldThrow bool and little bit later, where we are rendering chart, check that flag and based on it invert error into success, e.g., something like:

type TestJob struct {
  // ...
  ShouldFail bool `yaml:"shouldFail"`
}

// ...

func (t *TestJob) Run(
	targetChart *chart.Chart,
	cache *snapshot.Cache,
	result *TestJobResult,
) *TestJobResult {
  // ...
  outputOfFiles, err := t.renderChart(targetChart, userValues)
  if err != nil {
    if t.ShouldFail {
      result.Passed = true
    } else {
      result.ExecError = err
    }
    return result
  }
  if t.ShouldFail {
    result.Passed = false
    return result
  }
  // ...
}

this will allow us to have something like:

suite: service
templates:
  - service.yaml
tests:
  - it: should not allow non numeric port
    values:
      - ./values/required.yaml
    set:
      port: 'foo'
    shouldThrow: true
  - it: should not allow negative port
    values:
      - ./values/required.yaml
    set:
      port: -42
    shouldThrow: true

and now it will protect us from unwanted json schema changes

ps: or even funnier, it might be a secret naming, aka if test starts with something like: "- it: should fail if ...." then it will kick in :)