Compiler short circuits when validating schemas, not providing the full list of errors
hysmio opened this issue · 0 comments
Hey! Noticed something while building while using extensions. It seems that Compile
short circuits validation when errors occur in a document. I imagined that it would validate all schemas even if errors occurred in one.
This very well could be a design decision, but wanted to clarify.
4 fairly verbose examples can be found here:
https://goplay.tools/snippet/dhM5tkI82j1
The examples are:
- returns meta missing properties
Actual output is the expected output
jsonschema file:///home/user/projects/golang-playground/schema.json compilation failed [I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json# [I#] [S#/required] missing properties: 'foo', 'foobar'
- returns
https://json-schema.org/draft/2020-12/schema
validation error, but notmeta
extension validation error
Actual output:
jsonschema file:///home/user/projects/golang-playground/schema2.json compilation failed [I#] [S#] doesn't validate with https://json-schema.org/draft/2020-12/schema# [I#] [S#/allOf/3] allOf failed [I#] [S#/allOf/3/$ref] doesn't validate with 'https://json-schema.org/draft/2020-12/meta/validation#' [I#/minLength] [S#/properties/minLength/$ref] doesn't validate with '/$defs/nonNegativeIntegerDefault0' [I#/minLength] [S#/$defs/nonNegativeIntegerDefault0/$ref] doesn't validate with '/$defs/nonNegativeInteger' [I#/minLength] [S#/$defs/nonNegativeInteger/minimum] must be >= 0 but found -1
Expected output:
jsonschema file:///home/user/projects/golang-playground/schema2.json compilation failed [I#] [S#] doesn't validate with https://json-schema.org/draft/2020-12/schema# [I#] [S#/allOf/3] allOf failed [I#] [S#/allOf/3/$ref] doesn't validate with 'https://json-schema.org/draft/2020-12/meta/validation#' [I#/minLength] [S#/properties/minLength/$ref] doesn't validate with '/$defs/nonNegativeIntegerDefault0' [I#/minLength] [S#/$defs/nonNegativeIntegerDefault0/$ref] doesn't validate with '/$defs/nonNegativeInteger' [I#/minLength] [S#/$defs/nonNegativeInteger/minimum] must be >= 0 but found -1 [I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json# [I#] [S#/required] missing properties: 'foo', 'foobar'
- returns
meta2
required properties error, but notmeta#/foobar/minLength
Actual output: (this shows two children errors in one message)
jsonschema file:///home/user/projects/golang-playground/schema3.json compilation failed [I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json# [I#] [S#/required] missing properties: 'foo' [I#/foobar] [S#/properties/foobar/minLength] length must be >= 3, but got 1
Expected output:
jsonschema file:///home/user/projects/golang-playground/schema3.json compilation failed [I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta.json# [I#] [S#/required] missing properties: 'foo' [I#/foobar] [S#/properties/foobar/minLength] length must be >= 3, but got 1 [I#] [S#] doesn't validate with file:///home/user/projects/golang-playground/extension_meta_2.json# [I#] [S#/required] missing properties: 'bar'
- **Validate returns the same singular meta error when calling
.Error()
but
DetailedOutput()
returned the correct errors**
Actual output:jsonschema: '' does not validate with file:///home/hayden/projects/golang-playground/schema4.json#/maximum: must be <= 8 but found 9
Expected output: (ValidationError.GoString())
[I#] [S#] doesn't validate with file:///home/hayden/projects/golang-playground/schema4.json# [I#] [S#/maximum] must be <= 8 but found 9 [I#] [S#/foobar] when foo is bar, numbers must be >= 10, but got 9
I've created a PR here for Compile
supporting multiple errors.
For ValidationError.Error()
I would suggest using GoString
or creating a way to show multiple errors.