/lidy-go

Primary LanguageGoMozilla Public License 2.0MPL-2.0

GoReport GoDoc License

The YAML-YAML type-oriented schema validation tool

Lidy is :

  • A language parser dedicated for DSLs (Domain Specific Languages) encoded in YAML.
  • The Lidy schema-type language, a YAML language to specify how to check YAML files
  • An engine to run the Lidy schema on YAML files
  • A rich deserialisation tool (if you want it to)

Currently, two versions have been developped :

  • A javascript version lidy-js
  • A golang version lidy-go

Each version is available in its own directory at the root of the repository. Both versions fullly implement and respect the specification described in the documentation.

However, currently the way to use the parser differs between the two versions:

  • lidy-go uses the concept of builder (see specific documentation)
  • lidy-js uses a similar interface (and in particular the principle of listeners) to that offered by the antlr tool.

Content

JSON schema

What's the point of Lidy, when there's already JSON schema?

  • YAML: Lidy targets YAML rather than JSON. Of course, it does work with JSON perfectly fine.
  • Refs: In Lidy, refs are first class citizens, they are just like in programming languages: <name>, as opposed to JSON Schema's heavy { ref: "#/<name>" }, see below.
  • Line numbers: Lidy is meant to assist your users with writing YAML: Lidy provides the line numbers at which the checking failed.
  • Algebriac data types: Lidy schema are similar to Algebriac data types. They have union types (_oneOf), positional product types (_list), named product types (_map), and combined types (_merge). (N.B. parameterized types aren't yet there, but they are on our short list).
  • Rich deserialisation: Lidy provides support for rich deserialisation. It is a core use-case. This includes access to the source line numbers.
  • Custom checkers: Writing a custom value checker is also a core use-case. Actually, it's just as easy as writing a type deserialiser since Lidy handles the two through the same interface.
  • Language Recognition: Lidy is a lexical and grammar parser that can build and walk parse trees. It is completely equivalent to a tool like antlr for DSLs encoded in YAML. The point is antlr does not allow to manage languages built on top of YAML.

About lidy's refs

Where you used to write "a": { "ref": "#/$def/b" }, you now write "a": "b". Lidy does not support accessing non-root nodes. All nodes that must be referred to must be at the root of the Lidy schema.

Note: Lidy does not yet support remote references.

Example

main.go

package main

import (
	"fmt"

	"github.com/ditrit/lidy"
)

func main() {
	result, errorList := lidy.NewParser(
		"treeDefinition.yaml",
		[]byte(`
main: tree

tree:
  _map:
    name: string
    children:
      _listOf: tree
`),
	).Parse(
		lidy.NewFile(
			"treeContent.yaml",
			[]byte(`
name: root
children:
  - name: leafA
    children: []
  - name: branchB
    children:
    - name: leafC
      children: []
  - name: leafD
    children: []
`),
		),
	)

	if len(errorList) > 0 {
		panic(errorList[0])
	}

	mapResult := result.(lidy.MapResult)

	fmt.Println(mapResult)
}

Alternatives: YAML Schema validators

Here's a list of schema validators we could find:

Also see the dedicated page on JSON Schema Everywhere.

And a few more project(s):

  • Azuki [source], just a Map evaluation tool (Java)

None has the feature-set of Lidy, nor its type-oriented approach.

Using Regex

If you need a regex to match a well-known format, think of going shopping for it before you start writing it. Ressource: RgxDB.

Documentation

See DOCUMENTATION.md

Short reference

Glossary

Expression
A lidy expression specifies a way to check a yaml value
Rule
A user rule declaration gives a rule name to an expression
Builder
A builder is a user-provided function which can process the data read by a rule, and change its content, or produce an error
Scalar type
Scalar types are predefined lidy rules which allow to check for a given scalar type, i.e. a type that is not a container
Container checker
A container checker allows to create a lidy expression for a YAML map or a YAML sequence matching certain types

Lidy expression

A lidy expression is either:

  • The name of a predefined lidy rule
  • The name of a lidy rule defined in the same document
  • A YAML map which associates one or more lidy keywords to its YAML argument. See Lidy checker forms.
    • Note: Not all keyword combinations are valid

Also see lidy expression.

Predefined lidy rules

Also see predefined lidy rules.

Scalar types

  • boolean
  • float
  • int -- integer
  • string
  • nullType -- null

Also see Scalar rules.

Predefined string checker rules

  • timestamp -- ISO 8601 datetime
  • binary -- a base64 encoded binary blob, with space characters allowed

Also see Predefined string checker rules.

any - Any yaml content

  • any -- any yaml structure. See any

Container checkers

Map checkers

  • _map -- followed by a map of exact keys to lidy expressions
  • _mapOf -- Example: _mapOf: { string: int }
  • _merge -- create a map checker merging the keys of the listed map checkers
  • _mapFacultative -- like _map, but the specified entries aren't mendatory

List checkers

Composite checkers

  • _oneOf -- accept a list of lidy expressions and select the first that matches, or fail

Container checkers

  • _nb -- the container must exactly have the specified number of entries
  • _max -- the container must have at most the specified number of entries
  • _min -- the container must have at least the specified number of entries

Scalar checkers

Not yet in Lidy

Range

  • _range -- would apply only to numbers
    • Examples for floats: (0 <= float), (1 < float < 10), (float < 0)
    • Examples for integers: (0 <= int <= 9)

Parameter-less string checkers

Somewhat likely to be added (because it wouldn't make lidy heavier):

  • lang.regex.re2
  • lang.json
  • lang.yaml

Less likely to be added, but still considered:

  • lang.regex.pcre
  • lang.csv
  • lang.jsonc
  • lang.json4
  • lang.hjson
  • lang.strictyaml
  • lang.html
  • lang.xml

Functional types (aka type parameter aka template types)

Declare a parameter type name:

<ContentType>: []
# the <> are forbidden in lidy identifiers. This form is detected as a
# parameter type name declaration

Declare a functional type:

tree<ContentType>:
  _map:
    name: string
    children: treeChildren
# treeChildren requires a parameter: `treeChildren<ContentType>`
# but lidy is smart enougth to pass it from the parent automatically, since they
# uses the same type name

treeChildren<ContentType>:
  _listOf: treeOrContent

treeOrContent<ContentType>:
  _oneOf:
    - tree
    - ContentType

Refer to the functional type:

main: tree<boolean>

Contributing

If you have any idea that you'd like to see added to Lidy, please create an issue in the issue tracker to share your feature request with us (remember to search-check for duplicate issues first).

You're also welcome to report bugs, ask questions or use the issue tracker as you see fit. We try to be welcoming.

Developing

Cloning:

git clone https://github.com/ditrit/lidy
cd lidy

Running Lidy's tests:

go test

Testing, with the control offered by ginkgo:

# Install
# go get github.com/onsi/ginkgo/ginkgo

# Test
ginkgo

When visiting the .spec.hjson files, you can prefix a group description or criterion description with PENDING or FOCUS to disable running it, or to focus on it. Lidy's specification runner will pass them to Ginkgo using PDescribe, FDescribe, PSpecify and FSpecify, accordingly.