lidy-js: A JavaScript repository from Design it, Run it

The YAML-YAML type-oriented schema validation tool

Lidy is :

A language parser dedicated for DSLs (Domain Specific Languages) encoded in YAML.
The Lidy schema-type language, a YAML language to specify how to check YAML files
An engine to run the Lidy schema on YAML files
A rich deserialisation tool (if you want it to)

Currently, two versions have been developped :

A javascript version lidy-js
A golang version lidy-go

Each version is available in its own directory at the root of the repository. Both versions fullly implement and respect the specification described in the documentation.

However, currently the way to use the parser differs between the two versions:

lidy-go uses the concept of builder (see specific documentation)
lidy-js uses a similar interface (and in particular the principle of listeners) to that offered by the antlr tool.

JSON schema

What's the point of Lidy, when there's already JSON schema?

YAML: Lidy targets YAML rather than JSON. Of course, it does work with JSON perfectly fine.
Refs: In Lidy, refs are first class citizens, they are just like in programming languages: <name>, as opposed to JSON Schema's heavy { ref: "#/<name>" }, see below.
Line numbers: Lidy is meant to assist your users with writing YAML: Lidy provides the line numbers at which the checking failed.
Algebriac data types: Lidy schema are similar to Algebriac data types. They have union types (_oneOf), positional product types (_list), named product types (_map), and combined types (_merge). (N.B. parameterized types aren't yet there, but they are on our short list).
Rich deserialisation: Lidy provides support for rich deserialisation. It is a core use-case. This includes access to the source line numbers.
Custom checkers: Writing a custom value checker is also a core use-case. Actually, it's just as easy as writing a type deserialiser since Lidy handles the two through the same interface.
Language Recognition: Lidy is a lexical and grammar parser that can build and walk parse trees. It is completely equivalent to a tool like antlr for DSLs encoded in YAML. The point is antlr does not allow to manage languages built on top of YAML.

About lidy's refs

Where you used to write "a": { "ref": "#/$def/b" }, you now write "a": "b". Lidy does not support accessing non-root nodes. All nodes that must be referred to must be at the root of the Lidy schema.

Note: Lidy does not yet support remote references.

Example

main.go

package main

import (
	"fmt"

	"github.com/ditrit/lidy"
)

func main() {
	result, errorList := lidy.NewParser(
		"treeDefinition.yaml",
		[]byte(`
main: tree

tree:
  _map:
    name: string
    children:
      _listOf: tree
`),
	).Parse(
		lidy.NewFile(
			"treeContent.yaml",
			[]byte(`
name: root
children:
  - name: leafA
    children: []
  - name: branchB
    children:
    - name: leafC
      children: []
  - name: leafD
    children: []
`),
		),
	)

	if len(errorList) > 0 {
		panic(errorList[0])
	}

	mapResult := result.(lidy.MapResult)

	fmt.Println(mapResult)
}

Alternatives: YAML Schema validators

Here's a list of schema validators we could find:

Kwalify, [source (mirror?)], [website] (Ruby and Java, v0.7.2, 2010-07-18)
pykwalify [source], [documentation] (Python, v1.7.0, 2018-08-03)
Rx [source], [website] (Js, Perl, PHP, Python, Ruby, v0.200006, 2014-05-21)

Also see the dedicated page on JSON Schema Everywhere.

And a few more project(s):

Azuki [source], just a Map evaluation tool (Java)

None has the feature-set of Lidy, nor its type-oriented approach.

Using Regex

If you need a regex to match a well-known format, think of going shopping for it before you start writing it. Ressource: RgxDB.

Documentation

See DOCUMENTATION.md

Short reference

Glossary

Expression: A lidy expression specifies a way to check a yaml value
Rule: A user rule declaration gives a rule name to an expression
Builder: A builder is a user-provided function which can process the data read by a rule, and change its content, or produce an error
Scalar type: Scalar types are predefined lidy rules which allow to check for a given scalar type, i.e. a type that is not a container
Container checker: A container checker allows to create a lidy expression for a YAML map or a YAML sequence matching certain types

Lidy expression

A lidy expression is either:

The name of a predefined lidy rule
The name of a lidy rule defined in the same document
A YAML map which associates one or more lidy keywords to its YAML argument. See Lidy checker forms.
- Note: Not all keyword combinations are valid

Also see lidy expression.

Predefined lidy rules

Also see predefined lidy rules.

Scalar types

boolean
float
int -- integer
string
nullType -- null

Also see Scalar rules.

Predefined string checker rules

timestamp -- ISO 8601 datetime
binary -- a base64 encoded binary blob, with space characters allowed

Also see Predefined string checker rules.

`any` - Any yaml content

any -- any yaml structure. See any

Container checkers

Map checkers

_map -- followed by a map of exact keys to lidy expressions
_mapOf -- Example: _mapOf: { string: int }
_merge -- create a map checker merging the keys of the listed map checkers
_mapFacultative -- like _map, but the specified entries aren't mendatory

List checkers

_list -- (the equivalent of _map but for sequences. It could have been named _seq)
_listOf
_listFacultative

Composite checkers

_oneOf -- accept a list of lidy expressions and select the first that matches, or fail

Container checkers

_nb -- the container must exactly have the specified number of entries
_max -- the container must have at most the specified number of entries
_min -- the container must have at least the specified number of entries

Scalar checkers

_regex -- applies only to strings. Accepted syntax at https://github.com/google/re2/wiki/Syntax
_in -- an exact enumeration of terminal YAML values the value must be part of

Not yet in Lidy

Range

_range -- would apply only to numbers
- Examples for floats: (0 <= float), (1 < float < 10), (float < 0)
- Examples for integers: (0 <= int <= 9)

Parameter-less string checkers

Somewhat likely to be added (because it wouldn't make lidy heavier):

lang.regex.re2
lang.json
lang.yaml

Less likely to be added, but still considered:

lang.regex.pcre
lang.csv
lang.jsonc
lang.json4
lang.hjson
lang.strictyaml
lang.html
lang.xml

Functional types (aka type parameter aka template types)

Declare a parameter type name:

<ContentType>: []
# the <> are forbidden in lidy identifiers. This form is detected as a
# parameter type name declaration

Declare a functional type:

tree<ContentType>:
  _map:
    name: string
    children: treeChildren
# treeChildren requires a parameter: `treeChildren<ContentType>`
# but lidy is smart enougth to pass it from the parent automatically, since they
# uses the same type name

treeChildren<ContentType>:
  _listOf: treeOrContent

treeOrContent<ContentType>:
  _oneOf:
    - tree
    - ContentType

Refer to the functional type:

main: tree<boolean>

Contributing

If you have any idea that you'd like to see added to Lidy, please create an issue in the issue tracker to share your feature request with us (remember to search-check for duplicate issues first).

You're also welcome to report bugs, ask questions or use the issue tracker as you see fit. We try to be welcoming.

Developing

Cloning:

git clone https://github.com/ditrit/lidy
cd lidy

Running Lidy's tests:

go test

Install Lidy in your project:

npm install "https://github.com/ditrit/lidy-js.git#main" --save

Generate parser according to your grammar in your project:

# Place your grammar in src/lidy/[YOUR_GRAMMAR_NAME].yml.
# Create a generate-parser.js in [YOUR_PROJECT]/scripts with this content: 
import('lidy-js/parser/node_parse.js').then(({ preprocess }) => {
  preprocess('src/lidy/[YOUR_GRAMMAR_NAME].yml');
});

# Add this line to your package.json script:
"generate:parser": "node scripts/generate_parser.js"

# Then run it:
npm run generate:parser

# Replace this in the generated js file
import { parse as parse_input } from '../parser/parse.js'
# by
import { parse as parse_input } from 'lidy-js';

Testing, with the control offered by ginkgo:

# Install
# go get github.com/onsi/ginkgo/ginkgo

# Test
ginkgo

When visiting the .spec.hjson files, you can prefix a group description or criterion description with PENDING or FOCUS to disable running it, or to focus on it. Lidy's specification runner will pass them to Ginkgo using PDescribe, FDescribe, PSpecify and FSpecify, accordingly.

ditrit/lidy-js

The YAML-YAML type-oriented schema validation tool

Content

JSON schema

About lidy's refs

Example

Alternatives: YAML Schema validators

Using Regex

Documentation

Short reference

Glossary

Lidy expression

Predefined lidy rules

Scalar types

Predefined string checker rules

any - Any yaml content

Container checkers

Map checkers

List checkers

Composite checkers

Container checkers

Scalar checkers

Not yet in Lidy

Range

Parameter-less string checkers

Functional types (aka type parameter aka template types)

Contributing

Developing

`any` - Any yaml content