RNV -- Relax NG Compact Syntax Validator in C
Version 1.7
Table of Contents
News since 1.6
New since 1.5
Aknowledgements
Package Contents
Installation
Invocation
Limitations
Applications
ARX
RVP
User-Defined Datatype Libraries
Datatype Library Plug-in
Scheme Datatypes
New versions
Abstract
RNV is an implementation of Relax NG Compact Syntax,
http://relaxng.org/compact-20021121.html. It is written in ANSI C,
the command-line utility uses Expat,
http://www.jclark.com/xml/expat.html. It is distributed under BSD
license, see license.txt for details.
RNV is a part of an on-going work, and the current code can have bugs
and shortcomings; however, it validates documents against a number of
grammars. I use it.
News since 1.6
The format for error messages is similar to that of Jing (file name,
line and column are colon-separated). Entities and DTD processing is
moved out of RNV, use XX, available from the same download location,
to expand entities.
New since 1.5
Better reporting: required and permitted content is reported
separately; it helps debug grammars. Several bugfixes; I relied on an
acquired test suite and published schemata, but have found that I can
make more bugs than they cover, thus a reworked an extended test suite
is now used for testing. The code has also been cleaned up and
simplified in places during porting to Plan9.
Aknowledgements
I would like to thank those who have helped me develop RNV.
Dave Pawson has been the first user of the program.
Alexander Peshkov helps me with testing and I have been able to
correct very well hidden errors with his help.
Sebastian Rahtz encouraged me to continue working on RNV since the
first release, and has helped me to improve it on more than one
occasion.
Package Contents
Note
I have put rnv.exe and arx.exe, Win32 executables statically linked
with a current version of Expat from
http://expat.sourceforge.net/, into a separate distribution
archive (with name ending in -win32bin). It contains only the program
binaries and should be available from the same location as the source
distribution.
The package consists of:
* the license, license.txt;
* the source code, *.[ch];
* the source code map, src.txt;
* Makefile.bsd for BSD make;
* Makefile.gnu for GNU Make;
* Makefile.bcc for Win32 and Borland C/C++ Compiler;
* tools/xck, a simple shell script I am using to validate documents;
* tools/*.rnc, sample Relax NG grammars;
* scm/*.scm, program modules in Scheme, for Scheme Datatypes
Library;
* the log of changes, changes.txt;
* this file, readme.txt.
* Other scripts, samples and plug-ins appear in tools/ eventually.
Installation
On Unix-like systems, run make -f Makefile.gnu or make -f
Makefile.bsd, depending on which flavour of make you have;
Makefile.bsd should probably work on SysV, but, unfortunately, I have
no place to check for the last couple of years. If you are using Expat
1.2, define EXPAT_H as xmlparse.h instead of expat.h).
On Windows, use rnv.exe. To recompile from the sources, use
Makefile.bcc with Borland C/C++ Compiler, or create a makefile or
project for your environment.
Invocation
The command-line syntax is
rnv {-q|-p|-c|-s|-v|-h} grammar.rnc {document1.xml}
If no documents are specified, RNV attempts to read the XML document
from the standard input. The options are:
-q
names of files being processed are not printed; in error
messages, expected elements and attributes are not listed;
-n <num>
sets the maximum number of reported expected elements and
attributes, -q sets this to 0 and can be overriden;
-p
copies the input to the output;
-c
if the only argument is a grammar, checks the grammar and
exits;
-s
uses less memory and runs slower;
-v
prints version number;
-h
displays usage summary and exits.
Limitations
* RNV assumes that the encoding of the syntax file is UTF-8.
* Support for XML Schema Part 2: Datatypes is partial.
+ ordering for duration is not implemented;
+ only local parts of QName values are checked for equality,
ENTITY values are only checked for lexical validity.
* The schema parser does not check that all restrictions are obeyed,
in particular, restrictions 7.3 and 7.4 are not checked.
* RNV for Win32 platforms is a Unix program compiled on Win32. It
expects file paths to be written with normal slashes; if a schema
is in a different directory and includes or refers external files,
then the schema's path must be written in the Unix way for the
relative paths to work. For example, under Windows, rnv that uses
..\schema\docbook.rnc to validate userguide.dbx should be invoked
as
rnv.exe ../schema/docbook.rnc userguide.dbx
Applications
The distribution includes several utilities built upon RNV; they are
listed and described in the following sections.
ARX
ARX is a tool to automatically determine the type of a document from
its name and contents. It is inspired by James Clark's schema location
approach for nXML,
http://groups.yahoo.com/group/emacs-nxml-mode/message/259, and is
a development of the idea described in
http://relaxng.org/pipermail/relaxng-user/2003-December/000214.htm
l.
ARX is a command-line utility. The invocation syntax is
arx {-n|-v|-h} document.xml arx.conf {arx.conf}
ARX either prints a string corresponding to the document's type or
nothing if the type cannot be determined. The options are:
-n
turns off prepending base path of the configuration file to the
result, even if it looks like a relative path (useful when the
configuration file and the grammars are in separate
directories, or for association with something that is not a
file);
-v
prints current version;
-h
displays usage summary and exits.
The configuration file must conform to the following grammar:
arx = grammars route*
grammars = "grammars" "{" type2string+ "}"
type2string = type "=" literal
type = nmtoken
route = match|nomatch|valid|invalid
match = "=~" regexp "=>" type
nomatch = "!~" regexp "=>" type
valid = "valid" "{" rng "}" "=>" type
invalid = "!valid" "{" rng "}" "=>" type
literal=string in '"', '"' inside must be prepended by '\'
regexp=string in '/', '/' inside must be prepended by '\'
rng=Relax NG Compact Syntax
Comments start with # and continue till the end of line.
Rules are processed sequentially, the first matching rule determines
the file's type. Relax NG templates are matched against file contents,
regular expressions are applied to file names. The sample below
associates documents with grammars for XSLT, DocBook or XSL FO.
grammars {
docbook="docbook.rnc"
xslt="xslt.rnc"
xslfo="fo.rnc"
}
valid {
start = element (book|article|chapter|reference) {any}
any = (element * {any}|attribute * {text}|text)*
} => docbook
!valid {
default namespace xsl = "http://www.w3.org/1999/XSL/Transform"
start = element *-xsl:* {not-xsl}
not-xsl = (element *-xsl:* {not-xsl}|attribute * {text}|text)*
} => xslt
=~/.*\.xsl/ => xslt
=~/.*\.fo/ => xslfo
ARX can also be used to link documents to any type of information or
processing.
RVP
RVP is abbreviation for Relax NG Validation Pipe. It reads validation
primitives from the standard input and reports result to the standard
output; it's main purpose is to ease embedding of a Relax NG validator
into various languages and environment. An application would launch
RVP as a parallel process and use a simple protocol to perform
validation. The protocol, in BNF, is:
query ::= (
quit
| start
| start-tag-open
| attribute
| start-tag-close
| text
| end-tag) z.
quit ::= "quit".
start ::= "start" [gramno].
start-tag-open ::= "start-tag-open" patno name.
attribute ::= "attribute" patno name value.
start-tag-close :: = "start-tag-close" patno name.
text ::= ("text"|"mixed") patno text.
end-tag ::= "end-tag" patno name.
response ::= (ok | er | error) z.
ok ::= "ok" patno.
er ::= "er" patno erno.
error ::= "error" patno erno error.
z ::= "\0" .
* RVP assumes that the last colon in a name separates the local part
from the namespace URI (it is what one gets if specifies `:' as
namespace separator to Expat).
* Error codes can be grabbed from rvp sources by grep _ER_ *.h and
OR-ing them with corresponding masks from erbit.h. Additionally,
error 0 is the protocol format error.
* Either er or error responses are returned, not both; -q chooses
between concise and verbose forms (invocation syntax described
later).
* start passes the index of a grammar (first grammar in the list of
command-line arguments has number 0); if the number is omitted, 0
is assumed.
* quit is not opposite of start; instead, it quits RVP.
The command-line syntax is:
rvp {-q|-s|-v|-h} {schema.rnc}
The options are:
-q
returns only error numbers, suppresses messages;
-s
takes less memory and runs slower;
-v
prints current version;
-h
displays usage summary and exits.
To assist embedding RVP, samples in Perl (tools/rvp.pl) and Python
(tools/rvp.py) are provided. The scripts use Expat wrappers for each
of the languages to parse documents; they take a Relax NG grammar (in
the compact syntax) as the command line argument and read the XML from
the standard input. For example, the following commands validate
rnv.dbx against docbook.rnc:
perl rvp.pl docbook.rnc < rnv.dbx
python rvp.py docbook.rnc < rnv.dbx
The scripts are kept simple and unobscured to illustrate the
technique, rather than being designed as general-purpose modules.
Programmers using Perl, Python, Ruby and other languages are
encouraged to implement and share reusable RVP-based components for
their languages of choice.
User-Defined Datatype Libraries
Relax NG relies on XML Schema Datatypes to check validity of data in
an XML document. The specification allows the implementation to
support other datatype libraries, a library is required to provide two
services, datatypeAllows and datatypeEqual.
A powerful and popular technique is the use of string regular
expressions to restrict values of attributes and character data.
However, XML Schema regular expressions must be written as single
strings, without any parameterization; they often grow to several
dozens of characters in length and are very hard to read or debug.
A solution for these problem would be to allow the user to define
custom datatypes and to specify them in a high-level programming
language. The user can then either use regular expressions as such,
employ lex for lexical analysis, or any other technique which is best
suited for each particular case (for example XSL FO datatypes would
benefit from a custom datatype library). With many datatype libraries
eventually implemented, it is likely that a clearer picture of the
right language for validation of data will eventually emerge.
RNV provides two different ways to implement this solution; I believe
that they correspond to different tastes and traditions. In both
cases, a high-level language can be used to implement a datatype
library, the language is not related to the implementation language of
RNV, and RNV need not be recompiled to add a new datatype library.
Datatype Library Plug-in
A datatype plug-in is an executable. RNV invokes it as either
program allows type key value ... data
or
program equal type data1 data2
program is the executable's, name, the rest is the command line; key
and value pairs are datatype parameters and can be repeated. The
program is executed for each datatype in library
http://davidashen.net/relaxng/pluggable-datatypes; if the exit status
is 0 for success, non-zero for failure.
Both RNV and RVP can use pluggable datatypes, and must be compiled
with DXL_EXC set to 1 (make DXL_EXC=1) to support them, in which case
they accept an additional command-line option -d with the name of the
plugin as the argument. An implementation of XML Schema datatypes as a
plugin (in C) is included in the distribution, see xsdck.c. For
example,
rnv -d xsdck xslt-dxl.rnc $HOME/work/docbook/xsl/*/*.xsl
will validate all DocBook XSL stylesheets on my workstation against a
grammar for XSLT 1.0 modified to use RNV Pluggable Datatypes Library
instead of XML Schema Datatypes.
Scheme Datatypes
Another way to add custom datatypes to RNV is to use the built-in
Scheme interpeter (SCM,
http://www.swiss.ai.mit.edu/~jaffer/SCM.html) to implement the
library in Scheme, a dialect of Lisp. This solution is more flexible
and robust than the previous one, but requires knowledge of a
particular programming language (or at least desire to learn it, and
the result is definitely worth the effort).
To support it, SCM must be installed on the computer, and RNV or RVP
must be compiled with DSL_SCM set to 1 (make DSL_SCM=1), in which case
they accept an additional option -e with the name of a scheme program
as an argument. The datatype library is bound to
http://davidashen.net/relaxng/scheme-datatypes; a sample
implementation is in scm/dsl.scm. For example,
rnv -e scm/dsl.scm xslt-dsl.rnc $HOME/work/docbook/xsl/*/*.xsl
check the stylesheets against an XSLT 1.0 grammar modified to use an
RNV Scheme Datatypes Library implemented in scm/dsl.scm.
A Datatype Library in Scheme must provide two functions in top-level
environment:
(dsl-equal? string string string)
and
(dsl-allows? string '((string . string)*) string)
To assist development of datatype libraries, a Scheme implementation
of XML Schema Regular Expressions is included in the distribution as
scm/rx.scm. The Regular Expression library is not just a way to
re-implement the built-in datatypes. Owing to flexibility of the
language it is much easier to write and debug regular expressions in
Scheme, even if they are to be used with built-in XML Schema Datatypes
in the end. For example, a regular expression for e-mail address, with
insignificant simplifications, is:
pattern=
"(\(([^\(\)\\]|\\.)*\) )?"
~ "([a-zA-Z0-9!#$%&'*+\-/=?\^_`{|}~]+"
~ "(\.[a-zA-Z0-9!#$%&'*+\-/=?\^_`{|}~]+)*"
~ """|"([^"\\]|\\.)*")"""
~ "@"
~ "([a-zA-Z0-9!#$%&'*+\-/=?\^_`{|}~]+"
~ "(\.[a-zA-Z0-9!#$%&'*+\-/=?\^_`{|}~]+)*"
~ "|\[([^\[\]\\]|\\.)*\])"
~ "( \(([^\(\)\\]|\\.)*\))?"
which, even split into four lines, is ugly-looking and hard to read.
Meanwhile, it consists of a few repeating subexpressions, which could
easily be factored out, but the syntax does not have the means for
that.
Using Scheme interpreter, it is as simple as
(define addr-spec-regex
(let* (
(atom "[a-zA-Z0-9!#$%&'*+\\-/=?\\^_`{|}~]+")
(person "\"([^"\\\\]|\\\\.)\"")
(location "\\[([^\\[\\]\\\\]|\\\\.)*\\]")
(domain (string-append atom "(\\." atom ")*")))
(string-append
"(" domain "|" person ")"
"@"
"(" domain "|" location ")")))
This code is much simpler to read and debug, and then the parts can be
joined and added to the grammar for production use. Furthermore, it is
easy to implement the parsing of structured regular expressions
embedded into parameters of datatypes in Relax NG itself. dsl.scm, the
sample datatype library, can handle parameter s-pattern with regular
expressions split into named parts, and the example above becomes:
s-pattern="""
comment = "\(([^\(\)\\]|\\.)*\)"
atom = "[a-zA-Z0-9!#$%&'*+\-/=?\^_`{|}~]+"
atoms = atom "(\." atom ")*"
person = "\"([^\"\\]|\\.)*\""
location = "\[([^\[\]\\]|\\.)*\]"
local-part = "(" atom "|" person ")"
domain = "(" atoms "|" location ")"
start = "(" comment " )?" local-part "@" domain "( " comment ")?"
"""
addr-spec-dsl.rnc is included in the distribution.
New versions
Visit http://davidashen.net/ for news and downloads.