tefra/xsdata-w3c-tests

On running W3C schema tests

brunato opened this issue · 2 comments

brunato commented

Hi,

I saw that you use xmlschema for testing XSD 1.1 schemas but not for XSD 1.0 schemas, where you preferred lxml's XMLSchema validator because it produces less false positives. I've not deepen what do you consider as a false positive case but running your W3C tests with xmlschema v1.1.2 also for XSD 1.0 seems to be slower, as expected, but slightly better than using the lxml validator.

Also my test script for running W3C XSD tests reports a succesful rate that is near the 100% for schema tests and more than 98% for instance tests. I've also tryed lxml's XMLSchema validator for schema tests but it has hundreds of failures and some cases where it stucks.

For me running the W3C tests was not so straightforward. Sometimes there are pathological cases that require the use of optional parameters. For example the xmlschema's validator is based on a meta-schema that can be shared whenever possible to speed up and to save memory, and this choice has an implication on some of the W3C XSD tests, that has to be run disabling the pre-built metaschema.

Best regards
Davide

tefra commented

Hi @brunato thx for reaching out, at the time I switched to a combination of lxml and xmlschema for the instance validations, I think I noticed some false positives, marking instance cases as successful when they weren't and that scared me a bit.

I took another stub at it here #11 it looks really good

  • 👍1
tefra commented

will close this with a promise to open a few issues I have found in the xmlschema repo.

  • 👍1