katef/libfsm

negative character classes produce an ABNF output that kgt does not parse

classabbyamp opened this issue · 4 comments

Not sure if this should be here or on katef/kgt.

When converting a regex that includes a negative character class to ABNF form, it generates the output fine, but when passing this to kgt, it fails with 1:11: Syntax error: expected production rule separator

Minimal Example:

$ re -bpl abnf '[^0-9]' | tee /dev/stderr | kgt -l abnf -e svg | isvg
e = OCTET - %x30-39

1:11: Syntax error: expected production rule separator
...  # errors continue from other parts of the pipeline
katef commented

Ah thank you!

I think this is valid ABNF output, and re(1) is doing the right thing here (although of course it could be written more compactly without the subtraction).

However kgt doesn't implement subtraction. It really ought to give a more helpful error message than "syntax error". The reason it's not implemented, is that subtraction for CFGs in general isn't well defined! It's bewildering that it's part of the ABNF spec.

Any suggestions for how to proceed?

Not a clue. You know a lot more about all of this than I do, I just stumbled on this as I was messing around.

Sorry for potentially making this worse, but isn't subtraction only defined in ISO EBNF, rather than ABNF? If so, that would mean there's both an issue in libfsm for using subtraction in ABNF output, and an issue in kgt for not handling subtraction in ISO EBNF input.

For the libfsm side: the libfsm parsers only generate trivial subtractions where both the LHS and RHS can only ever match a single character, right? If so, would it make sense to (re)define the subtraction operator to have that as a hard requirement? The printers could then make use of that to transform them to avoid the subtraction by rewriting [^0-9] as [\x00-/:-\xFF] where subtraction is not supported, or actually write out [^0-9] where subtraction is supported.