Automatic whitespace handling?

[stevefan1999-personal]

Stemming from #305, I think this is a pretty sane issue that is vital to everyone yet nobody talks about it. We can't be putting _ everywhere and I think it is valid to have some kind of automatic rule that consumes whitespace on the fly.

Maybe you can hide it away by using a tokenizer like logos - Rust (docs.rs) but not only this introduces external dependencies but also that you lost some native integrations with peg (for example, string slicing would certainly not work)

[kevinmehall]

It would certainly be possible for the macro to insert a call to a whitespace() rule between each and every concatenated expression. But then you need a way to turn it off for specific rules or subexpressions, because you don't want to allow whitespace in the middle of the lexer-style rules (e.g. between the characters of an identifier). You also need a way to specify where whitespace is required vs optional. The rust-peg syntax for calling the _, __, and ___ rules with minimal syntax is designed to make this as easy as possible with the normal semantics of rules.

Pest has this feature, but pest requires the ~ operator for concatenation, while in rust-peg you just put expressions next to each other with no operator. So you could equivalently say that _ is the rust-peg concatenation operator when you want to allow whitespace and end up with the same number of characters in the grammar as pest. That avoids all the complexity of trying to auto-insert calls to a whitespace rule, and makes it completely explicit.

That seems good enough to me, but if you have a concrete proposal I'd be happy to hear it.