A simple library to abstract the gritty process of lexical parsing.
This package is available on nuget.
For extended samples, see /samples/.
- Symbols are always parsed as single units. That is, a token containing more than one symbol will never be generated by the tokenizer or lexer. The same goes for whitespace (CRLF is an exception).
- LexerTokenType.AlphaNumeric will always contain alphabetical characters. It may or may not contain numeric characters, and will never be purely numeric.