alex/rply

Add support for "|" inside productions

plafer opened this issue · 5 comments

plafer commented

Cleans up the use case where one has many small productions. For example,

@pg.production("prod: TOKEN_1")
# ... <24 lines>
@pg.production("prod: TOKEN_26")
def func(p):
    pass

Would be simplified to

@pg.production("prod: TOKEN_1 | ... | TOKEN_26")
def func(p):
    pass

More specifically, I need this right now, as I need to accept (and throwaway) all tokens until I see a newline.

I'd be happy to submit a PR if you guys think it's a valuable feature to have.

alex commented
nobodxbodon commented

We have a production rule bin_expr : expr | expr where | stands for BitOr, which was fine with 0.7.7 but broken on 0.7.8. It seems to be caused by this change. I wonder if there's a way to get around? Some kind of escaping or else?

leahcornelius commented

Perhaps \ escapes it? If not then add a lexer rule to match | to VERTICLE_PIPE or similar and use that? @nobodxbodon

nobodxbodon commented

@leocornelius thanks for the suggestion. Just tried this test case and it does seem to work:

    def test_arithmetic(self):
        lg = LexerGenerator()
        lg.add("NUMBER", r"\d+")
        lg.add("BITOR", r"\|")

        pg = ParserGenerator(["NUMBER", "BITOR"], precedence=[
            ("left", ["BITOR"]),
        ])

        @pg.production("main : expr")
        def main(p):
            return p[0]

        @pg.production("expr : expr BITOR expr")
        def expr_binop(p):
            return BoxInt(operator.or_(p[0].getint(), p[2].getint()))

        @pg.production("expr : NUMBER")
        def expr_num(p):
            return BoxInt(int(p[0].getstr()))

        lexer = lg.build()
        parser = pg.build()

        assert parser.parse(lexer.lex("1|8")) == BoxInt(9)
leahcornelius commented

See #101, it should now be fixed :)