Wrong line number for multiline strings with variables
quasilyte opened this issue · 0 comments
quasilyte commented
I believe there is a regression bug introduced in 2c64915 e4a208e (#93) that is still present on master branch.
Problematic code example:
$foo = "test
$var";
That 2-line literal would be scanned as 3-line literal, its StartLine=1 and EndLine=3.
I believe this happens because we emit newlines twice:
- Once for the
constant_string_new_line
actions. - And another time for
newline
.
We probably want to avoid adding redundant newline in either of these places.
I tried to come up with a good solution today but my knowledge of Ragel limited.
Full Reproducer (in form of test)
// add this test to node/scalar/t_encapsed_test.go and run it.
func TestEncapsedMultiline(t *testing.T) {
src := `<? "test
$var";`
expected := &node.Root{
Position: &position.Position{
StartLine: 1,
EndLine: 2,
StartPos: 3,
EndPos: 16,
},
Stmts: []node.Node{
&stmt.Expression{
Position: &position.Position{
StartLine: 1,
EndLine: 2,
StartPos: 3,
EndPos: 16,
},
Expr: &scalar.Encapsed{
Position: &position.Position{
StartLine: 1,
EndLine: 2,
StartPos: 3,
EndPos: 15,
},
Parts: []node.Node{
&scalar.EncapsedStringPart{
Position: &position.Position{
StartLine: 1,
EndLine: 2,
StartPos: 4,
EndPos: 10,
},
Value: "test\n\t",
},
&expr.Variable{
Position: &position.Position{
StartLine: 2,
EndLine: 2,
StartPos: 10,
EndPos: 14,
},
VarName: &node.Identifier{
Position: &position.Position{
StartLine: 2,
EndLine: 2,
StartPos: 10,
EndPos: 14,
},
Value: "var",
},
},
},
},
},
},
}
php7parser := php7.NewParser([]byte(src), "7.4")
php7parser.Parse()
actual := php7parser.GetRootNode()
assert.DeepEqual(t, expected, actual)
php5parser := php5.NewParser([]byte(src), "5.6")
php5parser.Parse()
actual = php5parser.GetRootNode()
assert.DeepEqual(t, expected, actual)
}
It seems like this problem described in Maintaining char & line counts in a scanner response:
The only worry is backtracking. If your
scanner patterns backtrack over newlines, then you've got double
counting happening.