heredoc_content doesn't return a token
mjambon opened this issue · 1 comments
mjambon commented
Input:
$ cat heredoc2.rb
query = <<-SQL
SELECT * FROM #{table}
WHERE #{type} = true
SQL
The current tree-sitter parse
doesn't show tokens:
$ tree-sitter parse heredoc2.rb
(program [0, 0] - [4, 0]
(assignment [0, 0] - [0, 14]
left: (identifier [0, 0] - [0, 5])
right: (heredoc_beginning [0, 8] - [0, 14]))
(heredoc_body [0, 14] - [3, 3]
(heredoc_content [0, 14] - [1, 14])
(interpolation [1, 14] - [1, 22]
(identifier [1, 16] - [1, 21]))
(heredoc_content [1, 22] - [2, 6])
(interpolation [2, 6] - [2, 13]
(identifier [2, 8] - [2, 12]))
(heredoc_content [2, 13] - [3, 0])
(heredoc_end [3, 0] - [3, 3])))
The new output (work in progress https://github.com/ahlinc/tree-sitter/pull/4) show a missing token for heredoc_content
:
We'd like to get its content in the CST like the other tokens (leaves of the tree) e.g. like heredoc_end
=SQL
.
mjambon commented
I was mistaken here. It's just that multiline tokens aren't printed out by the tool yet.