stsewd/tree-sitter-rst

Non-local trailing whitespace sensitivity.

Closed this issue · 3 comments

Carreau commented

Take the following that have no trailing whitespace.

::

    [[1., 0., 0.],
     [0., 1., 2.]]

::

    >>> np.set_printoptions(threshold=sys.maxsize)  # sys module should be imported

You could also implement the reduce as follows::

    >>> def ufunc_reduce(ufct, *vectors):

.. seealso::
   :doc:`numpy-for-matlab-users`

It parses to

document [0, 0] - [15, 0]
  literal_block [2, 4] - [3, 18]
  literal_block [7, 4] - [7, 83]
  paragraph [9, 0] - [11, 41]
    literal_block [11, 4] - [11, 41]
  directive [13, 0] - [13, 12]
    name: type [13, 3] - [13, 10]
  paragraph [14, 3] - [14, 32].  # this is wrong
    interpreted_text [14, 3] - [14, 32]
      role [14, 3] - [14, 8]

at 1 trailing whitespace after the first literal block:

::

    [[1., 0., 0.],
     [0., 1., 2.]]
 <- there is a whitespace here now
::

    >>> np.set_printoptions(threshold=sys.maxsize)  # sys module should be imported

You could also implement the reduce as follows::

    >>> def ufunc_reduce(ufct, *vectors):

.. seealso::
   :doc:`numpy-for-matlab-users`

And it parses properly.

document [0, 0] - [15, 0]
  literal_block [2, 4] - [3, 18]
  literal_block [7, 4] - [7, 83]
  paragraph [9, 0] - [11, 41]
    literal_block [11, 4] - [11, 41]
  directive [13, 0] - [14, 32]
    name: type [13, 3] - [13, 10]
    body: body [13, 12] - [14, 32]
      content [14, 3] - [14, 32]

Note: taken from there https://numpy.org/doc/stable/user/quickstart.html

Screen.Recording.2021-12-02.at.11.01.17.PM.mov
Carreau commented

note that

 .. seealso::
-   :doc:`numpy-for-matlab-users`
+    :doc:`numpy-for-matlab-users`

Fixes it as well, and so does adding trailing space on other blanks lines like the line just above see-also

I'm not sure this is intended, and a "it should be consistently indented 3/4 spaces" is a valid answer,

stsewd commented

Thanks for reporting! I think this is a bug with the stack that keeps track of indentation levels

tree-sitter-rst/src/tree_sitter_rst/parser.c

Lines 903 to 907 in a551461

scanner->advance(scanner);
int indent = get_indent_level(scanner);
if (indent != scanner->back(scanner) || !is_adornment_char(scanner->lookahead)) {
scanner->push(scanner, scanner->back(scanner) + 1);
}

probably it shouldn't be +1, but the actual indentation level of the block, or maybe there is a dedent missing.

I'm on vacations now, so I'll put some time to look at the bugs in the next days :D

Carreau commented