Implement external tree-sitter scanner for handling eof

[silvanshade]

Implement external tree-sitter scanner for handling eof

[silvanshade]

I went ahead and implemented this. However, after experimenting with it, it doesn't seem like we actually need it.

The reason I thought we did is because something was causing the server to stop parsing documents when there were trailing comments. I thought maybe this was some weird interaction with how we were handling whitespace (explicitly, i.e., overriding extras).

But whatever was causing that problem seems to have gone away, so I'll close this.

Here is the implementation of the scanner in case the issue resurfaces:

#include <tree_sitter/parser.h>

enum TokenType
{
  EOF
};

extern "C"
{
  const void *const tree_sitter_wat_external_scanner_create()
  {
    return NULL;
  }

  void tree_sitter_wat_external_scanner_destroy(const void *const payload)
  {
  }

  void tree_sitter_wat_external_scanner_reset(const void *const payload)
  {
  }

  unsigned tree_sitter_wat_external_scanner_serialize(const void *const payload, char *const buffer)
  {
    return 0;
  }

  void tree_sitter_wat_external_scanner_deserialize(const void *const payload, char *const buffer, unsigned const length)
  {
  }

  const bool tree_sitter_wat_external_scanner_scan(const void *const payload, TSLexer *const lexer, const bool *const valid_symbols)
  {
    if (valid_symbols[EOF])
    {
      if (lexer->lookahead == 0)
      {
        lexer->result_symbol = EOF;
        return true;
      }
    }
    return false;
  }
}