microsoft/tolerant-php-parser

Any plans for an official API to convert from Microsoft\PHPParser\Node to \ast\Node (nikic/php-ast) and/or \PhpParser\Node (nikic/php-parser)?

TysonAndre opened this issue · 4 comments

Related to #47 and #36 , didn't see anything mentioning this when searching issues.

A lot of existing tooling uses one of these two parsers, and there are potential use cases for being able to parse syntactically invalid files.

Examples of potential use cases:

  • print code style issues in addition to syntax errors, without a developer having to fix the file and run a check again. (e.g. phpcs?)
  • other static analysis tools being able to check for issues such as undefined methods, calling functions with invalid param types, etc. in other parts of syntactically invalid files
  • other code for autocompletion based on nikic/php-parser

Or, do you recommend creating a separate project depending on tolerant-php-parser?


Aside: if existing tooling needs to know something about any incomplete expressions/statements, adding a property such as $node->tolerantAST to the derived data structures may help

Good question. No official plans at the moment, but we're currently working with @felixfbecker to port php-language-server to this parser, and after that work is complete, it will be easier for us to understand the tradeoffs of such an approach.

but we're currently working with @felixfbecker to port php-language-server to this parser

php-language-server has been ported, it seems.

On an unrelated note, I've been working on https://github.com/TysonAndre/php-parser-to-php-ast/ for converting PHP-Parser to php-ast (tests pass, but there's probably some uncommon syntax I've missed), which would help with converting to php-ast if tolerant-php-parser to PHP-Parser conversion existed.

https://github.com/TysonAndre/php-parser-to-php-ast is 90% (Edit: 98%) done. Test cases aren't comprehensive, there's still bugs in what remains.

Two modes of operation are planned (Edit: and implemented):

  1. Omitting tokens (and acting as though semicolons, etc. existed) so that the result is a valid AST (E.g. converting $x = $a + to $x = $a;), so that code using it can analyze a substitute valid AST.

    This is my main use case.

  2. Adding placeholders (e.g. converting $x = $a + \__INCOMPLETE_EXPR__;), so that code using it could try to manually handle those placeholders.

Still haven't decided how/if features such as autocompletion would work if I proceeded (E.g. should there be an option to return an SplObjectStorageMap mapping an ast\Node to the corresponding Microsoft\PhpParser\Node/Tokens (or adding a property such as $node->tolerantAST to the derived data structures may help, but plugins and node dumpers may misbehave)? Should the ast\Node start including a column/byte offset?)

https://github.com/TysonAndre/php-parser-to-php-ast is in a usable state. The question asked in the original issue has been answered.

Converting to nikic/php-parser's nodes is feasible, but would be very time consuming to implement, and I don't have use cases for PhpParser\Node.