Estree AST or SWC AST
qwertydelle opened this issue · 12 comments
Ok so we might have a problem the AST swc_ecma_parser produces is very different from the EStree compatible AST that acorn uses. The swc_estree_compat library provides a way to transform SWC AST to an EStree compatible AST but it makes overall parsing much slower than just using the SWC AST and it also requires the nightly compiler which is unstable. Should we use the nightly compiler and risk some features being removed, build our own AST transformer or just use the SWC AST?
Using the SWC AST is what I think is the right path but I want to see what other people think.
If we're using SWC for code generation, then it would be better to use their AST I think. There are some custom nodes that Svelte uses, but these are used for compiling the JS AST. I've used the Estree AST as it was similar to the one Svelte uses.
I also think switching to the SWC AST would be a good call.
SWC is the de-facto JavaScript transpiler in the Rust world and I would highly recommend to use it for non-trivial code generation. SWC itself lacks good documentation support (back then I investigated it) and using another layer of translation might worsen the lack of docs given Visitor/VisitorMut is not easy to work with :(
Other alternatives like rslint is less supported, sadly.
@HerringtonDarkholme have you found any good alternatives to Visitor/VistorMut while working on Rusty Vue?
Rusty Vue uses totally different strategy though. It is more like "find and catch" game:
find AST node with certain pattern and replace their text content.
For example, in https://github.com/vuejs/core/blob/main/packages/reactivity-transform/src/reactivityTransform.ts#L472
{ foo } is transpiled to { prop: __props_prop }. The rewrite is implemented in string manipulation based on magic-string.
This approach is more performant (only need parsing) and easy (Visitor pattern is quite pedantic I would say).
The cons is that it might not be composable because different rewriting cannot be applied at the same time. So things like transpiling ES6 to ES5 is hard for such approach.
Whether it is good approach depends on svelte's compilation output.
I implemented my own walker to walk through the AST and get all the values that will update. Check out src::compiler::analyse::extract_variables_that_change. Now I can generate a .js file but I need to work on getting reactivity working. Check out the rustle::tests folder.
(DISCLAIMER: shameless promotion)
I have a spin-off project https://github.com/ast-grep/ast-grep for finding/replacing AST nodes that fits Vue's reactivity transformation.
ast-grep uses tree-sitter to parse both JS/TS/CSS so that I can have a unified interface to manipulate source code. It also uses a pattern to find AST node in easier way.
It might also help Svelte compilation. If you are interested I will help any issue you have when integrating the tool.
That sounds exactly like something I was looking for. Does it work with the swc_ecma_ast? I need something that looks through an AST and finds all the variables that will change for example UpdateExpr. Then to make the output reactive, it needs to change a function like
let counter = 0;
const increment = () => counter++;into something like
let counter = 0;
const increment = () => (counter++, lifecycle.update(["counter"]));It does not work with swc_ecma_ast sadly since it is backed by tree-sitter.
However I tried it with ast-grep based on YAML (I can also implement it in Rust but YAML is easier to experiment).
The result is something like below:
With this rule:
rule:
all:
- pattern: $B
- any:
- pattern: $A++
- pattern: $A--
- pattern: ++$A
- pattern: --$A
- pattern: $A = $
- pattern: $A += $
- pattern: $A -= $
fix: ($B, lifecycle.update(["$A"]))
I have implemented playground support for trying ast-grep! See the above example in the following link
Sorry for the late response, I've been quite busy with other stuff. The playground looks awesome, thanks for the example! I'll look into ast-grep to see if/how we could use it in rustle!
For now I've implemented a custom AST walker for the swc_ecma_ast which we should use instead of swc_estree_ast. I want the project to use the least amount of dependencies as it's possible. Maybe in the future we'll maybe need to use ast-grep if we're going to need more complex transformations.