buxlabs/abstract-syntax-tree

Comments Question

drolsen opened this issue · 5 comments

drolsen commented

Forgive me if is a dumb question (as I'm new to AST), but are comments on the roadmap anytime soon as something to also include in the parse tree ?

I'm absolutely in love with this in combination with a auto documentation setup I have in my project style-guide, but without comments the lack of contextual descriptions is sorely missed.

emilos commented

hey @drolsen there are no stupid questions :) the library depends on cherow which creates an estree compatible abstract syntax tree. Abstract syntax trees do not hold comments by default as it can be considered as a optional input that's not necessary for valid program execution (same for additional whitespaces, semicolon existence etc.).

Concrete syntax tree can in theory hold comments as nodes (please note that this is not trivial as comments can be part of function arguments etc.). There's an ongoing discussion to include comments as nodes in estree here.

You could probably extract the comments in a different way (e.g. with some simple parser), but I'm not sure if it fits your use case.

drolsen commented

Thank you very much.
This gets me pointed in the right direction. You are correct, it looks like I'm going to have to extract the comments from the source prior to parsing; its putting it contextually back together that is going to be the battle I can see.

Thanks again for your time!

drolsen commented

And I'm back! :)
@emilos Would you mind updating the README to note both use of cherow, and that you pass standard cherow options to the parse method (https://github.com/buxlabs/abstract-syntax-tree/blob/master/src/parse.js#L3).

[solved]
Looks like with the loc: true option enabled, cherow includes line numbers! This means when used in conjunction with extract-comment module, we have an interesting opportunity.

Because cherow removes comments before parsing, the line numbers cherow reports happens to be the same line numbers extract-comment module reports (so long as intended comments sit directly above methods, objects, variables etc). With a simple cross object check of matching line numbers, I was able to capturing comments in context of AST.

emilos commented

@drolsen please feel free to create a PR, a note in API > Static Methods > parse would be great :)

  • 👍1
mycarrysun commented

And I'm back! :)
@emilos Would you mind updating the README to note both use of cherow, and that you pass standard cherow options to the parse method (https://github.com/buxlabs/abstract-syntax-tree/blob/master/src/parse.js#L3).

[solved]
Looks like with the loc: true option enabled, cherow includes line numbers! This means when used in conjunction with extract-comment module, we have an interesting opportunity.

Because cherow removes comments before parsing, the line numbers cherow reports happens to be the same line numbers extract-comment module reports (so long as intended comments sit directly above methods, objects, variables etc). With a simple cross object check of matching line numbers, I was able to capturing comments in context of AST.

I see you said you solved it in terms of capturing the comments - but were you able to generate the source again with the comments in-tact?