kotlinx/ast

How to get the entire AST instead of just the summary?

ToineHulshof opened this issue · 10 comments

ToineHulshof commented

Hi!

The title says it basically. How do you get the entire AST, with DefaultAstNodes converted to the Klass Data classes, from a kotlinfile. In the example the following function is invoked:

kotlinFile.summary()

This function does not include the body of a function for example.
I tried to create my own function:

kotlinFile.complete()

private val completeTreeMap: TreeMap = DefaultTreeMap(
    annotationsMapper,
    declarationsMapper,
    expressionsMapper.flattenNames("functionDeclaration"),
    globalsMapper,
    kotlinDefaultMapper,
    modifierMapper,
    stringMapper,
    typesMapper
)

fun List<Ast>.complete(): AstResult<List<Ast>> {
    return completeTreeMap.treeMap(this)
}

fun Ast.complete(): AstResult<List<Ast>> {
    return listOf(this).complete()
}

I tried to change the Mappers in the DefaultTreeMap, but couldn't get a configuration to be able to visit the statements in the body of functions.

Hopefully this is enough information to be able to resolve my problem.

Thanks in advance!

drieks commented

Hi!

Here is a unit test file containing a function inside a class:
https://github.com/kotlinx/ast/blob/master/grammar-kotlin-parser-test/src/commonMain/resources/testdata/Class.kt.txt

This is the "raw" AST:
https://github.com/kotlinx/ast/blob/master/grammar-kotlin-parser-test/src/commonMain/resources/testdata/Class.raw.txt

And this is the result of summary():
https://github.com/kotlinx/ast/blob/master/grammar-kotlin-parser-test/src/commonMain/resources/testdata/Class.summary.txt

So you want to use this part of the summary result, for example of "annotated2":

    KlassDeclaration(fun annotated2 Int)
      KlassAnnotation(Annotated)
      KlassModifier(private, visibilityModifier)
      KlassDeclaration(parameter x Int)
        KlassAnnotation(AnotherAnnotation)

but you also want to access the "raw" AST of this "annotated2" function?
(Lines 294-424:
https://github.com/kotlinx/ast/blob/master/grammar-kotlin-parser-test/src/commonMain/resources/testdata/Class.raw.txt#L294-L424)

It should be possible to attach the raw ast to the summary nodes, I will look into this.

drieks commented

Hi @ToineHulshof,

I added a new "raw" Field to Klass (and therefore all sub classes like KlassDeclaration):

sealed class Klass() : AstGroup {
    abstract val raw: Ast
}

(bc66e2d#diff-169a69e435ce6e0db18b509e932cc6adR6-R8)

You can now call kotlinFile.summary(), look for a KlassDeclaration in the result and call raw on it.

Please let me know if this is your use case. The new code is already located in the master branch.

ToineHulshof commented

Hi @drieks

Thanks for the quick reply! I think this is what I had in mind, although I have not tested it yet, since I am currently accessing your build through Jitpack, which is not updated at the moment. As I am quite new to Kotlin development, I don't know how to use your master branch in another way (except for downloading the entire repo and adding it to my project, which might be a possibility).

drieks commented

Hi @ToineHulshof,

please try version bc66e2da3a using jitpack.

drieks commented

Hi,

I added a new Option:
kotlinFile.summary(attachRawAst = true)

attachRawAst: Boolean is now a required argument for summary.

The Version is c35b50fa44

  • 👍1
miniyoung84 commented

Hello,

I am having the same issue and I turned attachRawAst to true in the Example files and it's not changing the output. Is there more I have to do for it to work?

Thanks

drieks commented

Hi @miniyoung84,

you can see the {{RawAst}} e.g. in the debugger:
image

It is not printed because normally it is very big...

miniyoung84 commented

Hello @drieks ,

Thank you so much for your answer. Sorry if this is a stupid question, but may I ask what debugger is being used there? I'm unable to locate it on mine.

drieks commented

Hi @miniyoung84, the Screenshot was taken from IntelliJ IDEA (https://kotlinlang.org/docs/tutorials/jvm-get-started.html)