amazon-science/cceval

It seems only python tree-sitter is used.

starlitsky2010 opened this issue · 2 comments

I found that only in postprocess_code_lines function parser was used.

def postprocess_code_lines(prompt, completion, parser, lang):
    try:
        if lang in ["java", "csharp", "typescript"]:
            return get_bracket_lang_statement(completion)
        elif lang == "python":
            return get_python_one_statement(prompt, completion, parser)
    except Exception as e:
        return completion

Parser in get_python_one_statement is to get whether the current code strings are legal or illegal for relative programming language.
For other language, such as JAVA, Csharp, typescript,we may obtain more convincing data if we also use tree-sitter.
Am I right?

Thanks

Yes, your understanding is right.

Thanks