ECMAScriptParser

Champions: Finding one...

Author: Jack Works

Stage: N/A

This proposal describes adding an ECMAScriptParser to JavaScript. Just like DOMParser in HTML and Houdini's parser API in CSS.

The problem and opportunity

Usage A: Check if a piece of code is syntax-correct

Some "feature detection" (See: https://github.com/Tokimon/es-feature-detection/blob/master/syntax/es2019.json) use eval to check if some syntax-level feature is available. Like

try {
    eval('async function x() {}')
    asyncFunctionSupported = true
} catch {}

Or in some other scenarios, we just want to check if a piece of code is syntax-correct instead of eval it.

Usage B: Sandbox

Some sandbox proposal/libraries do need an AST transformer to transform or reject the dangerous JS code.

A sandbox proposal, realms, it's polyfill is using RegExp to reject "HTML style comments" and reject ESModules. Using RegExp is very likely to make false positives on valid codes.

But if they want to reduce false positives, they have to load a babel, typescript or whatever what parser into the library and run it to transform the code into the safe pattern.

Due to the lack of built-in parser, they imported ...

Babel as a parser in realms-shim (Agoric/realms-shim#15)
TypeScript Compiler as a parser in (Agoric/realms-shim#47)
TypeScript as a transformer to transform ESModule into SystemJS (https://github.com/Jack-Works/loader-with-esmodule-example/)

False positive rejection examples:

Usage C: Join the parser step

This part may be hard to accomplish. Need to discuss if it is needed or even possible.

Just like CSS Houdini's parser can parse CSS in a programmable way, this ECMAScript parser may let developer handle the step of parsing the source code in a programmable way.

Don't know what the API should be like, just for example:

const JSXElement = new ECMAScriptParser.Grammar('JSXElement', [
    JSXSelfClosingElement,
    new ECMAScriptParser.Grammar(
        JSXOpeningElement,
        {
            type: JSXChildren,
            optional: true
        },
        JSXClosingElement
    )
])
const jsxParser = new ECMAScriptParser()
const primaryExpression = parser.getGrammar('PrimaryExpression')
primaryExpression.extends(JSXElement)
jsxParser.parse(`const expr = <a />`)

Goal

Generate AST from source code
Generate source code from an AST
[Maybe] a built-in AST walker to replace AST Nodes on demand
(If AST is not plain object,) provide a way to construct new AST Nodes.
[Maybe] a set of API that extends ECMAScript Parser (like we can create a special parser for JSX).

API design

class ECMAScriptParser {
    parse(source: string): ECMAScriptAST

    compile(ast: ECMAScriptAST): string

    static visitChildNodes(mapFunction: (beforeTransform: ECMAScriptAST) => ECMAScriptAST): ECMAScriptAST

    // [Maybe]
    addGrammar(gr: Grammar): this
    getGrammar(grName: string): Grammar | null
    replaceGrammar(gr: Grammar): this
    deleteGrammar(gr: Grammar): this
    removeGrammar(gr: Grammar): this
    static Grammar = class Grammar {
        // [wait for discussion]
    }
}

What shape is type ECMAScriptAST?

wait for discussion

Example usage

Used to check if new Syntax is supported

try {
    new ECMAScriptParser().parse(`
const res = await fetch(jsonService)
case (res) {
  when {status: 200, headers: {'Content-Length': s}} ->
    console.log(\`size is \${s}\`),
  when {status: 404} ->
    console.log('JSON not found'),
  when {status} if (status >= 400) -> {
    throw new RequestError(res)
  },
}`)
} catch {
    // https://github.com/tc39/proposal-pattern-matching
    console.log('Your browser does not support Pattern Matching')
}

Use in Realms API

const parser = new ECMAScriptParser()
const secure = new Realms({
    transforms: [
        {
            rewrite: context => {
                const ast = parser.parse(context.src)
                ECMAScriptParser.visitChildNodes(node => {
                    if (node.kind === ECMAScriptParser.SyntaxKind.WithStatement) {
                        throw new SyntaxError('with statement is not supported')
                    } else if (node.kind === ECMAScriptParser.SyntaxKind.ImportDeclaration) {
                        return ECMAScriptParser.createImportDeclaration(
                            node.decorators,
                            node.modifiers,
                            node.importClause,
                            ECMAScriptParser.createStringLiteral(
                                '/safe-import-transformer.js?src=' + node.moduleSpecifier.toString()
                            )
                        )
                    }
                    return node
                })
                return context
            }
        }
    ]
})

secure.evaluate(`with (window) {}`)
// SyntaxError: with statement is not supported
source.evaluate(`import x from './y.js'`)
// will evaluate: `import x from '/safe-import-transformer.js?src=./y.js'`

[Maybe] Using in enhance ECMAScript's syntax

import { enhanceParser, jsxCompiler } from 'react-experimental-jsx-parser-in-browser'
/*
JSX extends the PrimaryExpression in the ECMAScript 6th Edition (ECMA-262) grammar:
PrimaryExpression :
    JSXElement
    JSXFragment
*/
const jsxParser = enhanceParser(new ECMAScriptParser())
const ast = jsxParser.parse(`const link = <a />`)
const code = jsxParser.compile(
    jsxCompiler({
        jsxFactory: 'React'
    })
)
console.log(code)
// const link = React.createElement('a', {})

Jack-Works/proposal-ecmascript-parser