overlookmotel/livepack

Two-phase serialization

Opened this issue · 1 comments

Initially discussed in #169 (comment)

What

Serialization should be broken up into 2 phases:

1. Trace: Construct a "program graph"

"Program graph" is similar to an "abstract syntax tree" (AST), except it represents values, rather than code syntax. "Tracing" is analogous to the "parse" phase of Babel.

In this phase, every value referenced within the program is converted to a "record" representing its type, properties, and dependencies ("record" is analogous to an AST "node").

The entire graph of dependencies (properties of objects, scope of functions) are traced from program entry point (tree root), recursively, until all values reachable by the program are visited and added to the graph.

Unlike an AST, a program graph may be cyclic, and each record may have multiple dependencies and dependents upon other records.

2. Serialize

The serialize phase takes the program graph and converts to a JS AST, which is then printed as JS code.

Why

Currently tracing and serialization are performed together in a single pass.

Splitting the process into 2 phases would:

  1. Allow tracing in any order, which enables various optimizations
  2. Make Livepack's codebase more readable
  3. Allow plugins to alter the program graph between the two phases: (similar to how Babel plugins transform the AST)

Progress

Some initial work on two-phase branch.

Most recent iteration of implementation on two-phase3 branch.

This implementation covers almost everything except functions and code splitting (the hardest parts!).