SwiftStudies/OysterKit

Refactor token streaming

Closed this issue · 0 comments

At the moment there is little difference in implementation (and therefore potentially expected memory/performance profiles as well as behavioural characteristics) for streams, homogenous and heterogenous ASTs.

These areas could be refactored to improve

  • Ease of consumption
  • Provided the expected benefits (lazy and therefore low memory consumption)

In order to do this the following should be provided:

  • Streaming should identify "trigger" tokens that will be forwarded, but no others. Consideration will have to be given to range management on the node stack (or a new equivalent) but nodes should not be created
  • There is an attempt to leverage commonality in behaviour between homogenous and heterogenous AST generation. This should be preserved but without the complexity of having to provide an IR and a constructor and still being left in a situation where a lot of casting has to be done