samuelgoto/proposal-block-params

Alternative strawman

dead-claudia opened this issue · 8 comments

dead-claudia commented

I decided to take a bit to sketch out an alternative strawman, to see what I could do. Key takeaways:

  • Coming up with a concise, well-defined syntax and semantics set is hard. As if that wasn't already obvious, though... 😄
  • I had to create an additional implicit context channel to avoid this conflicting issues. It was much better than introducing this footgun: forEach(array) do (item) { this.consume(item) }.
  • I had to figure out a better system to solve the scoping issue, and I decided to settle on no sigil on top-level DSLs, @ for dependent DSLs, and tentatively :: for top-level control flow DSLs and : for dependent control flow DSLs. I know the former could conflict with decorators and the latter with labels, but I made mention of how to address it (as in, decorators and labels come first, respectively).
  • Managing control flow is incredibly hard, even at the conceptual level. It's bad enough to just incorporate synchronous, immediate values into the mix, but add the ability to suspend the context, and I quickly had to jump to coroutines, just to have a sane execution model for the DSLs themselves.
  • Managing completions without reifying them is pretty difficult to come up with solutions for. It wasn't exactly a simple thought process to go from reified completions to just using syntax. I used try/catch as a base because they already deal with abrupt completions.
  • I tried dealing with the exact completions in the spec when modeling the control flow DSLs, but I found a few complications and glitches with that reasoning (and why I opted to model my "inline completions" based on outside behavior instead):
    • break could mean the DSL itself, or it could mean an outer loop.
    • continue is really just a glorified block-level return, so there's little point in discerning them.
    • return breaks out of the loop and triggers the same exit sequence as break does.
    • throw can work a lot like return, but it can also just be an error handleable by the DSL itself.
    • Sometimes, you want to short-circuit and return a value for the DSL's callee, like in the select example here in this repo's README. This can't be modeled with a completion in terms of the current spec.

Here's a few other features specific to my rendition:

  • The implicit scope is accessible, and I do make it possible to read from and write to members of the context, to make it much more transparent and user-friendly.
  • I also took into account the possibility of computed properties, and tried to support that use case.
  • I found it easier to treat control flow arguments as thunks, and similarly, I made the syntactic variant use call-by-name, rather than call-by-value.

/cc @samuelgoto @rwaldron

samuelgoto commented
  • 👍1
samuelgoto commented

Wow, lots of good stuff in this alternative exploration. Let me try to unpack and discuss piece by piece (not sure what's the best way to do this, but lets give it a try).

Here are some of the ideas that I really liked but needs further clarification.

@@this, : and ::

This seems like a very interesting idea, but I'm not sure we are thinking of the same thing. Let me try to get some clarity here.

Blocks close over environment, but have phantom @@this context (syntax error outside DSL)

Is @@this something that is supposed to be used by the user? That is, does the user ever make a reference to @@this or sets value in @@this? If so, could you give me a concrete example on how @@this is used?

I had to figure out a better system to solve the scoping issue, and I decided to settle on no sigil on
top-level DSLs, @ for dependent DSLs, and tentatively :: for top-level control flow DSLs and : for
dependent control flow DSLs.

I'm not sure how you are defining "top-level DSLs" and "dependent DSLs", but it sounds a little awkward that you have to make the distinction between the two at a syntax level (i.e. use : for one and :: for the other).

I'm not sure what to call these, but I've seem a pattern that I agree that needs to be named / better understood.

  • foreach
  • select/case

On one hand, foreach should be able to be embedded inside any block as it is applicable anywhere (i.e. a foreach is meaningful in any context). when on the other hand, makes only sense inside of a select block.

I think that, perhaps, the key distinction here is how to resolve the identifiers. In the select / when example, it seems like it would be constructive to resolve when within an object that is passed by select. For example:

select (foo) {
  when (bar) { // there needs to be somehow a connection between when and select
    foreach ([1, 2, 3]) do (item) { // foreach, on the other hand, is de-coupled from everything else
      console.log(item)
    }
  }
}

Is this the problem that you are trying to solve with with @@this, : and ::? If so, could you specifically give me an example of what that could look like for the example above? Is the idea that would would write:

::select (foo) {
  :when (bar) { // : desugars to this.when(bar, function() { ... })
    ::foreach ([1, 2, 3]) do (item) { // :: desugars to foreach([1, 2, 3], function() { ... })
      console.log(item)
    }
  }
}

If so, why does one need to make a distinction between : and ::? Wouldn't just a single one suffice (i.e. isn't ::foo isomorphic to foo?)? For example:

select (foo) { // by default, desugars to select(foo, function() { ... })
  :when (bar) { // : desugars to this.when(bar, function() { ... })
    foreach ([1, 2, 3]) do (item) { // by default, desugars to foreach([1, 2, 3], function() { ... })
      console.log(item)
    }
  }
}

Pushing this even further, if we reverted the semantics between : and :: in your formulation, it would align with the bind operator proposal (this is somewhat the argument/conclusion that i was trying to make with this example). That is:

select (foo) { // by default, desugars to select(foo, function() { ... })
  ::when (bar) { // :: desugars to this.when(bar, function() { ... })
    foreach ([1, 2, 3]) do (item) { // by default, desugars to foreach([1, 2, 3], function() { ... })
      console.log(item)
    }
  }
}

Does that make sense?

Just so that I understand, the introduction of : and :: is due to a performance optimization compared to (b ? this : this.b : b)() which has with-like characteristics?

select (foo, function() {
  (this.when ? this.when : when) (bar, function() {
    (this.foreach ? this.foreach : foreach) ([1, 2, 3], function (item) {
      console.log(item)
    })
  })
})

Separately, what's the role of @@this? How is it different than this? Does it ever get used by the user?

dead-claudia commented

@samuelgoto

(These are answered somewhat out of order, but I've roughly sorted them topically.)

@@this

Is @@this something that is supposed to be used by the user? That is, does the user ever make a reference to @@this or sets value in @@this? If so, could you give me a concrete example on how @@this is used?

[...]

Separately, what's the role of @@this? How is it different than this? Does it ever get used by the user?

The reason @@this exists is because:

  1. I didn't want to leave a leaky abstraction
  2. It is a separate thing from this, but it still context-like.

The key difference is that @@this is the DSL's context, while this is the function's context. Here's how that difference pans out, using normal DSLs:

function call(obj, func) {
    return func.call(obj)
}

class Class {
    method() {
        const obj = {}
        const self = this

        call(obj) do {
            // This is the main distinction.
            assert(this === self)
            assert(@@this === obj)
        }
    }
}

Hierarchy and distinctions

I'm not sure how you are defining "top-level DSLs" and "dependent DSLs", but it sounds a little awkward that you have to make the distinction between the two at a syntax level (i.e. use : for one and :: for the other).

Is this the problem that you are trying to solve with with @@this, : and ::? If so, could you specifically give me an example of what that could look like for the example above?

First, to clarify, there's two different DSL syntaxes:

  • Normal DSLs:

    // Top-level
foo do {
    // Dependent
    @bar do {
        // ...
    }
}

// Transpiled
foo(function () {
    this.bar(function () {
        // ...
    })
})

  • Control flow DSLs:

    (Note: $execValue and $invoke are defined in the "Common helpers for below" section of this section.)

    // Top-level control flow DSLs
::foo do {
    // Dependent
    :bar do {
        // ...
    }
}

// Transpiled
let _completion$ = $execValue($invoke(foo, void 0, function *() {
    return yield {type: "call", value: $invoke(this.bar, this, function *() {
        // ...
    })}
})
if (_completion$.$type === "break") break
if (_completion$.$type === "abort") return _completion$.$value

Now to the original question regarding the distinction, it's something @rwaldron brought up here and I've brought up here. with uses dynamic scope lookup, because it conflates object properties with lexical variables. If a variable doesn't exist in the local scope, it has to check the global scope before it can check the object, so if you drop the sigil, it becomes like this at runtime (for the first case):

// Original
foo do {
    bar do {
        // ...
    }
}

// Transpiled
foo(function () {
    (typeof bar !== "undefined" ? bar : this.bar.bind(this))(function() {
        // ...
    })
})

That's not exactly a simple thing to compile, and engines can't easily elide the branch for non-globals - they'd have to keep a separate registry of all non-top-level DSL locations in the realm to properly patch it, and that would quickly get out of control in terms of memory, especially for heavy users (think: vdom, larger frameworks).

Pushing this even further, if we reverted the semantics between : and :: in your formulation, it would align with the bind operator proposal (this is somewhat the argument/conclusion that i was trying to make with this example). That is:

I understand, and I'm not too attached to the choice of sigils themselves. Just keep in mind, you may need to retain some sort of distinction as explained in the next section.

Control flow

I'm not sure what to call these, but I've seem a pattern that I agree that needs to be named / better understood.

[...]

Just so that I understand, the introduction of : and :: is due to a performance optimization compared to (b ? this : this.b : b)() which has with-like characteristics?

The key reason I introduced :: (for top-level) and : (for dependent) is for lexical analysis reasons. Here's why: they are not merely normal functions anymore, but are instead tasked with propagating control flow. They are required to admit non-local control flow tasks, and so you have two issues:

  1. Normal DSLs like a theoretical defer(ms) do { ... } or process.nextTick do { ... } (which would work today) cannot sensibly deal with control flow of any sort.
  2. Their inner workings have to be substantially different - they can't just be normal functions anymore. Consider for (const item of coll) ::unless(cond(item)) do { break } as an example.

So as a result, you can't call the same DSL both ways, and the question really becomes two-fold:

  1. How do you deal with syntactically modeling non-local control flow within parameters/callbacks?
  2. How do you deal with runtime modeling of control flow propagation within the DSLs themselves?

For each of these, you have two primary routes you can take. For the first:

  1. You could require a sigil or some other discriminating token or sequence to differentiate simple DSLs.
  2. You could parse it and compile two separate code paths, and move the "unexpected control flow operator" syntax error to runtime, waiting to check the callee's type first. (This requires that control flow DSLs are discernable from normal DSLs in some fashion, be it a symbol, an internal slot, etc.)

In this case, the first route is considerably simpler to implement, and fits in better with the rest of the language.

For the second (which I covered in more detail):

  1. You could force suspension points when parsing parameters (remember: eval("break") is a thing) and callbacks, and you could have them and the DSL return pseudo-completion values.
  2. You could define special syntax for defining DSLs that can handle control flow and have special call syntax.

If so, why does one need to make a distinction between : and ::? Wouldn't just a single one suffice (i.e. isn't ::foo isomorphic to foo?)?

select (foo) { // by default, desugars to select(foo, function() { ... })
:when (bar) { // : desugars to this.when(bar, function() { ... })
foreach ([1, 2, 3]) do (item) { // by default, desugars to foreach([1, 2, 3], function() { ... })
console.log(item)
}
}
}

Yes for the most part, but the distinction to be made is select(foo) do { ... } vs ::select(foo) do { ... }, and they have completely different call sequences, if you look at the transpiled equivalents for the normal vs control flow DSLs. In particular, normal DSL blocks transpile to simple functions, while control flow DSLs use coroutines under the hood. (An implementation might choose to use a stackful coroutine system to better optimize this if the syntactic variant is chosen.)

One last thing: the control flow side of things can easily be punted for now, since it's a much more complicated beast than the simple DSLs initially proposed. Even Kotlin only supports non-local returns (not break or continue), so that's a thing to take into account.

samuelgoto commented

Lots of good observations again, let me try to break things down and comment things separately.

One last thing: the control flow side of things can easily be punted for now, since it's a much more
complicated beast than the simple DSLs initially proposed. Even Kotlin only supports non-local
returns (not break or continue), so that's a thing to take into account.

I think that's a fairly reasonable route to take too. I do actually think that dealing with non-local abrupt termination (e.g. break and continue and return) could be looked at separately (i.e. purely as a sequencing strategy).

Let me dive into that route for a second.

What if we used the block param's parameters list to pass @@this around through a reserved Symbol (e.g. Symbol.parent, e.g. {[Symbol.parent]: REFERENCE}) and created :: as a short hand to access it?

For example:

foo {
  ::bar {
  }
}

As this:

// all block params takes as an argument an object that can
// contain a @@this context passed in via a Symbol.parent
// key.
foo (({[Symbol.parent: @@this]}) => {
  // ::method turns into @@this.method()
  @@this.bar (({[Symbol.parent]}) => {
  })
})

So that, for example, foo could pass bar as the following:

function foo(block) {
  block({[Symbol.parent]: {
    bar(inner) {
      inner()
    }
  }});
}

So, back to our canonical example:

select(foo) {
  ::when(bar) {
  }
}

Gets transpiled to:

select(foo, ({[Symbol.parent]: @@this1}) => {
  @@this.when(bar, ({[Symbol.parent]: @@this2}) => {
    // ...
    // break and continue throw SyntaxError here
  })
})

And maybe nesting could be done through @@this like the following:

function select(expr, block) {
  // block is an arrow function, so block.call() sets @@this
  // which can be accessed through ::
  block({[Symbol.parent]: {
      when(cond, inner) {
        if (expr == cond) {
          inner.call(); // guaranteed not to have a break/continue statement
        }
      }
    }
  });
}

WDYT?

dead-claudia commented

Not super keen on it. The reason being, this works fine on the DSL's side, since it's just a simple remapping. The callee's side closes over this, so the DSL can't pass a meaningful this context itself (it's like passing this to an arrow function). However, DSLs have to manage their own context (specifically @@this), so we do have a this-like slot we can use. So it's pretty easy to just repurpose this in the block to act like @@this rather than the normal this. (It also makes DSLs easier to define.)

I think that's a fairly reasonable route to take too. I do actually think that dealing with non-local abrupt termination (e.g. break and continue and return) could be looked at separately (i.e. purely as a sequencing strategy).

To clarify, that's why I factored out the control flow idea out into a separate file from the main proposal.

Thought I'd correct you with the syntax of my proposal: there's an extra do between the name and arguments.

// Correct
foo(...args) do {
    // ...
}

// Incorrect
foo(...args) {
    // ...
}

The reason for the extra keyword is to avoid future hostility, to avoid ambiguity with parameterized blocks, and to avoid forcing a particular brace style.

// Conflicts with the current pattern matching proposal
match(foo) {}

// Currently a function call + block
foo(...args)
{
    // ...
}

// This would *not* be a problem
foo(...args) do
{
   // ...
}

// Is this a parameterized block or a double call expression?
foo(bar) (baz) {
    // ...
}

// This would *not* be a problem
foo(bar) do (baz) {
   // ...
}
samuelgoto commented

I'm getting increasingly excited about the approach that I outlined because it addresses a consistent feedback that I've been getting that messing with this isn't a great idea.

I tried to write down a simpler version of my explanation here and started prototyping it with the transpiler here.

I'm going to update the text of the proposal to reflect this formulation, but I think it is a step forward in the right direction (albeit it doesn't address yet break/continue/yield/await, but like I said earlier, I think we should look at nesting/scoping separately from abrupt interruption).

samuelgoto commented

I'm going to update the text of the proposal to reflect this formulation

done.

https://github.com/samuelgoto/proposal-block-params
https://gitpitch.com/samuelgoto/proposal-block-params

dead-claudia commented

@samuelgoto Don't forget to add the do for parameter-less blocks, as explained in my last comment (starting from "Thought I'd correct you with the syntax of my proposal"). I noticed that you missed that part with your update.