postTask and scheduling microtasks: can the API make this ergonomic?

[shaseley]

We've heard previously and recently that microtasks can be a problem for developers because they can arbitrarily extend the length of tasks, and that having an easy/ergonomic way to schedule them as macrotasks would be beneficial. Developing postTask seems like a good opportunity to evaluate if there's anything we can do to help, especially through the API itself.

I threw together this proposal based on recent partner conversations and on previous ideas we had when we started thinking about scheduling.

Problem

Microtasks can limit the effectiveness of scheduled code by arbitrarily extending the length of tasks. Often times developers want async/macrotask behavior but end up with synchronous code.

For example:

scheduler.postTask(foo)
 .then((result) => doSomethingExpensive(result))
 .then((result) => doSomethingElseExpensive(result))
 .then((result) => andSoOn(result));

Or with async-await syntax:

(async function() {
  let result = await scheduler.postTask(foo);
  result = await doSomethingExpensive(result);
  result = await doSomethingElseExpensive(result);
  await andSoOn(result);
})();

Both of these examples can be problematic if the microtasks are long, i.e. higher priority work like input doesn't have a chance to run in between functions. Or this might be intended, either for correctness or efficiency.

Proposal

Give developers an ergonomic choice between a microtask and macrotask.

• scheduler.postTask returns a subclass of Promise that has a .schedule() method.
• The .schedule callback runs in a macrotask via scheduler.postTask, with the signal set to scheduler.currentTaskSignal.

Example:

scheduler.postTask(foo, {signal: someSignal})
 .schedule((result) => doSomethingExpensive(result))
 .schedule((result) => doSomethingElseExpensive(result))
 .schedule((result) => andSoOn(result))

Proposal++

Extend native Promises to support .schedule(), which is used by async schedule.

(async function() {
  let result = await scheduler.postTask(foo, {signal: someSignal});

  result = await schedule doSomethingExpensive(result);
  result = await schedule doSomethingElseExpensive(result);
  await schedule andSoOn(result);
})();

The JS layer would need to pass the callback to the agent to schedule the task.

Open Questions

• Should developers be able to pass a signal or priority to .schedule(), or is inheriting scheduler.currentTaskSignal the right thing to do? Passing priority/signal is probably more difficult for the native version.

Alternatives

Non-API Options

Things we could do that don't involve changing API shape:

1. Defer resolving the task's Promise (the one returned by scheduler.postTask) by scheduling a task that resolves the Promise, giving higher priority work a chance to run

2. Add yield points between all microtasks for postTask tasks

These options don't give developers a choice, might lead to poor performance when synchronous behavior is desired, and will likely lead to confusion since behavior differs from existing Promises. This could also potentially make polyfills have different (breaking) behavior, and minimally make polyfilling difficult.

Developer Options

The direct alternative is for developers to use scheduler.postTask for posting macrotasks:

scheduler.postTask(foo, {signal: someSignal})
 .then((result) => scheduler.postTask(() => doSomethingExpensive(result), {signal: someSignal}))
 .then((result) => scheduler.postTask(() => doSomethingElseExpensive(result), {signal: someSignal}))
 .then((result) => scheduler.postTask(() => andSoOn(result), {signal: someSignal}))

Or with async-await syntax:

(async function() {
  let result = await scheduler.postTask(foo, {signal: someSignal});
  result = await scheduler.postTask(doSomethingExpensive(result), {signal: someSignal});
  result = await scheduler.postTask(doSomethingElseExpensive(result), {signal: someSignal});
  await scheduler.postTask(andSoOn(result), {signal: someSignal});
})();

These can be made a little cleaner by adding a schedule() method as shorthand, similar to the proposal:

function schedule(task) {
  return scheduler.postTask(task, {signal: scheduler.currentTaskSignal});
}

(async function() {
  let result = await scheduler.postTask(foo, {signal: someSignal});
  result = await schedule(() => doSomethingExpensive(result));
  result = await schedule(() => doSomethingElseExpensive(result));
  await schedule(() => andSoOn(result));
})();

scheduler.postTask(foo, {signal: someSignal})
 .then((result) => schedule(() => doSomethingExpensive(result)))
 .then((result) => schedule(() => doSomethingElseExpensive(result)))
 .then((result) => schedule(() => andSoOn(result)))

This still isn't as ergonomic as the .schedule() proposal, but maybe it's good enough? And if so, maybe it should be supported natively in the platform?

Another option is to add yield points between functions, using a custom yield() function or the proposed scheduler.yield().

(async function() {
  let result = await scheduler.postTask(foo);
  await scheduler.yield();
  result = await doSomethingExpensive(result);
  await scheduler.yield();
  result = await doSomethingElseExpensive(result);
  await scheduler.yield();
  await andSoOn(result);
})();

The same could be done in Promise chains:

let result;

scheduler.postTask(foo)
 .then((r) => { result = r; return scheduler.yield(); })
 .then(() => doSomethingExpensive(result))
 .then((r) => { result = r; return scheduler.yield(); })
 .then(() => doSomethingElseExpensive(result))
 .then((r) => { result = r; return scheduler.yield(); })
 .then((r) => andSoOn(result));

Something else?

These are just things that can be done in conjunction with postTask, but maybe there are other completely different APIs that could help?

[shaseley]

/cc @domenic to tell me if this is impossible and a terrible idea or if there might be some hope :).

[domenic]

Both of these examples can be problematic if the microtasks are long

I'm a bit confused what is meant by "the microtasks are long" in these examples. Are doSomethingExpensive and friends totally synchronous, with no yielding to the event loop? If so, why are they being called with await? If they are not totally synchronous, then they are probably deferring to the event loop anyway?

scheduler.postTask returns a subclass of Promise that has a .schedule() method.

Promise subclassing has not worked so far; in particular it has bad interactions with await (which you seem to be aware of given the existence of Proposal++). So I'm a bit nervous about this and would wonder if we can explore less disruptive options. For example

Another option is to add yield points between functions, using a custom yield() function or the proposed scheduler.yield().

is good, but you could also shorten it to

  let result = await scheduler.postTask(foo);
  result = await scheduler.yield(doSomethingExpensive(result));
  result = await scheduler.yield(doSomethingElseExpensive(result));
  await scheduler.yield(andSoOn(result));

if you added the syntactic sugar of allowing scheduler.yield() taking a promise. If that's still too noisy, then you could consider creating a new function on the global scope with the same behavior, e.g.

  let result = await scheduler.postTask(foo);
  result = await schedule(doSomethingExpensive(result));
  result = await schedule(doSomethingElseExpensive(result));
  await schedule(andSoOn(result));

which is very close to your Proposal++ but without introducing new syntax or promise subclasses. I think that'd be nicer not just from an implementation/spec complexity point of view, but also by introducing fewer new concepts for web developers to learn; it's just a function call.

[shaseley]

Both of these examples can be problematic if the microtasks are long

I'm a bit confused what is meant by "the microtasks are long" in these examples. Are doSomethingExpensive and friends totally synchronous, with no yielding to the event loop? If so, why are they being called with await? If they are not totally synchronous, then they are probably deferring to the event loop anyway?

Sorry for the confusion. Yes, the idea here is that they are synchronous and don't yield to the event loop. As to why they are being called with await, I was converting the previous example (which just uses .then()) to async-await syntax.

I don't have a great sense of how much library/1P code uses Promises in a synchronous way like this, e.g. for ergonomic reasons, but we've heard anecdotally that long Promise chains that end up being synchronous can be a problem. And there is some data to back this up from traces we've analyzed. For example, looking 99th p-tile of calls to RunMicrotasks shows ~60 microtasks being run (in a single checkpoint!).

scheduler.postTask returns a subclass of Promise that has a .schedule() method.

Promise subclassing has not worked so far; in particular it has bad interactions with await (which you seem to be aware of given the existence of Proposal++). So I'm a bit nervous about this and would wonder if we can explore less disruptive options. For example

As I was writing this up and thinking more about it, I agree that subclassing (especially in this case) seems like a bad idea exactly for that reason. I think if we wanted to pursue something like this it should be done for native Promises.

Another option is to add yield points between functions, using a custom yield() function or the proposed scheduler.yield().

is good, but you could also shorten it to

  let result = await scheduler.postTask(foo);
  result = await scheduler.yield(doSomethingExpensive(result));
  result = await scheduler.yield(doSomethingElseExpensive(result));
  await scheduler.yield(andSoOn(result));

if you added the syntactic sugar of allowing scheduler.yield() taking a promise. If that's still too noisy, then you could consider creating a new function on the global scope with the same behavior, e.g.

  let result = await scheduler.postTask(foo);
  result = await schedule(doSomethingExpensive(result));
  result = await schedule(doSomethingElseExpensive(result));
  await schedule(andSoOn(result));

which is very close to your Proposal++ but without introducing new syntax or promise subclasses. I think that'd be nicer not just from an implementation/spec complexity point of view, but also by introducing fewer new concepts for web developers to learn; it's just a function call.

Yeah, this is what I ended up settling on as probably the best userspace option (I arrived at this too above). I think for async-await syntax it's just as good or better than syntax changes from a developer perspective, and certainly less complex from an implementation/spec point-of-view.

I have mixed feelings on .schedule(foo) vs. .then(schedule(foo)). I was leaning towards the former, but actually I think the latter does read better? But anyways, at this point I don't think the complexity vs. benefit trade-off warrants trying to make changes in the JS layer for this.