Asynchronous operations?
Closed this issue · 4 comments
I learned that the upcoming version 1.0 will feature support for AsyncIterable. However, it seems that all operations (map, filter, etc.) will still expect synchronous callbacks. It would be great if these callbacks could be asynchronous. That would allow scenarios like these:
async function loadOrder(orderId) { ... }
const ordersWithManyComments = await Query.fromAsync(orderIds)
.map(loadOrder) // this is asynchronous
.filter(async order => (await order.loadComments()).length > 10) // this, too
.toArray();I've been considering that as well and will likely add support for it. The biggest issue is that it will make async queries significantly slower as you have to wait a turn for every callback as well as every item.
Good point. Due to the streaming architecture, elements would be processed sequentially, not in parallel. In my example, no two loadOrder calls would ever run concurrently. As a result, a manual await Promise.all(...) -- though less elegant -- would usually be faster.
Is there any way around that? Some strategy that would allow multiple elements to be processed concurrently?
I wonder: Parallel LINQ must have the same problem. It probably comes down to clever dependency analysis. But I'm afraid I don't know enough of the inner workings of either library.
The update does not include support for concurrent mapping, however. I might consider that for the future, but JavaScript doesn't really support multiple threads of execution (I/O, web workers, and forked processes notwithstanding).