Streamable implementation?

[krlmlr]

Is the current implementation ready for streaming? I think the criteria would be that it can, based on a specification, accept values from e.g. a JSON stream "as they come in", without knowing e.g. the number of objects in a vector.

Is that even in scope?

[mgirlich]

Which benefits do you see? Reduced memory consumption or other things as well?
And do you have use cases for this already?

Current challenges and questions:

• How would tibblify() know when a field starts and when it ends? This would require to change the interface of most functions a little I guess (but this seems doable).
• The size is currently checked early on for memory allocation. This would have to be changed to growable vectors (this would be an argument to have another go at cpp11).
• The size is used in many for loops.

Overall, I guess this is doable. But I'm not sure whether it is worth supporting that early on or whether we want tibblify to mature a bit more.

[krlmlr]

The use case would be parsing JSON with just one pass. But in general we need a second pass to infer the spec anyway, so perhaps worth holding off a bit.