streaming parsing for very long strings: new feature proposal

[mewalig]

In the spirit of streaming, I'd like to propose an alternate approach for handling string callbacks, which would do so in a streaming fashion so that arbitrarily large strings can be handled with a fixed amount of memory and in a way that enables further streamed processing of strings. If this is already possible please let me know! as far as I can tell it currently is not.

So for example, there could be an additional string callback e.g.:

static int yajl_handle_string_partial(void *ctx, const unsigned char *stringVal, size_t len);

where yajl_handle_string_partial() would only be used if the string exceeds the length of the buffer (without requiring the buffer to grow). The user can then choose to process the string however they want.

To take it a little further, options could provide flexibility for user-defined buffers e.g.

char my_buffer[4096];
yajl_config(handle, yajl_string_buffer, my_buffer);
yajl_config(handle, yajl_string_buffer_size, sizeof(my_buffer));

Any interest in adding this feature? Happy to help if so

[nkhorman]

The changes that I've made to add a generic framework for sourcing the json content might be what your looking for. #160 / https://github.com/nkhorman/yajl