Is it possible to stream results

[jcoglan]

Apologies for using the issue tracker for this but I wasn't sure how else to get hold of you. We're looking for a way to stream results back from Postgres, is this possible?

I've tried something like

conn.query 'select * from people' do |result|
  # handle result
end
conn.set_single_row_mode

but when I do this, the result set is empty. Presumably I need to listen for each row being emitted, but I'm not sure how.

[royaltm]

AFAIK the set_single_row_mode requires special interpretation of get_result() results.
The docs for set_single_row_mode states that you need to use low-level api function - get_result many times, until the last result which has 0 rows is returned.
Unfortunately the em-pg-client for returning query result emulates get_last_result in an async manner and is always returning only the last one. So according to the docs you will always get the 0 row result from your query using set_single_row_mode and the above construction.
I think the set_single_row_mode should be patched to set em-pg-client internal flag to process returned results differently, perhaps calling argument block several times with each result.
Thanks for the issue anyway. I think the streaming for processing large result is interesting enough for me to implementing it in some near future.

[jcoglan]

Thanks for considering this. It seems like a use case perfectly suited to event-based APIs, but as it is I will probably use the get_result API in a background thread.

[royaltm]

I think of patching get_result itself to maintain pg api compatibility. The patched get_result will return a deferrable or in synchrony version will yield current fiber, then return the result.
In the synchrony scenario, you would use it exactly the way it is shown in the example from docs and benefit from all async-magic under the hood.

[royaltm]

I've added some basic support for single row mode. Its api is kind of crude though, however you should be able to stream data asynchronously. See examples/single_row_mode.rb.
I'm still working on a more convenient api for this with (lazy)enumerator support and so.

Please use gem >= 0.3.2. Version 0.3.1 has horribly slow (~10 times slower) async get_result implementation.