How do I enable paging?

Question

How do I enable paging?

retorquere opened this issue 7 years ago · 4 comments

I think I'm hitting the default chunk size on my query -- I get exactly 10.000 pointe when I know there should be more. How can I page through the full result set? Preferably automatically?

Answer 1 · 2017-11-14T14:11:29.000Z

You need to set the chunk_size option in the client, which will enable JSON streaming (and result sets larger than 10k entries):

client = InfluxDB::Client.new "mydb", chunk_size: 10_000

Please note that the Ruby client will consume the whole data set, before client.query returns (or yields).

An alternative strategy is to page manually through the results. This requires you to add an ORDER BY time to your query and remembering the last timestamp. The next queries then need to include a WHERE time < $lastTimestamp (or time > $lastTimestamp, depending on which the order direction). This gets complicated though, if you need additional grouping clauses...

Answer 2 · 2017-11-14T14:19:45.000Z

I did connect with chunk_size:

InfluxDB::Client.new(OPTIONS.database, username: OPTIONS.database_username, password: OPTIONS.database_password, time_precision: 'ms', chunked: true, chunk_size: 1000)

but even then, slurping everything into memory would be too much. I'll go with the timestamp-chunks as you suggested.

Answer 3 · 2017-11-14T14:53:18.000Z

Aren't influx query results always ORDERed by time?

Answer 4 · 2017-11-14T14:59:57.000Z

The order depends on your query. You can of course order by any other field and not just by time :-)

Your time literal is invalid, it must be RFC3339 compatible (you need at least full date Y-m-d) :

time >= '2015-10-01' and time < '2015-11-01'