Pagination mode

[xray-robot]

What is your idea? Provide a use case.

We would like to have pagination, as it will allow to manage requests and know in advance approximately how much data will be transferred.

As a draft, I might suggest:

• Request headers:
  Passing the Prefer: count=estimated parameter to the headers will instruct the database to get the number of records and Range: 0-9 will specify which range to select (but the results will be cut off by --pageSize argument, see below)
• Response headers:
  Returns Content-Range: 0-9/3182

That's how it's done in PostgREST: https://api.koios.rest/#overview--pagination-offsetlimit

It might also be worth adding offset and limit arguments to the endpoint, for quick access to pagination.
It is also necessary to have a startup argument --pageSize 1000 (or something), to set the default value of the maximum page size.

Why is it a good idea?

Tracking and managing long-running requests. It will also prevent malicious requests to public APIs (we plan to make a free public Kupo cluster, with Cloudflare WAF on top of it, but long-running queries will put a heavy load on it, even to server fails).

[xray-robot]

Oh, sorry, I found a discussion about it: #75

[KtorZ]

I was about to point you to it. Would you mind closing this ticket and commenting on #75 instead?

The pagination by slot-number is available for a few releases already. I am however trying things out regarding internal query ordering to possibly speed up paginated requests even more. At the moment, pagination happens after the main query on matches.