Paging Results

[spudmcq]

Hi, we're just starting out with Reladomo.

We see from the FAQ that paging isn't integral.

What is the recommended way to handle paging large results?

[mohrezaei]

There are several patterns that work well instead of paging.

If you just need the top, for example, for a UI display, you can use setMaxObjectsToRetrieve on the list object. Set the value to a decent multiple of a display page. Chances are, nobody is going to page through all the way. Don't forget to use some sort of order-by.

If you're processing the whole list, you should break it up. There are two good strategies here:

1. Use an application specific break up method. For example, if you're processing trades, you can break it up by account or product. You can use range queries or in-clauses. Pick a dimension that'll do the breakup reasonably evenly and is well-indexed (leading column in an index). An advance pattern here is to do an aggregate query first to estimate how to do the break-up.
2. Use forEachWithCursor instead. If all you need is the list itself with no deep fetches, this will work fine. If you need to deep fetch, the MultiThreadedBatchProcessor is a very high throughput pattern than can help.