Pagination is not working

Hi,

in a project using FSXA API I was confused why my fetching logic did not work. Both fields 'totalPages' and 'size' were undefined after inspecting result of fetchByFilter. After watching source code and comparing with CaaS Result JSON, it's obvious: The result JSON does not contain ANY information how many items result for query. There are no properties '_total_pages' or '_size'!

javascript-content-api-library/src/modules/FSXARemoteApi.ts

Line 656 in 2e8aa57

totalPages: data['_total_pages'],

Here is an (anonymized) result of CaaS:

{
  "_id": "2a3819fe-f4ca-4f61-a5eb-34a9c1910433.preview.content",
  "_etag": {
    "$oid": "63033cd2f7bd9c3c604baeac"
  },
  "streams": [
    {
      "uri": "crud",
      "stages": [
        {
          "_$match": {
            "_$or": [
              {
                "operationType": "insert"
              },
              {
                "operationType": "update"
              },
              {
                "operationType": "replace"
              },
              {
                "operationType": "delete"
              }
            ]
          }
        },
        {
          "_$project": {
            "fullDocument::_id": 1,
            "fullDocument::fsType": 1,
            "operationResult": 1,
            "documentKey": 1,
            "operationType": 1
          }
        }
      ]
    }
  ],
  "_embedded": {
     "rh:doc": [

     ]
  }
  "_returned": 5,
  "_links": {
    "crud": {
      "href": "/<project-name>/2a3819fe-f4ca-4f61-a5eb-34a9c1910433.preview.content/_streams/crud"
    }
  }
}

Hello, if you can look at the line

javascript-content-api-library/src/modules/FSXARemoteApi.ts

Line 602 in 2e8aa57

if (unmappedItems.length === 0) {

there is an early return if there are no items returned from the CaaS.

The case is I get results, but CaaS result does not contain _total_pages, so line 656 returns undefined

Can you please provide what do you get from CaaS when you get results?

The result CaaS content is in my first comment

Could you provide the code line of the fetch?
I guess you do not provide the additional Argument "count". Data about pagination is only added if count is set to truthy, as it has performance implications for the underlying database.
See:

javascript-content-api-library/CHANGELOG.md

Line 524 in 9d370c5

    
           If you are using "additionalParams" you can extend the pagination data with "totalPages" and "size" by passing `count: true`. Using "count" has performance implication for it involves querying the collection twice: once for counting and once of actually retrieving the data;

Thanks a lot, version 10.16.2 with additionalParams solved the problem.

@fblenkle sorry for closing the issue that early. We missed the open question from @neo-reply-lukas.
In general we don't recommend to use the "count" parameter, since it slows down request times.

We encourage developers to use more "dynamic" pagination approaches when you don't know exactly how many pages exist - or e.g. infinite scrolling.

Your question is a good motivation to write documentation for that topic, so thank you for bringing it up 😄.

OK. Internaly we fixed this issue last week with a "do-while" loop (first time in my life ^^) as long as resulting items is lower pagesize and started discussion to change it. But after your comment I assume we will leave it.

A performant approach would be to only ask about pagination data only on the first fetch and use that info for all subsequent fetches.

..or to request it lazy when the request delivers exactly the same amount of hits as the page-size