API returns incorrectly sorted duplicate entries

[tomrcht]

Hey !

I'm not sure if this is intended behavior but while trying to extract all answers I'm getting duplicate values. However, I get different behaviors with different paginations.

---

Basically, I have ~50 forms with some containing thousands of answers. When trying to get answers for each forms, I sometimes get duplicate values when my pagination is above 100. Furthermore, I don't get the same duplicates if I sort the answers by default or not.

Below is an example of fetching the answers for a form with a few thousands answers:

[
  { /* non duplicate answer */ },
  {
    "landing_id": "0796d29dcdd7f8a55378eec5ad2e87cf",
    "token": "0796d29dcdd7f8a55378eec5ad2e87cf",
    "response_id": "0796d29dcdd7f8a55378eec5ad2e87cf",
    "landed_at": "2019-04-18T08:42:34Z",
    "submitted_at": "2019-04-18T08:43:34Z",
    "metadata": {
      "user_agent": "...",
      "platform": "other",
      "referer": "https://xxx.typeform.com/to/abc",
      "network_id": "abc",
      "browser": "default"
    },
    "answers": [{
      "field": {
        "id": "123",
        "type": "picture_choice",
        "ref": "c4c30a62-447e-4a9a-9571-c03d03f778eb"
      },
      "type": "choice",
      "choice": {
        "label": "2"
      }
    }]
  },
  { /* non duplicate answers with incoherent sorting* */ },
  {
    "landing_id": "0796d29dcdd7f8a55378eec5ad2e87cf",
    "token": "0796d29dcdd7f8a55378eec5ad2e87cf",
    "response_id": "0796d29dcdd7f8a55378eec5ad2e87cf",
    "landed_at": "2019-04-18T08:42:34Z",
    "submitted_at": "2019-04-18T08:43:34Z",
    "metadata": {
      "user_agent": "...",
      "platform": "other",
      "referer": "https://xxx.typeform.com/to/abc",
      "network_id": "abc",
      "browser": "default"
    },
    "answers": [{
      "field": {
        "id": "123",
        "type": "picture_choice",
        "ref": "c4c30a62-447e-4a9a-9571-c03d03f778eb"
      },
      "type": "choice",
      "choice": {
        "label": "2"
      }
    }]
  },
]

*incoherent sorting means here that a few entries after the duplicate one are correctly sorted (in this case with the default sort) but will jump to a seemingly random date after. Also note that the duplicate entries have the same dates but are not one after the other.

To get each page, I do the following:
const { total_items, items } = await client.responses.list({ uid, before, completed: true })

Thanks for the help, wish y'all a nice day

[MichaelSolati]

@tomrcht sorry for the delayed response, has this still been an issue? The api client doesn't modify the response from the API, so if this is an issue it is likely one from the API (which I haven't been able to replicate in my tests)

[tomrcht]

@MichaelSolati this was still an issue last time we tested this, however I can't test this again since I've changed job since :)

I'm closing the issue since I can't give you any more informations :)

[MichaelSolati]

Got it, well thank you for coming back and giving an update. If anyone else has a similar issue we'll go from there.

[NitinBindal-Hevo]

@MichaelSolati I am facing this similar issue. Typeform is sending me duplicate data in all the pages like whatever data is present in 1st page it's sending the exact same data in other pages as well.Even when the page_count is 2