mastodon/mastodon

Account feed doesn't return enough items if statuses were deleted

kensanata opened this issue · 2 comments

kensanata commented

I want to backup all my toots using the Atom feed. The Atom feed is paginated with next links pointing to the various URLs. My tool follows this chain of next links and runs into a page with no next link even though we haven't reached the beginning of my tooting life. Is this limit on purpose or a bug?

Downloading https://octodon.social/@kensanata...
Downloading https://octodon.social/users/kensanata.atom...
Downloading https://octodon.social/users/kensanata.atom?max_id=242067...
Downloading https://octodon.social/users/kensanata.atom?max_id=238093...
Downloading https://octodon.social/users/kensanata.atom?max_id=235430...
Downloading https://octodon.social/users/kensanata.atom?max_id=230812...
Downloading https://octodon.social/users/kensanata.atom?max_id=228327...
Done
  • I searched or browsed the repo’s other issues to ensure this is not a duplicate.
  • This bug happens on a tagged release and not on master (If you're a user, don't worry about this).
unarist commented

We pick 20 stream_entry records which is associated to status, then renders as Atom feed. However, stream_entry won't be deleted even if associated status was deleted.

In a result, feed may not have 20 toots (also may be empty!), then we assume we don't have more items. It's a bug.

  • 👍3
Gargron commented

Nowadays you can use ActivityPub outbox for this which paginates correctly; or the REST API, which also paginates correctly. Closing