Funky Pagination Behavior Fetching User Favorites

[peterhanneman]

I ran into some rather bizarre pagination behavior when using the user.favorites.tracks() function.

When passing a page_size parameter I never receive exactly that many tracks back regardless of which offset or page_size I supply.

I'm spitballing that the reason is due to the Tidal API's internal cursor skipping over tracks that may have been deleted or geo-locked since the user added them to their favorites. So even when a page size of say 500 is requested I typically see track counts like 418, 433, 494, etc actually returning.

This is the function I came up with to get around this funky behavior which I hope is helpful for others.

Perhaps this would make sense to implement as a function in the library itself?

"""
Fetch all favorite tracks for a Tidal user, regardless of how many there are.
Deduplicates tracks by ID after fetching all pages.
"""
def fetch_all_user_favorite_tracks(user, page_size=1000):
    all_tracks = []
    offset = 0
    while True:
        new_tracks = user.favorites.tracks(
            limit=page_size,
            offset=offset,
            order=ItemOrder.Date,
            order_direction=OrderDirection.Ascending
        )
        if not new_tracks:
            break
        new_track_count = len(new_tracks)
        print(f"Offset: {offset}, Fetched: {new_track_count}")
        all_tracks.extend(new_tracks)
        offset += new_track_count
        sleep(0.1)

    print(f"Total tracks fetched (with possible duplicates): {len(all_tracks)}")

    seen_track_ids = set()
    deduped_tracks = []
    for track in all_tracks:
        track_id = getattr(track, "id", None)
        if track_id is not None and track_id not in seen_track_ids:
            seen_track_ids.add(track_id)
            deduped_tracks.append(track)

    print(f"Total deduplicated tracks: {len(deduped_tracks)}")

    return deduped_tracks

[tehkillerbee]

I am planning to add the following code from mopidy-tidal that essentially achieves the same thing, and fetches the objects in parallel. It works with all types (tracks, albums, artists)

I think it also handles duplicates properly, but it might need to be tested further.

https://github.com/tehkillerbee/mopidy-tidal/blob/main/mopidy_tidal/workers.py

You can see an example usage here:
https://github.com/tehkillerbee/mopidy-tidal/blob/7ca32f0ee3cf7d4eea03930027d491eade528386/mopidy_tidal/library.py#L263