`rows_per_page` query / pagination logic broken both in web UI and API
decathorpe opened this issue · 7 comments
It looks like the logic behind pagination (both in the Web UI and in the REST API) is broken, and likely has been for a while.
For example, this query explicitly asks for 50 rows per page, but the first page only contains 10 results:
https://bodhi.fedoraproject.org/updates/?user=decathorpe&search=rust-&submitted_since=2024-09-01&submitted_before=2024-10-01&rows_per_page=50
It looks like the pagination splits on number of builds instead of the number of updates, leading to wrong results.
The same can be seen in the API:
import requests
ret = requests.get("https://bodhi.fedoraproject.org/updates/?user=decathorpe&search=rust-&submitted_since=2024-09-01&submitted_before=2024-10-01&rows_per_page=50")
len(ret.json()["updates"])
# 10If the number of builds on a page is larger than the value of rows_per_page, this pagination bug also looks like it will swallow up some builds from appearing on any page of the query.
Yep, I observed those problems in the past, but I never figured out how to properly fix. The hint about pagination being splitted by builds instead of updates is really useful (I didn't noticed before). I'll need help from someone with better SQL skills, though...
Oh, looks like my report is a duplicate of #4130 then?
Yes, it's the same issue.
As I understand, when joining the builds table, the SQL query returns a row for each Update/build combo.
So, having a single update with 10 rust-* builds, when searching for rust- the query returns 10 rows and the pagination applies there. After that, the results are deduplicated by Update.id. That means that a pagination of 5 will result in two pages with the same update displayed.
I did a quick test by apply distinct() to the main query and at first glance it seems to fix the issue. I need to verify that I don't break anything else (the fix seems too easy...).
The other option of switch from using joins to subqueries works too, but it causes much slower queries (~40% slower).
What a simple fix for a gnarly problem 😅 Thank you so much! Looking forward to this landing.
I have deployed an updated Bodhi version to staging for test.
I hope to have 8.2 ready and deployed right after F41 freeze ends.
This is now deployed in prod ;-)
Awesome - thanks!