darylhjd/mangadesk

[BUG] Not all follows retrieved

scott-tancock opened this issue · 6 comments

scott-tancock commented

Issue

When logged in, only the first 50 followed manga are fetched from Mangadex

Expected behaviour: After logging in, all followed manga are retrieved
Actual behaviour: After logging in, only 50 followed manga are retrieved
Steps to reproduce: Log in to an account with > 50 followed manga
Additional info: Looks like you're only sending one request to the server with limit=50. Instead, you should read the 'total' token returned at the end of the list of manga and loop the fetch with offset = offset + limit each time.

Example

<log in>
<start of req sequence>
Req 0: /user/follows/manga?limit=50
Resp 0: <Manga[0..49]>, total: 812, limit: 50, offset: 0
Req 1: /user/follows/manga?limit=50&offset=50
Resp: <Manga[50..99]>, total: 812, limit: 50, offset: 50
Req 2: /user/follows/manga?limit=50&offset=100
Resp 2: <Manga[100..149]>, total:812, limit: 50, offset: 100
...
Req 15: /user/follows/manga?limit=50&offset=750
Resp 15: <Manga[750..799]>, total:812, limit: 50, offset:750
Req 16: /user/follows/manga?limit=12&offset=800
Resp 16:  <Manga[800..811]>, total: 812, limit: 12, offset: 800
<end of sequence>

Evidence

Fetched manga (50 counted):
image
Response from the sever using Postman (812 follows total):
image

darylhjd commented

ngl. I never expected anyone to have this many follows.

  • 😄1
darylhjd commented

api seems to have problems with offset values (there seems to be an edge case where you cant get just the last followed manga).

i think this will take awhile to fix

darylhjd commented

@scott-tancock i will be releasing a test build addressing this issue soon. will also have an added feature for pagination (loading 800+ follows on a single page would probably burn down someone's PC and their home).

scott-tancock commented

Seems to be working to me. Fetches all 812 Manga, and each item can be accessed as normal. Few odd quirks with the
interaction:

  • When loading a page of follows, the current selection jumps around a bit. It gets set back to the previous location once all 100 follows are loaded, but it looks a tad weird. Probably just a quirk of tview and not possible to fix.
  • If moving between follows pages quickly (press tab again before current page has finished loading), the cursor ends up moving down the page somewhat. This is probably a race condition between the page load and setting the cursor back to the top after the previous page load. This really isn't a major issue, just a quirk.
  • The combination of tab / ctrl+b is rather strange. It would be great to change it to a common combination such as ctrl+f / ctrl+b or tab / shift-tab.
  • Maybe set the final page of follows to show the total number of follows in the top status bar. e.g. current: "Page 9 (801-900)", suggested: "Page 9 (801-812)".

^ Those are all just minor matters. If no-one else is having an issue with this patch, the issue can be closed. Pretty impressive how fast you fixed this.

darylhjd commented

1st point:

  • its tview.

2nd point:

  • yea it just starts updating the table through all the pages lol, but the end result should still be the last page you tabbed/ctrlb to. i am thinking of a solution to pass context through goroutines to stop previous updates but i havent figured it out yet.

3rd point:

  • key space is getting a little too crowded now. i wanted backtab (shift + tab) but it doesnt work on my machine and so safe to say i wont be using it.

4th point:

  • ok good point. only excuse at this point is because i was lazy to check total entries :>