[BUG] Mangadesk fails to load images
dimox99 opened this issue · 17 comments
There seems to be a new issue at the moment.
Until yesterday everything was fine, but when I try downloading mangas now I get a lot of "failed" messages. When looking into it, I noticed that it's somehow related to the image size. At least according to my observation. Images below 1 MB in size are downloaded without trouble. Images around 1.5 MB may cause issues but you might succeed on retry. But if there's any image beyond 2.2 MB it fails downloading for sure. Mangadesk will download images up to that troublesome image, then stop. It will fail on redo until the retry limit is reached.
I tried checking by redownloading a chapter with larger images that I already downloaded some days ago. There was no issue then, now it fails and the images won't load. I also tried on a system with a different ISP, same result.
When accessing chapters through the website, it works, the images will load and they can be saved manually to hard disk one by one, so it's not that the chapters are broken.
Currently using the latest Mangadesk version (0.7.7)
I guess you can check things with any manga title, but if you want one that fails for me, then try this:
https://mangadex.org/title/265da069-ca97-434a-ba54-d118ed9b6e7f
For me, it fail to download chapter 87 completely - first image is over 3MB in size. On the website it's working just fine.
thanks for the report! will take a look when i have the time
i have tried downloading this particular manga chapter and it seems to work okay all the time for me.
maybe you could provide a screenshot of the error (and perhaps also the log generated by the program)?
For me the issue persists, I'm not logging in, Mdesk always runs in guest mode.
I looked into the log, seems the error is this:
wsarecv: An existing connection was forcibly closed by the remote host.
This is what angadesk shows - sometimes images might still be downloaded, but most of the time I'm just getting nothing.
paste the entire log here so i can get the fuller picture.
are you also perhaps using any form of firewall/vpn etc. that may affect certain connections?
for now, you may try setting forcePort443 to true and see if that helps.
I tried forceport, doesn't help
I can tell you I tried without vpn. The firewall isn't an issue either. I tested with my normal system which has a firewall and also with an older notebook that is basically a clean windows without any security software. both produce the error.
I'm using data option for quality. I switched to data-saver for a test. After that, I could download several chapters that won't load otherwise. Switching back to data (quality) also brings the disconnects back. Maybe this helps.
As for the logfile - here you go. What I did was basically start, use guestmodem, search a manga then then tried loading a single chapter. After I got error message, I retried... error again, retried again... during this, the first two images got downloaded, but finally retry limited reached.
2022/05/19 04:35:32 logging.go:49: Session started at 2022-05-19 04-35-32
2022/05/19 04:35:33 service.go:27: Initialised starting screen.
2022/05/19 04:35:33 service.go:31: Running app...
2022/05/19 04:35:34 main_page.go:40: Creating new main page...
2022/05/19 04:35:34 main_page.go:209: Using guest main page.
2022/05/19 04:35:34 main_page.go:227: Setting guest table...
2022/05/19 04:35:34 main_page.go:216: Setting guest grid...
2022/05/19 04:35:34 main_page.go:220: Finished setting guest grid.
2022/05/19 04:35:35 main_page.go:330: Finished setting guest table.
2022/05/19 04:35:47 search_page.go:105: Setting new search results...
2022/05/19 04:35:47 main_page.go:227: Setting guest table...
2022/05/19 04:35:47 main_page.go:367: Settings guest table for search: "dungeon kurashi"
2022/05/19 04:35:47 main_page.go:330: Finished setting guest table.
2022/05/19 04:35:49 page_inputs.go:169: Selected row 1 on main page.
2022/05/19 04:35:49 manga_page.go:161: Setting up manga page chapter table...
2022/05/19 04:35:49 manga_page.go:295: Got 0 of 34 chapters
2022/05/19 04:35:51 page_inputs.go:193: Creating and showing confirm download modal...
2022/05/19 04:35:53 modals.go:47: Removing download_chapters_modal modal
2022/05/19 04:35:56 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49309->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:36:03 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:36:05 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49312->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:36:10 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:36:11 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49314->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:36:30 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:36:32 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49316->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:36:32 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:36:34 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49318->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:36:41 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:36:47 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49322->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:36:57 page_inputs.go:193: Creating and showing confirm download modal...
2022/05/19 04:36:57 modals.go:47: Removing download_chapters_modal modal
2022/05/19 04:37:01 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49325->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:37:08 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:37:10 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49327->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:37:11 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:37:13 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49329->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:37:13 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:37:15 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49331->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:37:15 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:37:17 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49333->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
2022/05/19 04:37:18 modals.go:47: Removing download_error_modal - Dungeon Kurashi no Moto Yuusha - map[1:{}] modal
2022/05/19 04:37:20 manga_page_utils.go:45: Error saving Dungeon Kurashi no Moto Yuusha - Chapter: 34, The girl with the petite figure makes a sudden move on him. - read tcp 10.20.137.146:49335->45.129.229.1:443: wsarecv: An existing connection was forcibly closed by the remote host.
very strange🤔
maybe you can try a vpn on? there is a slight chance they might have blacklisted your original IP for some reason.
(iirc, this used to happen when u spammed their server too many times)
I don't really think they blacklisted my IP since I don't have a static IP address. Then, as I also mentioned, using datasaver mode seems to work, at least for now. I guess if it's a blacklisting, then it wouldn't really matter whether I'm using data or data-saver option. Not to mention I can access their website and read stuff online. The issue seems to be related to the images size, especially if they are larger than 1 MB.
yes in the meantime just stick with data-saver for now.
unfortunately there's not much i can check moving forward as i cannot encounter the problem reliably on my side.
will let this issue stay open for a few weeks. if it resolves by itself, feel free to close the issue :)
Yes, I will do that. It could very well be a temporary ISP issue.
Ah, while I'm at it, short unrelated question: given how the API works, would it be possible to "open/browse" a chapter with Mangadesk before downloading its content? So that you can see a list of images and then for example mark some of them to download only those?
yes it will provide a list of pages (ie. the file names).
as for the marking etc... it is possible. just gotta see if i feel like implementing it or not lol
Well, the problem persists and appears might be caused by my ISP 😞
Apart from that, what I can see is that when an error is happening and I tell Mangadesk to retry, the programm won't start loading from the failed image. Instead it will download all images from the top again. Is this intended or an issue of how the API works?
thanks for the update! i hope the workaround of using data-saver is still working for you.
regarding the retry, yes this is intended, as the download mechanism works by chapter (but not by page).
i am not sure if retrying by page is possible, but i will explore this option when i have the time. i have the impression that it is.
yes, data-saver still works for now - as the image quality is reduced, so is the the file size. Though I must say, when it comes to longstrip mangas some uploaders create images with resolutions like e.g. 1000x30000. Not to mention jerks who put a full chapter into a single image. yuck... even data-saver will produce huge files that fail to load. At the very least I can still retrieve those through the website somehow.
Regarding information... does the API provide a way to retrieve how many images are stored inside a chapter? Or would it be possible to check the number of images and report how many are still missing? Or have an option to just skip an image that does not want to load?
I mean, when I look at the website interface, for example... I also think it should be possible. When you read chapters in single page view on the website, you can browse back and forth through the images... sometimes an image right in the middle of a chapter fails to load even on the website... then you see a "retry/reload image" Button instead. Pictures after a failed one are still loaded on the website, though. So I guess the API might be able to handle such requests.
yea it might be possible, but since this program is more of a downloader than a reader i never thought about downloading just single pages.
if you want, you can create a feature request outlining what you would like to see added to the programme. i will take a look when i have the time :)
sorry for the late reply btw, been busy.
since it seems no one else is facing the same problem, and there exists a workaround (as unfortunate as it may be), i will proceed to close the issue.
I figures something out... it's highly likely that this behavior is related to my ISP and applies to guest logins. This crap only happens when I use Mangadesk in guest mode with my regular provider, I get disconnected if the image size is a bit larger. 2MB seems fine, 3MB sometimes also... anything beyond that might or might not work at all. Sometimes it does, and then again it's basically not working at all and I can't get a single image downloaded because of d/c.
same machine, different provider, guest mode, data quality: no issues
different machine, same provider, guest mode, data quality: disconnects, depending on current bandwith/line speed
On the other hand, data-saver mode works every single time. But the image quality is significantly lower (not only always jpg but images also get notably downscaled) which sucks. I don't like to use that.
Now here's the thing. I never really logged in with Mangadesk, because it was not necessary to just fetch one or two titles. But after trying, I experienced no further disconnects, even with huge images 10MB+...
My best guess is that they added some protection on the server side to prevent anonymous leeching.
So, case closed, indeed.
thats some detailed investigation you did!
if someone else faces the same issue in the future it will be useful to know that this is a possible cause.
thanks for sharing the results of what u found out :D