Can't open images > 65535 on one edge, or total 500 Megapixels
Lt-knb opened this issue · 18 comments
And yes... It happens in the 64 bit version.
I either get a "Not enough memory available" or, with particularly huge resolutions, "Could not be read" without providing the reason.
If I enable HighQualityResampling, opening those images straight up leads to a crash.
I have 16 GB of RAM, so unless I'm missing something, that's not supposed to happen.
Also, the 32 bit version seems to be even more limited, to about 200 MB RAM images
Wow... hmm I see.
How did you arrive at the 1.3GB of RAM. Is that what the task manager says about the memory JPEGView is using?
Also, can you try turning off AutoDetect CPU in the INI? Set it to SSE, see if it still crashes. There are some other issues open for very big images that causes crashes, some have to do with CPU, others are still open
Are you also saying that if you turn OFF HighQualityResampling it doesn't crash?
Yes, I checked RAM usage, but IrfanView is also able to calculate how much RAM an image needs to be displayed.
Something interesting: FastStone is limited to 1.3 GB RAM images, but it's 32 bit. Also, the notes of the 64 bit version of IrfanView explicitly state that it's able to display images that need more than 1.3 GB of RAM, unlike the 32 bit version.
I tried changing CPU settings but it was still crashing.
Btw yes, after disabling resampling it stopped crashing, but I just checked and now it won't crash anyway if I enable it again.
It will just show the warnings and refuse to open them.
I checked again, and it used 3.5 GB to display a 1.7 GB RAM image. This is getting confusing.
I just opened a 1 GB RAM image and it's using 2.3 GB.
You can try to reproduce the issue by opening this image
I'm sorry, misclick...
I checked that image out... FSViewer shows it to be 70200x2885 (202 MP). JPEGView has an internal limitation of 65535 on any one dimension, regardless of 32-bit or 64-bit, so it for sure won't open that image.
I found the bug that causes it to crash in that instance though. It should just say cannot be read.
And it appears, the 65535 limitation is directly related to HighQualityResampling... I tried turning HighQualityResampling off, and bumping the image limitation, and JPEGView opens the image, but the image is corrupted.
I also can't open this image, which is 57000x10320.
Another image I can't open is just 31200x31440.
Can you reproduce?
I mean, they're smaller than 65535.
It should be able to open them
I also can't open this image, which is 57000x10320.
The 5700x10320 is 588 MP. There is another coded limitation of 500 MP... That one I'm not so sure why, but I'm assuming the original author put some limitation there so images wouldn't load forever. It's also possible when the original author added this limitation, it was over 10 years ago, and we all have 64GB RAM now lol.
It was set to 300 MP in 2014, and looks like it was changed to 500 MP in 2016.
I tried removing the limitation in code, and I could open that image properly. Addressing that is particularly hard... right now it's a hardcoded #define but I could make it a setting. But the limits are set differently for the 32-bit and 64-bit versions (having that as a setting but ignored for 32-bit is an option...)... I'm also not sure if these are artificial or practical limits... like exactly how much MP is supported per system is likely tied to RAM and available memory at the time
For example, I actually had to open this image on another system... the sandbox I use to develop on didn't have enough memory to open the image, even in 64-bit mode... I had to go to a system with 64GB memory to open that bigger image
Another image I can't open is just 31200x31440
That's about 981 MP, which is also hitting the limit above.
Maybe there's a RAM leak as well?
If I open that image on IrfanView, it only uses 1.7 GB...
Opening 57000x10320 image on FSViewer takes about 2.4GB - as viewed by Task Manager
Opening 57000x10320 image on JPEGview takes about 2.3GB - as viewed by Task Manager
Anyways, memory leaks wouldn't be the problem here... you're hitting the hard limitations.
But it's an interesting test case... I already fixed the bug that causes the crash, but how to allow bigger dimensions, etc, that's a different story
Oh, that's perfect actually. Why did you need to open it in the 64GB RAM system then?
I hope it's possible to remove both limitations
Oh, that's perfect actually. Why did you need to open it in the 64GB RAM system then?
Hehe, because I only have a few environments... super resource limited or super resource unlimited haha
I hope it's possible to remove both limitations
The 65535 dimensions does not appear to be doable with HQR on... with it off, I'm getting strange corruption at that size... I'd need more samples to play with... did you generate those yourself?
The actual 500 MP appears to be doable, and I need to think about what's the best way to implement it... I also wonder what's the limit of 32-bit for MP before it runs out of memory...
The relevant limits are here: https://github.com/sylikc/jpegview/blob/master/src/JPEGView/MaxImageDef.h
I assume the original author added these to estimate roughly how much memory would be used when an image is decompressed, but I think there should be a single user-configurable setting that sets the max size.
I assume the original author added these to estimate roughly how much memory would be used when an image is decompressed, but I think there should be a single user-configurable setting that sets the max size.
I thinking of adding an option in the INI to control this, but only for the 64-bit version. I don't know what the hard limit on the 32-bit version is, but I'm sure it's pretty low. Then again, could probably use the same limit and just have it throw an out of memory error. I suspect the limit is to keep the file loading relatively speedy... those huge files takes a long time, and if it also fails to load, it's really expensive.
The other one is the 65535 one size image dimensions... @qbnu it's clear that the whole process will crash when HighQualityResample is on and the dimensions is huge (like that first sample image)... I can't figure out why, but probably something with the routines...
however, when HighQualityResample is off... the image loads! (I set the limit to some ridiculously high number so it wouldn't get caught)... but, the image comes out really distorted... that part I can't figure out why
The 500 megapixel limit is to prevent integer overflow. 1024 * 1024 * 500 * 4 channels = 2097152000, slightly under the max signed 32-bit integer. To set it higher we would need change a lot of ints to unsigned ints or longs.
The 500 megapixel limit is to prevent integer overflow. 1024 * 1024 * 500 * 4 channels = 2097152000, slightly under the max signed 32-bit integer. To set it higher we would need change a lot of ints to unsigned ints or longs.
Hmm... I wonder why I was able to set that to like 5000 and load one of the bigger images (~900 MP) without crashing...
@sylikc I was testing PSB support using this image (69536x22230) with the limit checks commented out. I found the same issue you saw where the image is stretched and zoomed in, so I tried only decoding the 60000x22230 area on the left. This works at first, but if you zoom in or out then everything past the first 2^30 pixels (2^32 bytes) is replaced with a different part of the image (edit: confirmed fixed by #307).
I also noticed that we use ::GetFileSize(hFile, NULL) in a lot of places, which doesn't work for files over 4GiB since it just returns the low 32 bits. Helpers::GetFileSize works properly but is only used once.
I also noticed that we use
::GetFileSize(hFile, NULL)in a lot of places, which doesn't work for files over 4GiB since it just returns the low 32 bits.Helpers::GetFileSizeworks properly but is only used once.
Does this mean we should blanket replace any GetFileSize() calls with the Helpers call?