Computing thumbnails lazily on an empty database can cause crashes and sometimes hangs.
Closed this issue · 1 comments
I've made a new database with 111 images and I start scrolling down.
and I get many error messages that look like this:
[!imread] [!?] Caught exception
<class 'cv2.error'>: c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 53747712 bytes in function OutOfMemoryError
</!!! WARNING !!!>
L______
[gtool] cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\c64f7557-a1ee-53bd-4bda-4a10887f6237.jpg seems corrupted.
+------
<!!! EXCEPTION !!!>
Traceback (most recent call last):
File "c:\users\joncrall\code\guitool\guitool\api_thumb_delegate.py", line 287, in run
thread._run()
File "c:\users\joncrall\code\guitool\guitool\api_thumb_delegate.py", line 275, in _run
img = gtool.imread(thread.img_path) # Read Image (.0424s) <- Takes most time!
File "c:\users\joncrall\code\vtool\vtool\image.py", line 94, in imread
raise IOError(msg)
IOError: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\c64f7557-a1ee-53bd-4bda-4a10887f6237.jpg seems corrupted.
[!run] thread failed
<type 'exceptions.IOError'>: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\c64f7557-a1ee-53bd-4bda-4a10887f6237.jpg seems corrupted.
</!!! EXCEPTION !!!>
L______
OpenCV Error: Unspecified error (The numpy array of typenum=2, ndims=3 can not be created) in allocate, file c:/Users/joncrall/code/opencv/modules/python/src2/cv2.cpp, line 228
OpenCV Error: Insufficient memory (Failed to allocate 53747712 bytes) in OutOfMemoryError, file c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp, line 52
+------
<!!! WARNING !!!>
[!imread] [!?] Caught exception
<class 'cv2.error'>: c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 53747712 bytes in function OutOfMemoryError
</!!! WARNING !!!>
L______
[gtool] cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\f8624f22-0f43-a34a-ddae-bba5ee1fceee.jpg seems corrupted.
+------
<!!! EXCEPTION !!!>
Traceback (most recent call last):
File "c:\users\joncrall\code\guitool\guitool\api_thumb_delegate.py", line 287, in run
thread._run()
File "c:\users\joncrall\code\guitool\guitool\api_thumb_delegate.py", line 275, in _run
img = gtool.imread(thread.img_path) # Read Image (.0424s) <- Takes most time!
File "c:\users\joncrall\code\vtool\vtool\image.py", line 94, in imread
raise IOError(msg)
IOError: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\f8624f22-0f43-a34a-ddae-bba5ee1fceee.jpg seems corrupted.
[!run] thread failed
<type 'exceptions.IOError'>: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\f8624f22-0f43-a34a-ddae-bba5ee1fceee.jpg seems corrupted.
</!!! EXCEPTION !!!>
L______
OpenCV Error: Unspecified error (The numpy array of typenum=2, ndims=3 can not be created) in allocate, file c:/Users/joncrall/code/opencv/modules/python/src2/cv2.cpp, line 228
OpenCV Error: Insufficient memory (Failed to allocate 53747712 bytes) in OutOfMemoryError, file c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp, line 52
+------
<!!! WARNING !!!>
[!imread] [!?] Caught exception
<class 'cv2.error'>: c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 53747712 bytes in function OutOfMemoryError
</!!! WARNING !!!>
L______
[gtool] cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\64cc0697-c135-b40c-dee7-b3bc359c22f4.jpg seems corrupted.
+------
<!!! EXCEPTION !!!>
Traceback (most recent call last):
File "c:\users\joncrall\code\guitool\guitool\api_thumb_delegate.py", line 287, in run
thread._run()
File "c:\users\joncrall\code\guitool\guitool\api_thumb_delegate.py", line 275, in _run
img = gtool.imread(thread.img_path) # Read Image (.0424s) <- Takes most time!
File "c:\users\joncrall\code\vtool\vtool\image.py", line 94, in imread
raise IOError(msg)
IOError: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\64cc0697-c135-b40c-dee7-b3bc359c22f4.jpg seems corrupted.
[!run] thread failed
<type 'exceptions.IOError'>: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\64cc0697-c135-b40c-dee7-b3bc359c22f4.jpg seems corrupted.
</!!! EXCEPTION !!!>
L______
What should happen is thumbnails should be silently computed in the background (maybe unless the utool.VERYVERBOSE flag is specified?)
Another related issue is when I click Batch->Compute all thumbnails
I'll get output that looks like this
[guitool.qtapp_loop()] ENTERING
[guitool.qtapp_loop()] starting qt app loop: qwin=<ibeis.gui.newgui.IBEISMainWindow object at 0x0DB5FDA0>
[guitool] qapp.setActiveWindow(qwin)
[guitool.qtapp_loop()] qapp.exec_() # runing main loop
[back] compute_thumbs
[ibsfuncs] preprocess_image_thumbs
[util_parallel.init_pool] initializing pool with 7 processes
[util_parallel._generate_parallel] executing 111 draw_thumb_helper tasks using 7 processes with chunksize=8
draw_thumb_helper: 0/111... rate=-1.00 iters per second. est_min_left=-1.00, total_min=0.00,
draw_thumb_helper: 5/111... rate=0.77 iters per second. est_min_left=2.33, total_min=0.11,
draw_thumb_helper: 9/111... rate=30.53 iters per second. est_min_left=0.06, total_min=0.11,
draw_thumb_helper: 20/111... rate=65.87 iters per second. est_min_left=0.02, total_min=0.11,
draw_thumb_helper: 43/111... rate=6.07 iters per second. est_min_left=0.19, total_min=0.18,
draw_thumb_helper: 85/111... rate=2.48 iters per second. est_min_left=0.18, total_min=0.46,
OpenCV Error: Unspecified error (The numpy array of typenum=2, ndims=3 can not be created) in allocate, file c:/Users/joncrall/code/opencv/modules/python/src2/cv2.cpp, line 228
OpenCV Error: Insufficient memory (Failed to allocate 53747712 bytes) in OutOfMemoryError, file c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp, line 52
+------
<!!! WARNING !!!>
[!imread] [!?] Caught exception
<class 'cv2.error'>: c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 53747712 bytes in function OutOfMemoryError
</!!! WARNING !!!>
L______
OpenCV Error: Unspecified error (The numpy array of typenum=2, ndims=3 can not be created) in allocate, file c:/Users/joncrall/code/opencv/modules/python/src2/cv2.cpp, line 228
OpenCV Error: Insufficient memory (Failed to allocate 53747712 bytes) in OutOfMemoryError, file c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp, line 52
+------
<!!! WARNING !!!>
[!imread] [!?] Caught exception
<class 'cv2.error'>: c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 53747712 bytes in function OutOfMemoryError
</!!! WARNING !!!>
L______
[gtool] cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\b13d8beb-1bff-a462-b2f4-83469ec9a6ad.jpg seems corrupted.
+------
<!!! EXCEPTION !!!>
[utool] Parallel Generation Failed!
<type 'exceptions.IOError'>: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\b13d8beb-1bff-a462-b2f4-83469ec9a6ad.jpg seems corrupted.
</!!! EXCEPTION !!!>
L______
__SERIAL_FALLBACK__ = True
[util_parallel._generate_serial] executing 111 draw_thumb_helper tasks in serial
draw_thumb_helper: 0/111... rate=-1.00 iters per second. est_min_left=-1.00, total_min=0.00,
[gtool] cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\9dfc2caa-a7fd-5cb1-ad2d-cf3d46e90b56.jpg seems corrupted.
draw_thumb_helper: 5/111... rate=2.11 iters per second. est_min_left=0.84, total_min=0.04,
draw_thumb_helper: 9/111... rate=2.12 iters per second. est_min_left=0.81, total_min=0.07,
draw_thumb_helper: 13/111... rate=2.16 iters per second. est_min_left=0.76, total_min=0.10,
draw_thumb_helper: 17/111... rate=2.20 iters per second. est_min_left=0.72, total_min=0.13,
OpenCV Error: Unspecified error (The numpy array of typenum=2, ndims=3 can not be created) in allocate, file c:/Users/joncrall/code/opencv/modules/python/src2/cv2.cpp, line 228
OpenCV Error: Insufficient memory (Failed to allocate 53747712 bytes) in OutOfMemoryError, file c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp, line 52
+------
<!!! WARNING !!!>
[!imread] [!?] Caught exception
<class 'cv2.error'>: c:/Users/joncrall/code/opencv/modules/core/src/alloc.cpp:52: error: (-4) Failed to allocate 53747712 bytes in function OutOfMemoryError
</!!! WARNING !!!>
L______
[gtool] cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\934cd2b9-3857-6808-d5d0-0176c3012a71.jpg seems corrupted.
Traceback (most recent call last):
File "C:\Users\joncrall\code\ibeis\ibeis\gui\guiback.py", line 44, in backreport_wrapper
result = func(back, *args, **kwargs)
File "C:\Users\joncrall\code\ibeis\ibeis\gui\guiback.py", line 65, in bacblock_wrapper
result = func(back, *args, **kwargs)
File "C:\Users\joncrall\code\ibeis\ibeis\gui\guiback.py", line 90, in wrapped_bslot
result = func(*args, **kwargs)
File "C:\Users\joncrall\code\ibeis\ibeis\gui\guiback.py", line 713, in compute_thumbs
ibsfuncs.compute_all_thumbs(back.ibs, eid=eid)
File "C:\Users\joncrall\code\ibeis\ibeis\ibsfuncs.py", line 1771, in compute_all_thumbs
preprocess_image_thumbs(ibs, **kwargs)
File "C:\Users\joncrall\code\ibeis\ibeis\ibsfuncs.py", line 1764, in preprocess_image_thumbs
six.next(gen)
File "c:\users\joncrall\code\utool\utool\util_parallel.py", line 208, in _generate_parallel
verbose=verbose, nTasks=nTasks):
File "c:\users\joncrall\code\utool\utool\util_parallel.py", line 228, in _generate_serial
result = func(args)
File "C:\Users\joncrall\code\ibeis\ibeis\ibsfuncs.py", line 1705, in draw_thumb_helper
img = gtool.imread(gpath) # time consuming
File "c:\users\joncrall\code\vtool\vtool\image.py", line 94, in imread
raise IOError(msg)
IOError: cannot read img_fpath=F:\data\sussanes_hsdbs\Zebra 2012\_ibsdb\images\934cd2b9-3857-6808-d5d0-0176c3012a71.jpg seems corrupted.
There seems to be a memory issue with the thumbnails. The image that is reported to be corrupted is not. Instead what I think is happening is too much memory is being allocated and not freed. We should probably see if we can reduce memory usage in whatever the create thumbnail function is.
print(ut.get_resource_usage_str()) should help with this.
It might also be a good idea to restrict the number of threads that can be spawned to compute thumbnails in the qthumbnail_delegate in guitool.
EDIT:
My system information:
[var] <type 'str'> platform.machine() = 'AMD64'
[var] <type 'str'> platform.node() = 'Ooo'
[var] <type 'str'> platform.processor() = 'Intel64 Family 6 Model 42 Stepping 7, GenuineIntel'
[var] <type 'str'> platform.python_branch() = ''
[var] <type 'tuple'> platform.python_build() = ('default', 'Jun 30 2014 16:03:49')
[var] <type 'str'> platform.python_compiler() = 'MSC v.1500 32 bit (Intel)'
[var] <type 'str'> platform.python_implementation() = 'CPython'
[var] <type 'str'> platform.python_revision() = ''
[var] <type 'str'> platform.python_version() = '2.7.8'
[var] <type 'tuple'> platform.python_version_tuple() = ('2', '7', '8')
[var] <type 'str'> platform.release() = '7'
[var] <type 'str'> platform.system() = 'Windows'
[var] <type 'tuple'> platform.uname() = ('Windows', 'Ooo', '7', '6.1.7601', 'AMD64', 'Intel64 Family 6 Model 42 Stepping 7, GenuineIntel')
[var] <type 'str'> platform.version() = '6.1.7601'
[var] <type 'int'> multiprocessing.cpu_count() = 8
[var] <type 'str'> sys.platform = 'win32'
[var] <type 'str'> os.getcwd() = 'C:\Users\joncrall\code\ibeis'
[var] <type 'bool'> utool.is64bit_python() = False
[var] <type 'int'> sys.maxint = 2147483647
Python Site:
[var] <type 'list'> len(site.getsitepackages()) = ['C:\Python27', 'C:\Python27\lib\site-packages']
[var] <type 'str'> site.getusersitepackages() = 'C:\Users\joncrall\AppData\Roaming\Python\Python27\site-packages'
Memory Status:
[psutil] total = 7.97 GB
[psutil] available = 6.24 GB
[psutil] used = 1.73 GB
[psutil] current = 79.70 MB
no module resources (doesnt exist on win32)
[var] <type 'str'> cv2.version = '3.0.0-dev'