Image-Py/sknw

No bounds checking on fill/trace buf access

Alex-Richman opened this issue · 5 comments

Alex-Richman commented

I'm having some trouble processing big 3d volumes using the library. The cause seems to be a lack of bounds checking on buf access in fill() and trace(), buf being statically sized. Due to the numba.jit usage it manifests as a segfault due to the out-of-bounds write.

Here's a PR to add that bounds checking, and allows caller specification of the fixed buffer size to build_sknw():
#2

I guess it could also be dynamically resized, but I'm not familiar with numba.jit flavoured python so I'll leave that.

yxdragon commented

I am sorry, so long time not watch this repo, your image is very large? are you sure the image is a skeleton image? I think 131072 is enough to contain a edge. maybe I should malloc 1/10 lenght of the 3d image. I think 1/10 size is enough for any edge.

numba cound not resize an array dynamically, we can only np.zeros(...) a longer one, and copy data in.

Alex-Richman commented

Yeah the images we're looking at are very big, up to 1024x1024x1024 with lots of connectivity in the skeleton.

I don't have the maximum edge lengths we acquired to hand, but I imagine some factor of the total image size would work well as an initialiser for the edge buffer.

Main thing I needed to fix locally though was the breaks preventing overflowing of that edge buffer, since the buffer overrun in numba causes a SEGFAULT not a clean python exception.

Thanks!

  • Alex.
yxdragon commented

can you export your data as png sequence and email to me? I think 1024^3 is not large, 131042 = 1024 * 128. may be there is something wrong other where!

victor000000 commented

131042 * 8(int64) = 1M
131042 *1024 * 8 = 1G

victor000000 commented

I found that in build_sknw(ske, multi=False, iso=True, ring=True)
if ske is of type np.uint8, then if node num > 255 - 10, node_num will be 0, that is a bug
image

so if node num > 255, we shoud make sure ske is of type np.int64

  • 👍1