Is compute_pyramid_patch_weight_loss correctly imlemented?
ternaus opened this issue · 7 comments
https://github.com/BloodAxe/pytorch-toolbelt/blob/develop/pytorch_toolbelt/inference/tiles.py#L33 can be deleted.
https://github.com/BloodAxe/pytorch-toolbelt/blob/develop/pytorch_toolbelt/inference/tiles.py#L28
https://github.com/BloodAxe/pytorch-toolbelt/blob/develop/pytorch_toolbelt/inference/tiles.py#L29
are never updated and stay zero?
P.S. Numpy is very slow. replacing sqrt and square speeds things up a lot.
I was trying to use ImageSlicer and CudaTileMerger for inference on big image, I trained UNET(RN34) with multiple channels as classes, using the ImageSlicer and CudaTileMerger gives abnormal result as seen in the attached snap, could you please suggest if I am missing something?
As seen on the comparison image many features which got extracted on simple mosaic method (stacking arrays without stride/overlap) which goes missing/improper shape using toolbelt.
Try to normalize the data you put to integrate_batch (your model output) - apply softmax there. (Merger calculates weighted average, so it's a problem if for some reason your data is not normalized between tiles).
If that does not help, check if the problem is only on image borders. There are minor issues on tiles lying on image borders (your problem looks different, but...). This is how it merges outputs of single color.
For example, at the bottom, between white and black there is (more or less) constant gray and preferable would be smooth gradient. Unfortunately I don't think there is an easy fix for that... (but depending on preference it can be slightly improved)
Hi! Thanks for pointing out for this issue. Will investigate that.
Meanwhile, you can use simple mean averaging instead pyramid, by sending np.ones(...) to weight= argument while creating CudaTileMerger and ImageSlicer. This should eliminate this artifacts.
Have you tried mean and it helped? (because my intuition suggests it should make things worse - unless there is really a bug I did not stumble upon yet)
I use mean averaging all the time without any issues. What is very important in tiled inference - make tiles overlap. In this way, edge effect can be dampened a lot. ImageSlicer(tile_size=512, tile_step=256, weight="mean") works best for me.
@BloodAxe FYI - I believe following weighting is better than current pyramid (image above) implementation:
x = np.arange(width) + 0.5
y = np.arange(height) + 0.5
De_x = np.minimum(x, width - x)
De_y = np.minimum(y, height - y)
res = De_x[np.newaxis].transpose() * De_y
This code produces:
You can also add more weight to centers of tiles by np.square(res), which produces:
In my opinion both of them look better than original one.
I was trying to use ImageSlicer and CudaTileMerger for inference on big image, I trained UNET(RN34) with multiple channels as classes, using the ImageSlicer and CudaTileMerger gives abnormal result as seen in the attached snap, could you please suggest if I am missing something?
As seen on the comparison image many features which got extracted on simple mosaic method (stacking arrays without stride/overlap) which goes missing/improper shape using toolbelt.
This issue got resolve with weight='mean', however actually the problem is not from the merger function, i use fastai unet predictions and wrapped under Cudatilemerger. And there was bug in my Windows version of code implementation, however in linuxvm is working fine on it.