High-resolution 3D Stitching Parallelization
Closed this issue · 5 comments
I am trying to stitch two large 3D images with >90% overlap using the SimpleMontage.py example script and am having memory/timeout issues at the montage.Update() stage. This script works perfectly with lower resolution versions of the same data.
I have already tried increasing the resources available (400GBs and 3hrs timeout duration) but continue having problems.
I have also tried montage.SetNumberOfWorkUnits(n_threads=64) and still get stuck at montage.Update().
Is there any way to parallelize any part of the stitching process to avoid these issues?
Parallelization is done on tiles. With just two tiles, there is not a lot of opportunity for parallelization, nor for reducing parallelism. Registration is done on the overlapping part of images, and you have pretty much the worst case here from memory consumption point of view: 2 tiles with 90% overlap.
What you could do is do the registration on the lower resolution version of data, then do the stitching on full resolution data. If you run into memory issues there, you might want to enable chunked processing. Beware: not all image formats support streaming.
Thank you for your response @dzenanz! I'll give this a try for my two large tile stitching problem :)
I have also tried to stitch 20 high-resolution tiles with 10%-ish overlap and am also seeing memory/timeout issues. Is there a native way to parallelize this using itkMontage?
It is parallel by default. You might want to limit parallelism to reduce memory use. For example, define environment variable called NSLOTS and set its value to 2.
Thank you!!
Closing the issue because the original and followup questions have been answered. Thank you :)