Why are all the augmented images I generated covered in green? I can't see the original images.
Closed this issue · 5 comments
What could be causing this issue?
Hi, I would recommend checking the value distributions of tI, SegPrior, and BoundaryPrior.
Values in tI should be floating numbers and should be in the range of 0 and 1.
SegPrior usually has values between 0 and 3.
Values in BoundaryPrior would be either 0 or 1.
Putting them together in lines 23 and 24 would definitely make some of the values in the resulted tI go above 1. Directly saving the new tI requires performing a normalization step first. If you use the max value to normalize the image, with the max values being too large and the original image with values being too small (dark-looking image), the resulting image would look very green after saving. This is mainly the problem/issue with regard to the normalization step.
A quick workaround would be the following.
Change line 23 and line 24 to the following:
reducer=5 # you can change this value as appropriate.
tI[:,:,1]=tI[:,:,1]+SegPrior/reducer
tI[:,:,2]=tI[:,:,2]+BoundaryPrior/reducer
tI[np.where(tI>1.0)]=1.0
#to save tI:
skimage.io.imsave(savepath, 255*tI)
The idea is to keep the original pixel values unchanged, adding augmentation with a smaller magnitude, and if values go above 1.0, make it 1.0. Note that all these changes are merely due to the image-saving need. If one applies SAMAug on the fly in training and testing (w/o saving them to images), then there is no need to apply the workaround changes.
First of all, thank you for taking the time to answer my question. I use SAMAug to enhance retinal disease images. However, in the resulting enhanced images, the entire eyeball is covered in green. Can these images be directly used for dynamic application in training?
Hi, thanks for your interest. If SAM does not segment anything useful in the image (e.g., finer structures other than the entire eyeball), then the following augmentation would not help. The usefulness of SAMAug highly depends on whether SAM provides useful segmentations. I would recommend checking the outputs of the SAM (those masks) and see if SAM segment/detects anything other than the entire eyeball.
Note that you may try to tune some parameters of SAM to see whether it gives you more objects in the segmentation masks. For example, the iou_threshold can be tuned down for resulting more objects. Details can be found in the SAM repository: https://github.com/facebookresearch/segment-anything/blob/6fdee8f2727f4506cfbbe553e23b895e27956588/segment_anything/automatic_mask_generator.py#L41
Thank you very much for your answer