[FEATURE] Extract multiple crops from one image
mdejeans opened this issue · 2 comments
Hello @am15h ,
First of all, thanks for your quality that's really awesome !
I need to preprocess an image such as I can extract multiple crops from it and stack them into a new axis with TensorImage/TensorBuffer. Thus the output shape for an input image will be ( crops_number, patch_height, patch_width, channels )
However the only function with imageProcessor that can crop an image ( ResizeWithCropOrPadOp(cropSize, cropSize) ) can only do one crop.
To do so, we can imagine a new processing function with the following definition:
ExtractCropsAndStack(cropsPosition, cropHeight, cropWidth) -> TensorImage
where
cropsPositionis aList<List<int>>of coordinates of the top-left position of each crop. (ex:[[0,0], [0,100], [100,0], [100, 100]])cropHeightis a int that specify the crop height in pixel (ex:100)cropWidthis a int that specify the crop width in pixel (ex:100)
So, if I want to extract 4 crops of a 200x200 image I can set cropsPosition=[[0,0], [0,100], [100,0], [100, 100]], cropHeight = 100 and cropWidth = 100 to get a TensorImage of shape (4, 100, 100, 3).
In such way with a variable number of parametrizable crop position, we can do other crops that centered ones, even if we only need one crop.
So here is the discussion about how to do realize such function and . Is creating a new function into image/ops/ the best way to do or it would be easier if ResizeWithCropOrPadOp can take crops parameters as arguments ?
Let me know if you would be interested to add this feature and how I can contribute to. If the discussion comes to a nice solution, I will be happy to create a branch for it and open a pull request.
Thanks a lot, @mdejeans.
In such way with a variable number of parametrizable crop positions, we can do other crops that centered ones, even if we only need one crop.
So here is the discussion about how to do realize such function and. Is creating a new function into image/ops/ the best way to do or it would be easier if ResizeWithCropOrPadOp can take crop parameters as arguments?
Having the flexibility to choose crop positions other than the center is definitely useful. We can add an optional parameter to ResizeWithCropOrPadOp for this purpose.
So, if I want to extract 4 crops of a 200x200 image I can set cropsPosition=[[0,0], [0,100], [100,0], [100, 100]], cropHeight = 100 and cropWidth = 100 to get a TensorImage of shape (4, 100, 100, 3).
It might not be possible, because the image processing pipeline and TensorImage are designed for a single image only, https://github.com/am15h/tflite_flutter_helper/blob/master/lib/src/image/tensor_image.dart#L90.
For multiple crops, a better option might be to manually create multiple TensorImage objects processed using different parameters in ResizeWithCropOrPadOp.
Please let me know what you think about this.