Invalid Number of Channels in Input Image (error in dirtyrollers.py)

I got the following error when I was running an automated pipeline on individual ink phase augmentations. This error only crept up for dirty rollers:

File "/home/manan/projects/Easter2/src/data_loader.py", line 102, in preprocess
    img = aug(img)
  File "/home/manan/anaconda3/lib/python3.7/site-packages/augraphy/augmentations/dirtyrollers.py", line 78, in __call__
    image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.1.0) ../modules/imgproc/src/color.simd_helpers.hpp:92: error: (-2:Unspecified error) in function 'cv::impl::{anonymous}::CvtHelper<VScn, VDcn, VDepth, sizePolicy>::CvtHelper(cv::InputArray, cv::OutputArray, int) [with VScn = cv::impl::{anonymous}::Set<3, 4>; VDcn = cv::impl::{anonymous}::Set<1>; VDepth = cv::impl::{anonymous}::Set<0, 2, 5>; cv::impl::{anonymous}::SizePolicy sizePolicy = (cv::impl::<unnamed>::SizePolicy)2; cv::InputArray = const cv::_InputArray&; cv::OutputArray = const cv::_OutputArray&]'
> Invalid number of channels in input image:
>     'VScn::contains(scn)'
> where
>     'scn' is 1

Thanks for letting us know. This should be fixed in the latest commit, you may uninstall Augraphy and install it again with:

!pip install git+https://github.com/sparkfish/augraphy

img.shape=(158, 3, 700), new_y=37, new_x=2000
OpenCV(4.1.0) ../modules/imgproc/src/resize.cpp:3361: error: (-215:Assertion failed) !dsize.empty() in function 'resize'

The augmentation and a few others are giving a shape with channels as axis=1, causing this error it seems.

It is expected because your image is not in shape of (y, x, channel) or (y,x), so you need to reshape your image first. If your image has more than 3 channels, it's better for you to augment each slice of image individually.

This may go to the larger issue of better support for color images. We may either need some guidance in the documentation, or provide a more informative exception that we originate within the library, or would it be possible to see that the shape is not what we expect and reshape it ourselves within the augmentation?

This may go to the larger issue of better support for color images. We may either need some guidance in the documentation, or provide a more informative exception that we originate within the library, or would it be possible to see that the shape is not what we expect and reshape it ourselves within the augmentation?

We can do that, but we might not know their shape is in which structure because they might be in (x,y,channel), (y,x,channel), (channel,y,x) or other permuted shape. It will be better for us to just define the expected input and all users should follow that to use the library.

It will be better for us to just define the expected input and all users should follow that to use the library.

I agree, the order of image channels is not guaranteed, so we should just state in the documentation which order we expect and let the user deal with it.

Since we can see that one of the dimensions has a length of 1, 3 or 4, then we can assume that dimension is a channel. Sure, there could be an image of size 4 or less for x or y dimension, but I think we can handle that rarity.

With the channel dimension determined, we could at least leverage that info to provide a more useful exception earlier in the process that indicates the need to reshape the image in (y, x, channel) format. This might be something to do at the pipeline level, and/or in code that is shared by all/most augmentations?

This might causing problem later because we didn't know if the other 2 x&y is in correct position or not. For example, if input image shape is (15,3,16), we can reshape it to either (15,16,3) or (16,15,3), we wouldn't know which is the correct shape.

img.shape=(158, 3, 700), new_y=37, new_x=2000
OpenCV(4.1.0) ../modules/imgproc/src/resize.cpp:3361: error: (-215:Assertion failed) !dsize.empty() in function 'resize'
The augmentation and a few others are giving a shape with channels as axis=1, causing this error it seems.

Here the output given by augraphy is of shape (158, 3, 700). Then reshaping in my further pipeline failed. I believe colour augmentations are giving this trouble. I got the same issue with dirty rollers, color paper, lighting gradient, jpeg. For all other single augmentations, my further pipeline worked. Here I found shape issues.

img.shape=(158, 3, 700), new_y=37, new_x=2000
OpenCV(4.1.0) ../modules/imgproc/src/resize.cpp:3361: error: (-215:Assertion failed) !dsize.empty() in function 'resize'
The augmentation and a few others are giving a shape with channels as axis=1, causing this error it seems.
Here the output given by augraphy is of shape (158, 3, 700). Then reshaping in my further pipeline failed. I believe colour augmentations are giving this trouble. I got the same issue with dirty rollers, color paper, lighting gradient, jpeg. For all other single augmentations, my further pipeline worked. Here I found shape issues.

Could you share us your code? It will be better for us to troubleshoot the issue by reproduce a same problem at our end.

@cs-mshah Are we able to close out this issue? Or is this still an open issue?

Exception handling was improved in this commit:

# Check if image has correct channel
if len(image.shape) > 2 and (image.shape[2] != 3):
    raise Exception(
        "Image should have channel number of 3 (BGR), but actual dimensions were {}.".format(
            image.shape,
        ),
    )

augraphy/augraphy/base/augmentationpipeline.py

Line 105 in 08954c2

# Check if image has correct channel