Problem with val.py. missing argument in val.py

[snowbrood]

Hi there!

I have been trying to run val.py which imports yolo_four_input

in the function _forward_augment

yi = self._forward_once(xi)[0] # forward
just one argument is passed to forward_once where it expects at least two. Why is x_ir missing? I think you don't want to augment IR images which is understood.

but how to resolve this.
Thank you!

[sebastianvitterso]

This sounds like something @ingin97 knows more about, but I'll have a look too. 👀

[sebastianvitterso]

yolo_four_input.py on branch four_input is the file in question, right @snowbrood?

As far as I can see, we never augmented our IR images in the four_input branch. From what I can see now, I'm not sure if we ever augmented at all on this branch, but I seem to recall that we did... Can you confirm this @ingin97?

To fix this, I think you will need to augment both the RGB and IR images (exactly in the same manner).

Something like this might work. I have not tested this. Feel free to add a PR for this if you make it work! 😄

    def _forward_augment(self, x_rgb, x_ir):
        img_size = x_rgb.shape[-2:]  # height, width
        s = [1, 0.83, 0.67]  # scales
        f = [None, 3, None]  # flips (2-ud, 3-lr)
        y = []  # outputs
        for si, fi in zip(s, f):
            xi_rgb = scale_img(x_rgb.flip(fi) if fi else x_rgb, si, gs=int(self.stride.max()))
            xi_ir = scale_img(x_ir.flip(fi) if fi else x_ir, si, gs=int(self.stride.max()))

            yi = self._forward_once(xi_rgb, xi_ir)[0]  # forward
            # cv2.imwrite(f'img_{si}.jpg', 255 * xi_rgb[0].cpu().numpy().transpose((1, 2, 0))[:, :, ::-1])  # save
            yi = self._descale_pred(yi, fi, si, img_size)
            y.append(yi)
        return torch.cat(y, 1), None  # augmented inference, train

[snowbrood]

yes, the file in question is yolo_four_input.py. I have just renamed it.
I also don't want to augment my depth images; how to prevent it from augmenting it .

Rgb Images should be augmented but I want to keep depth images as it is.

update: I have tried passing x_ir to the forward_augment function
and I get this

RuntimeError: The size of tensor a (108) must match the size of tensor b (84) at non-singleton dimension 3

[ingin97]

@snowbrood I think this is the option to do augmentation on inference/validation, which we did not have the time to do while we were working on this. Though if you think Test-Time Augmentation is worth the implementation effort, we are welcoming a PR.

Augmentation is added to the training script train.py though! So do not worry about that. Though the depth images are also augmented, because the pixels in the depth images have to be in the same position as those in the RGB image, if the image is flipped, rotated or tiled.

Conclusion: If you did not want to add TTA I think you just need to turn of augmentation while running the validation for it to work.

[sebastianvitterso]

I also don't want to augment my depth images; how to prevent it from augmenting it .

Why do you not want this? The augmentation is (among other thing) a process of cutting the images into pieces, so it's imperative that the depth images and the rgb images are cut into exactly the same pieces, otherwise it doesn't make much sense. So it should be an all-or-nothing choice. Either with or without augmentation.

[snowbrood]

I have disabled the augmentation and TTA , But I still get the error when I run val.py

I don't understand what's wrong .

[snowbrood]

I also don't want to augment my depth images; how to prevent it from augmenting it .

Why do you not want this? The augmentation is (among other thing) a process of cutting the images into pieces, so it's imperative that the depth images and the rgb images are cut into exactly the same pieces, otherwise it doesn't make much sense. So it should be an all-or-nothing choice. Either with or without augmentation.

you are right. I confused it with sth else

[sebastianvitterso]

I have disabled the augmentation and TTA , But I still get the error when I run val.py

I don't understand what's wrong .

I'm not exactly sure what's gone wrong here, but I'm assuming the error is raised when running the FourInputModel.forward(x_rgb, x_ir)-function? That should be the only model/module that accepts both RGB and IR/depth. Have you modified your code in any other way?

[ingin97]

I have disabled the augmentation and TTA , But I still get the error when I run val.py

I don't understand what's wrong .

Interesting this is only run when validating and during training. Though weird we did not get that error, you might try something like this:

if not training:
    if device.type != 'cpu':
            model(torch.zeros(1, 3, imgsz, imgsz).to(device).type_as(next(model.parameters())), torch.zeros(1, 1, imgsz, imgsz).to(device).type_as(next(model.parameters())))

Or maybe remove it and see if it still initializes properly?

[snowbrood]

I have disabled the augmentation and TTA , But I still get the error when I run val.py
I don't understand what's wrong .

Interesting this is only run when validating and during training. Though weird we did not get that error, you might try something like this:

if not training:
    if device.type != 'cpu':
            model(torch.zeros(1, 3, imgsz, imgsz).to(device).type_as(next(model.parameters())), torch.zeros(1, 1, imgsz, imgsz).to(device).type_as(next(model.parameters())))

Or maybe remove it and see if it still initializes properly?

I tried exactly what you wrote , before you wrote it. but it didn't work .

I commented it out and it works which is strange

I was able to train my model perfectly , I don't know why This was causing the problem .

anyhow all the same Thank you guys for such prompt response and your work has helped me alot.