Inference to single image

Question

Inference to single image

windspirit95 opened this issue 4 years ago · 6 comments

Hi,
I am trying to modify your code to inference to single image (with given path and detected boxes coordinates in xyxy form).
Here is what I have added to your inference.py:

However, when I run the test_net.py, which will call the single_inference function, I faced this error:
RuntimeError: size mismatch, m1: [43008 x 7], m2: [2048 x 1024] at C:/w/1/s/tmp_conda_3.7_100118/conda/conda-bld/pytorch_1579082551706/work/aten/src\THC/generic/THCTensorMathBlas.cu:290

Full traceback:
File "test_net.py", line 188, in
main()
File "test_net.py", line 145, in main
single_inference(model, img, boxes, "cuda")
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\engine\inference.py", line 184, in single_inference
prediction = single_image_compute(model, img, boxes, device, inference_timer)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\engine\inference.py", line 84, in single_image_compute
output = model(pil_img.to(device), targets)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\detector\generalized_rcnn.py", line 61, in forward
x, result, detector_losses = self.roi_heads(features, proposals, targets)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\roi_heads\roi_heads.py", line 26, in forward
x, detections, loss_box = self.box(features, proposals, targets)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\roi_heads\box_head\box_head.py", line 52, in forward
class_logits_causal_list = self.causal_predictor(x, proposals)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\roi_heads\box_head\roi_box_predictors.py", line 96, in forward
xzs = [self.z_dic(feature_pre_obj, dic_z, prior) for feature_pre_obj in feature_split]
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\roi_heads\box_head\roi_box_predictors.py", line 96, in
xzs = [self.z_dic(feature_pre_obj, dic_z, prior) for feature_pre_obj in feature_split]
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\roi_heads\box_head\roi_box_predictors.py", line 111, in z_dic
attention = torch.mm(self.Wy(y), self.Wz(dic_z).t()) / (self.embedding_size ** 0.5)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\linear.py", line 87, in forward
return F.linear(input, self.weight, self.bias)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\functional.py", line 1372, in linear
output = input.matmul(weight.t())
RuntimeError: size mismatch, m1: [43008 x 7], m2: [2048 x 1024] at C:/w/1/s/tmp_conda_3.7_100118/conda/conda-bld/pytorch_1579082551706/work/aten/src\THC/generic/THCTensorMathBlas.cu:290

Please help me to understand your model better ^^ Thank you ;)

Answer 1 · 2020-08-10T03:33:01.000Z

Plus that transforms.ToTensor() is from torchvision. The cfg I am using is VC-R-CNN/configs/caffe2/e2e_faster_rcnn_R_50_C4_1x_caffe2.yaml

Answer 2 · 2020-08-10T03:56:12.000Z

I forgot to load from checkpoints.
But since then, new issue is as follow:
Python version: 3.7
Is CUDA available: Yes
CUDA runtime version: 10.1.105
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] numpy==1.19.1
[pip3] torch==1.4.0
[pip3] torchvision==0.5.0
[conda] blas 1.0 mkl
[conda] mkl 2020.1 216
[conda] mkl-service 2.3.0 py37hb782905_0
[conda] mkl_fft 1.1.0 py37h45dec08_0
[conda] mkl_random 1.1.1 py37h47e9c7a_0
[conda] pytorch 1.4.0 py3.7_cuda101_cudnn7_0 pytorch
[conda] torchvision 0.5.0 py37_cu101 pytorch
Pillow (7.2.0)
2020-08-10 12:53:16,963 vc_rcnn.utils.checkpoint INFO: Loading checkpoint from ./model_final.pth
File "test_net.py", line 194, in
main()
File "test_net.py", line 95, in main
_ = checkpointer.load(ckpt, use_latest=args.ckpt is None)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\utils\checkpoint.py", line 62, in load
self._load_model(checkpoint)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\utils\checkpoint.py", line 98, in _load_model< load_state_dict(self.model, checkpoint.pop("model"))
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\utils\model_serialization.py", line 80, model.load_state_dict(model_state_dict)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 830, in self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for GeneralizedRCNN:
size mismatch for roi_heads.box.predictor.cls_score.weight: copying a param with shape torch.Size([80, size mismatch for roi_heads.box.causal_predictor.causal_score.weight: copying a param with size mismatch for roi_heads.box.causal_predictor.Wy.weight: copying a param with shape torch.Size([1024, size mismatch for roi_heads.box.causal_predictor.Wz.weight: copying a param with shape torch.Size([1024, > loaded from backbone.body.layer1.0.bn1.bias of shape (64,)
loaded from backbone.body.layer1.0.bn1.running_mean of shape (64,)
loaded from backbone.body.layer1.0.bn1.running_var of shape (64,)
loaded from backbone.body.layer1.0.bn1.weight of shape (64,)
loaded from backbone.body.layer1.0.bn2.bias of shape (64,)
loaded from backbone.body.layer1.0.bn2.running_mean of shape (64,)
loaded from backbone.body.layer1.0.bn2.running_var of shape (64,)
loaded from backbone.body.layer1.0.bn2.weight of shape (64,)
loaded from backbone.body.layer1.0.bn3.bias of shape (256,)
loaded from backbone.body.layer1.0.bn3.running_mean of shape (256,)
loaded from backbone.body.layer1.0.bn3.running_var of shape (256,)
loaded from backbone.body.layer1.0.bn3.weight of shape (256,)
loaded from backbone.body.layer1.0.conv1.weight of shape (64, 64, 1, 1)
loaded from backbone.body.layer1.0.conv2.weight of shape (64, 64, 3, 3)
loaded from backbone.body.layer1.0.conv3.weight of shape (256, 64, 1, 1)
loaded from backbone.body.layer1.0.downsample.0.weight of shape (256, 64, 1, 1)
loaded from backbone.body.layer1.0.downsample.1.bias of shape (256,)
loaded from backbone.body.layer1.0.downsample.1.running_mean of shape (256,)
loaded from backbone.body.layer1.0.downsample.1.running_var of shape (256,)
loaded from backbone.body.layer1.0.downsample.1.weight of shape (256,)
loaded from backbone.body.layer1.1.bn1.bias of shape (64,)
loaded from backbone.body.layer1.1.bn1.running_mean of shape (64,)
loaded from backbone.body.layer1.1.bn1.running_var of shape (64,)
loaded from backbone.body.layer1.1.bn1.weight of shape (64,)
loaded from backbone.body.layer1.1.bn2.bias of shape (64,)
loaded from backbone.body.layer1.1.bn2.running_mean of shape (64,)
loaded from backbone.body.layer1.1.bn2.running_var of shape (64,)
loaded from backbone.body.layer1.1.bn2.weight of shape (64,)
loaded from backbone.body.layer1.1.bn3.bias of shape (256,)
loaded from backbone.body.layer1.1.bn3.running_mean of shape (256,)
loaded from backbone.body.layer1.1.bn3.running_var of shape (256,)
loaded from backbone.body.layer1.1.bn3.weight of shape (256,)
loaded from backbone.body.layer1.1.conv1.weight of shape (64, 256, 1, 1)
loaded from backbone.body.layer1.1.conv2.weight of shape (64, 64, 3, 3)
loaded from backbone.body.layer1.1.conv3.weight of shape (256, 64, 1, 1)
loaded from backbone.body.layer1.2.bn1.bias of shape (64,)
loaded from backbone.body.layer1.2.bn1.running_mean of shape (64,)
loaded from backbone.body.layer1.2.bn1.running_var of shape (64,)
loaded from backbone.body.layer1.2.bn1.weight of shape (64,)
loaded from backbone.body.layer1.2.bn2.bias of shape (64,)
loaded from backbone.body.layer1.2.bn2.running_mean of shape (64,)
loaded from backbone.body.layer1.2.bn2.running_var of shape (64,)
loaded from backbone.body.layer1.2.bn2.weight of shape (64,)
loaded from backbone.body.layer1.2.bn3.bias of shape (256,)
loaded from backbone.body.layer1.2.bn3.running_mean of shape (256,)
loaded from backbone.body.layer1.2.bn3.running_var of shape (256,)
loaded from backbone.body.layer1.2.bn3.weight of shape (256,)
loaded from backbone.body.layer1.2.conv1.weight of shape (64, 256, 1, 1)
loaded from backbone.body.layer1.2.conv2.weight of shape (64, 64, 3, 3)
loaded from backbone.body.layer1.2.conv3.weight of shape (256, 64, 1, 1)
loaded from backbone.body.layer2.0.bn1.bias of shape (128,)
loaded from backbone.body.layer2.0.bn1.running_mean of shape (128,)
loaded from backbone.body.layer2.0.bn1.running_var of shape (128,)
loaded from backbone.body.layer2.0.bn1.weight of shape (128,)
loaded from backbone.body.layer2.0.bn2.bias of shape (128,)
loaded from backbone.body.layer2.0.bn2.running_mean of shape (128,)
loaded from backbone.body.layer2.0.bn2.running_var of shape (128,)
loaded from backbone.body.layer2.0.bn2.weight of shape (128,)
loaded from backbone.body.layer2.0.bn3.bias of shape (512,)
loaded from backbone.body.layer2.0.bn3.running_mean of shape (512,)
loaded from backbone.body.layer2.0.bn3.running_var of shape (512,)
loaded from backbone.body.layer2.0.bn3.weight of shape (512,)
loaded from backbone.body.layer2.0.conv1.weight of shape (128, 256, 1, 1)
loaded from backbone.body.layer2.0.conv2.weight of shape (128, 128, 3, 3)
loaded from backbone.body.layer2.0.conv3.weight of shape (512, 128, 1, 1)
loaded from backbone.body.layer2.0.downsample.0.weight of shape (512, 256, 1, 1)
loaded from backbone.body.layer2.0.downsample.1.bias of shape (512,)
loaded from backbone.body.layer2.0.downsample.1.running_mean of shape (512,)
loaded from backbone.body.layer2.0.downsample.1.running_var of shape (512,)
loaded from backbone.body.layer2.0.downsample.1.weight of shape (512,)
loaded from backbone.body.layer2.1.bn1.bias of shape (128,)
loaded from backbone.body.layer2.1.bn1.running_mean of shape (128,)
loaded from backbone.body.layer2.1.bn1.running_var of shape (128,)
loaded from backbone.body.layer2.1.bn1.weight of shape (128,)
loaded from backbone.body.layer2.1.bn2.bias of shape (128,)
loaded from backbone.body.layer2.1.bn2.running_mean of shape (128,)
loaded from backbone.body.layer2.1.bn2.running_var of shape (128,)
loaded from backbone.body.layer2.1.bn2.weight of shape (128,)
loaded from backbone.body.layer2.1.bn3.bias of shape (512,)
loaded from backbone.body.layer2.1.bn3.running_mean of shape (512,)
loaded from backbone.body.layer2.1.bn3.running_var of shape (512,)
loaded from backbone.body.layer2.1.bn3.weight of shape (512,)
loaded from backbone.body.layer2.1.conv1.weight of shape (128, 512, 1, 1)
loaded from backbone.body.layer2.1.conv2.weight of shape (128, 128, 3, 3)
loaded from backbone.body.layer2.1.conv3.weight of shape (512, 128, 1, 1)
loaded from backbone.body.layer2.2.bn1.bias of shape (128,)
loaded from backbone.body.layer2.2.bn1.running_mean of shape (128,)
loaded from backbone.body.layer2.2.bn1.running_var of shape (128,)
loaded from backbone.body.layer2.2.bn1.weight of shape (128,)
loaded from backbone.body.layer2.2.bn2.bias of shape (128,)
loaded from backbone.body.layer2.2.bn2.running_mean of shape (128,)
loaded from backbone.body.layer2.2.bn2.running_var of shape (128,)
loaded from backbone.body.layer2.2.bn2.weight of shape (128,)
loaded from backbone.body.layer2.2.bn3.bias of shape (512,)
loaded from backbone.body.layer2.2.bn3.running_mean of shape (512,)
loaded from backbone.body.layer2.2.bn3.running_var of shape (512,)
loaded from backbone.body.layer2.2.bn3.weight of shape (512,)
loaded from backbone.body.layer2.2.conv1.weight of shape (128, 512, 1, 1)
loaded from backbone.body.layer2.2.conv2.weight of shape (128, 128, 3, 3)
loaded from backbone.body.layer2.2.conv3.weight of shape (512, 128, 1, 1)
loaded from backbone.body.layer2.3.bn1.bias of shape (128,)
loaded from backbone.body.layer2.3.bn1.running_mean of shape (128,)
loaded from backbone.body.layer2.3.bn1.running_var of shape (128,)
loaded from backbone.body.layer2.3.bn1.weight of shape (128,)
loaded from backbone.body.layer2.3.bn2.bias of shape (128,)
loaded from backbone.body.layer2.3.bn2.running_mean of shape (128,)
loaded from backbone.body.layer2.3.bn2.running_var of shape (128,)
loaded from backbone.body.layer2.3.bn2.weight of shape (128,)
loaded from backbone.body.layer2.3.bn3.bias of shape (512,)
loaded from backbone.body.layer2.3.bn3.running_mean of shape (512,)
loaded from backbone.body.layer2.3.bn3.running_var of shape (512,)
loaded from backbone.body.layer2.3.bn3.weight of shape (512,)
loaded from backbone.body.layer2.3.conv1.weight of shape (128, 512, 1, 1)
loaded from backbone.body.layer2.3.conv2.weight of shape (128, 128, 3, 3)
loaded from backbone.body.layer2.3.conv3.weight of shape (512, 128, 1, 1)
loaded from backbone.body.layer3.0.bn1.bias of shape (256,)
loaded from backbone.body.layer3.0.bn1.running_mean of shape (256,)
loaded from backbone.body.layer3.0.bn1.running_var of shape (256,)
loaded from backbone.body.layer3.0.bn1.weight of shape (256,)
loaded from backbone.body.layer3.0.bn2.bias of shape (256,)
loaded from backbone.body.layer3.0.bn2.running_mean of shape (256,)
loaded from backbone.body.layer3.0.bn2.running_var of shape (256,)
loaded from backbone.body.layer3.0.bn2.weight of shape (256,)
loaded from backbone.body.layer3.0.bn3.bias of shape (1024,)
loaded from backbone.body.layer3.0.bn3.running_mean of shape (1024,)
loaded from backbone.body.layer3.0.bn3.running_var of shape (1024,)
loaded from backbone.body.layer3.0.bn3.weight of shape (1024,)
loaded from backbone.body.layer3.0.conv1.weight of shape (256, 512, 1, 1)
loaded from backbone.body.layer3.0.conv2.weight of shape (256, 256, 3, 3)
loaded from backbone.body.layer3.0.conv3.weight of shape (1024, 256, 1, 1)
loaded from backbone.body.layer3.0.downsample.0.weight of shape (1024, 512, 1, 1)
loaded from backbone.body.layer3.0.downsample.1.bias of shape (1024,)
loaded from backbone.body.layer3.0.downsample.1.running_mean of shape (1024,)
loaded from backbone.body.layer3.0.downsample.1.running_var of shape (1024,)
loaded from backbone.body.layer3.0.downsample.1.weight of shape (1024,)
loaded from backbone.body.layer3.1.bn1.bias of shape (256,)
loaded from backbone.body.layer3.1.bn1.running_mean of shape (256,)
loaded from backbone.body.layer3.1.bn1.running_var of shape (256,)
loaded from backbone.body.layer3.1.bn1.weight of shape (256,)
loaded from backbone.body.layer3.1.bn2.bias of shape (256,)
loaded from backbone.body.layer3.1.bn2.running_mean of shape (256,)
loaded from backbone.body.layer3.1.bn2.running_var of shape (256,)
loaded from backbone.body.layer3.1.bn2.weight of shape (256,)
loaded from backbone.body.layer3.1.bn3.bias of shape (1024,)
loaded from backbone.body.layer3.1.bn3.running_mean of shape (1024,)
loaded from backbone.body.layer3.1.bn3.running_var of shape (1024,)
loaded from backbone.body.layer3.1.bn3.weight of shape (1024,)
loaded from backbone.body.layer3.1.conv1.weight of shape (256, 1024, 1, 1)
loaded from backbone.body.layer3.1.conv2.weight of shape (256, 256, 3, 3)
loaded from backbone.body.layer3.1.conv3.weight of shape (1024, 256, 1, 1)
loaded from backbone.body.layer3.2.bn1.bias of shape (256,)
loaded from backbone.body.layer3.2.bn1.running_mean of shape (256,)
loaded from backbone.body.layer3.2.bn1.running_var of shape (256,)
loaded from backbone.body.layer3.2.bn1.weight of shape (256,)
loaded from backbone.body.layer3.2.bn2.bias of shape (256,)
loaded from backbone.body.layer3.2.bn2.running_mean of shape (256,)
loaded from backbone.body.layer3.2.bn2.running_var of shape (256,)
loaded from backbone.body.layer3.2.bn2.weight of shape (256,)
loaded from backbone.body.layer3.2.bn3.bias of shape (1024,)
loaded from backbone.body.layer3.2.bn3.running_mean of shape (1024,)
loaded from backbone.body.layer3.2.bn3.running_var of shape (1024,)
loaded from backbone.body.layer3.2.bn3.weight of shape (1024,)
loaded from backbone.body.layer3.2.conv1.weight of shape (256, 1024, 1, 1)
loaded from backbone.body.layer3.2.conv2.weight of shape (256, 256, 3, 3)
loaded from backbone.body.layer3.2.conv3.weight of shape (1024, 256, 1, 1)
loaded from backbone.body.layer3.3.bn1.bias of shape (256,)
loaded from backbone.body.layer3.3.bn1.running_mean of shape (256,)
loaded from backbone.body.layer3.3.bn1.running_var of shape (256,)
loaded from backbone.body.layer3.3.bn1.weight of shape (256,)
loaded from backbone.body.layer3.3.bn2.bias of shape (256,)
loaded from backbone.body.layer3.3.bn2.running_mean of shape (256,)
loaded from backbone.body.layer3.3.bn2.running_var of shape (256,)
loaded from backbone.body.layer3.3.bn2.weight of shape (256,)
loaded from backbone.body.layer3.3.bn3.bias of shape (1024,)
loaded from backbone.body.layer3.3.bn3.running_mean of shape (1024,)
loaded from backbone.body.layer3.3.bn3.running_var of shape (1024,)
loaded from backbone.body.layer3.3.bn3.weight of shape (1024,)
loaded from backbone.body.layer3.3.conv1.weight of shape (256, 1024, 1, 1)
loaded from backbone.body.layer3.3.conv2.weight of shape (256, 256, 3, 3)
loaded from backbone.body.layer3.3.conv3.weight of shape (1024, 256, 1, 1)
loaded from backbone.body.layer3.4.bn1.bias of shape (256,)
loaded from backbone.body.layer3.4.bn1.running_mean of shape (256,)
loaded from backbone.body.layer3.4.bn1.running_var of shape (256,)
loaded from backbone.body.layer3.4.bn1.weight of shape (256,)
loaded from backbone.body.layer3.4.bn2.bias of shape (256,)
loaded from backbone.body.layer3.4.bn2.running_mean of shape (256,)
loaded from backbone.body.layer3.4.bn2.running_var of shape (256,)
loaded from backbone.body.layer3.4.bn2.weight of shape (256,)
loaded from backbone.body.layer3.4.bn3.bias of shape (1024,)
loaded from backbone.body.layer3.4.bn3.running_mean of shape (1024,)
loaded from backbone.body.layer3.4.bn3.running_var of shape (1024,)
loaded from backbone.body.layer3.4.bn3.weight of shape (1024,)
loaded from backbone.body.layer3.4.conv1.weight of shape (256, 1024, 1, 1)
loaded from backbone.body.layer3.4.conv2.weight of shape (256, 256, 3, 3)
loaded from backbone.body.layer3.4.conv3.weight of shape (1024, 256, 1, 1)
loaded from backbone.body.layer3.5.bn1.bias of shape (256,)
loaded from backbone.body.layer3.5.bn1.running_mean of shape (256,)
loaded from backbone.body.layer3.5.bn1.running_var of shape (256,)
loaded from backbone.body.layer3.5.bn1.weight of shape (256,)
loaded from backbone.body.layer3.5.bn2.bias of shape (256,)
loaded from backbone.body.layer3.5.bn2.running_mean of shape (256,)
loaded from backbone.body.layer3.5.bn2.running_var of shape (256,)
loaded from backbone.body.layer3.5.bn2.weight of shape (256,)
loaded from backbone.body.layer3.5.bn3.bias of shape (1024,)
loaded from backbone.body.layer3.5.bn3.running_mean of shape (1024,)
loaded from backbone.body.layer3.5.bn3.running_var of shape (1024,)
loaded from backbone.body.layer3.5.bn3.weight of shape (1024,)
loaded from backbone.body.layer3.5.conv1.weight of shape (256, 1024, 1, 1)
loaded from backbone.body.layer3.5.conv2.weight of shape (256, 256, 3, 3)
loaded from backbone.body.layer3.5.conv3.weight of shape (1024, 256, 1, 1)
loaded from backbone.body.stem.bn1.bias of shape (64,)
loaded from backbone.body.stem.bn1.running_mean of shape (64,)
loaded from backbone.body.stem.bn1.running_var of shape (64,)
loaded from backbone.body.stem.bn1.weight of shape (64,)
loaded from backbone.body.stem.conv1.weight of shape (64, 3, 7, 7)
loaded from roi_heads.box.causal_predictor.Wy.bias of shape (1024,)
loaded from roi_heads.box.causal_predictor.Wy.weight of shape (1024, 1024)
loaded from roi_heads.box.causal_predictor.Wz.bias of shape (1024,)
loaded from roi_heads.box.causal_predictor.Wz.weight of shape (1024, 1024)
loaded from roi_heads.box.causal_predictor.causal_score.bias of shape (80,)
al_score.weight loaded from roi_heads.box.causal_predictor.causal_score.weight of shape (80, 2048)
loaded from roi_heads.box.predictor.cls_score.bias of shape (80,)
loaded from roi_heads.box.predictor.cls_score.weight of shape (80, 1024)
br> in load_state_dict
load_state_dict
1024]) from checkpoint, the shape in current model is torch.Size([80, 2048]).
shape torch.Size([80, 2048]) from checkpoint, the shape in current model is torch.Size([80, 4096]).
1024]) from checkpoint, the shape in current model is torch.Size([1024, 2048]).
1024]) from checkpoint, the shape in current model is torch.Size([1024, 2048]).

The checkpoints I'm using is your pretrained model_final.pth. Thank you.

Answer 3 · 2020-08-11T06:02:31.000Z

Hi, Thanks for your interests to our work.
From your last issue about pretrained model loading, it seemed that is the mismatch between the pretrained model and your model on causal predictor.

Please note that the config we use is the e2e_mask_rcnn_R_101_FPN_1x.yaml.
Please check here about the embedding size. The original value is 1024 and it seemed that you change it to 2048. That's different from the pretrained model.

Answer 4 · 2020-08-11T06:03:44.000Z

Plus that transforms.ToTensor() is from torchvision. The cfg I am using is VC-R-CNN/configs/caffe2/e2e_faster_rcnn_R_50_C4_1x_caffe2.yaml

For the R_50_C4, the head structural is different as I mentioned above. That means, you cannot use the pretrained model.

Answer 5 · 2020-08-12T01:58:41.000Z

Hi,
Thank you for your help. Seem like the problem solved, but I still got another mismatch tensor size issue:
Traceback (most recent call last):
File "test_net.py", line 191, in
main()
File "test_net.py", line 148, in main
single_inference(model, img, boxes, "cuda")
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\engine\inference.py", line 189, in single_inference
prediction = single_image_compute(model, img, boxes, device, inference_timer)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\engine\inference.py", line 89, in single_image_compute
output = model(pil_img.to(device), targets)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\detector\generalized_rcnn.py", line 52, in forward
features = self.backbone(images.tensors)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\container.py", line 100, in forward
input = module(input)
File "D:\Anaconda3\envs\vc_rcnn\lib\site-packages\torch\nn\modules\module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "D:\Video_Surveillance_project\VC-R-CNN\vc_rcnn\modeling\backbone\fpn.py", line 64, in forward
last_inner = inner_lateral + inner_top_down
RuntimeError: The size of tensor a (51) must match the size of tensor b (52) at non-singleton dimension 3

Answer 6 · 2020-08-14T12:03:12.000Z

Hi, it seems the problem is in the backbone (resnet). This part actually I don't modify. So maybe the problem is about the wrong input image or you change the resnet architecture.