Use model at inference

[oconnor127]

Hey,
first things first: Great paper!
I am currently trying to run your model at inference and therefor used the script demo/demo.py and passed the arguments
--config-file ifc_repo/configs/COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x.yaml
-output <path_to_output_file>
--video-input <path_to_input_file>
--opts MODEL.WEIGHTS detectron2://COCO-InstanceSegmentation/mask_rcnn_R_101_FPN_3x/138205316/model_final_a3ec72.pkl

Everything works fine, but I think thats not using your model right? Putting r101.pth for the WEIGHTS and R101_ytvis.yaml for the config-file does not work ("KeyError: 'Non-existent config key: MODEL.IFC' "). So how can I use your pretrained model at inference just to visualize and test for results?

[oconnor127]

Ok I think its resolved. Had to use the script in repo/projects/IFC/demo/demo.py
If thats correct I think it can be closed.

[sukjunhwang]

@oconnor127
Yes, that's the correct file. I am glad you solved it :)

[fshamsafar]

Hi,

I am running this demo IFC/projects/IFC/demo/demo.py with arguments as below. But there are some errors starting from line 52, TypeError: list indices must be integers or slices, not str..

By adding ...

predictions = predictions[0]
predictions = predictions["instances"]
image_size = predictions.image_size

the error is solved but continues in other ways, which are related to attributes of Instances.

I’d be happy if you could help :)

--config-file
/IFC/projects/IFC/configs/R101_coco.yaml
--input
/Frames/
--output
./Frames_predictions
--checkpoint
/IFC/models/r101.pth

[sukjunhwang]

Hi @fshamsafar,

Could you please elaborate what predictions holds right before entering the line 52 and how the error continues?

[fshamsafar]

Thank you @sukjunhwang for your reply.

predictions is a 1-element list as below:

[{'instances': Instances(num_instances=99, image_height=1080, image_width=1920, fields=[scores: tensor(...), pred_classes: tensor(...), pred_masks: tensor(...), pred_boxes: Boxes(tensor(…))])}]

After these lines:

predictions = predictions[0]
predictions = predictions["instances"]
image_size = predictions.image_size

it would be like:

Instances(num_instances=99, image_height=1080, image_width=1920, fields=[scores: tensor(...), pred_classes: tensor(...), pred_masks: tensor(...), pred_boxes: Boxes(tensor(…))])

And the error is:

pred_scores = predictions["pred_scores"]
File ".../IFC/detectron2/structures/instances.py", line 139, in __getitem__
ret.set(k, v[item])
IndexError: too many indices for tensor of dimension 1

In my case, I do not get to a dictionary which is needed for the following lines. Also, there is mismatching in naming, e.g. "scores" instead of "pred_scores".

[sukjunhwang]

Hi, @fshamsafar
thank you for the details.

When you are trying to visualize a video, you should use YouTube-VIS configuration file, which would be /IFC/projects/IFC/configs/R101_ytvis.yaml, instead of R101_coco.yaml you used.

Using the COCO config file directs the code to inference in COCO-style, which causes the problem.

Could you please try with the config I mentioned? Hope it solves the problem.

[fshamsafar]

Thanks @sukjunhwang :) It worked.