about SBR pretrained model

Question

about SBR pretrained model

kadirbeytorun opened this issue 5 years ago · 5 comments

Hey,
I used your pretrained model for SBR model, and results are impressive, though I have one question regarding that. Did you train the model only on 300W dataset, meaning it doesnt include 300VW as well?
If so, if we could train it with 300W and 300VW combined, should we expect a significant increase in accuracy?
Regards, Kadir

Answer 1 · 2019-10-18T10:33:58.000Z

Thanks for your interest and question. "Training on 300W and unlabeled 300VW" should improve the accuracy on 300VW compared to "Training on 300W only". It might not improve the accuracy on 300W. Since our SBR focuses on the temporal consistency, it will help the detector to be stable on video but might not help it to be accurate.

Answer 2 · 2019-10-18T10:52:07.000Z

Hey,
Thanks for the very fast reply. I don't use 300W or 300VW for benchmarking, but solely for training purposes. What I mean by 'increase in accuracy' is actually in real-time cases, e.g, test videos and USB cameras.

And by the way, what do you mean by unlabeled 300VW? Isn't the 300VW dataset labeled the same way as 300W, but instead of pictures, labeled in each frame?
Thanks in advance.

Answer 3 · 2019-10-19T10:17:51.000Z

In the real-time cases, the detector might not have accuracy gain, due to the distribution change between the training and test videos.

"unlabeled 300VW" means train SBR with 300VW video data without using its facial keypoint labels.

Answer 4 · 2019-10-21T06:04:15.000Z

I closed the issue, but since your answers are really improving my knowledge, I wanted to ask a couple more questions on the topic.

If we are using 300VW dataset without its annotations, can't we use just any video dataset instead of 300VW?

How about we extract the frames of 300VW, add them to 300W as individual pictures (maybe 1/5 of frames), use their annotations to create bounding box mat files, thus expanding our annotated dataset, and then use another unlabeled video dataset (like youtube dataset) for improving SBR?

Also could we add Menpo face landmark dataset to increase our dataset, assuming we created it's mat files and got it to same format as 300W?

Answer 5 · 2019-10-21T06:25:15.000Z

Thanks for your interest.

1, Yes, we can use any video dataset. We use 300VW just because they are popular.

2, Yes, it is good idea.

3, Yes, you can. You do not need to create mat file, you just need to create the list file as 300W or 300VW (https://github.com/D-X-Y/landmark-detection/blob/master/SBR/scripts/sbr_example.sh#L3).