zhangzjn/APB2FaceV2

How to use this on custom videos

niladridutt opened this issue · 5 comments

niladridutt commented

Right now the dataloader can only parse ANNVii dataset it seems. How can I use it on custom videos? How to get the .t7 file?

zhangzjn commented

You need to get synchronized image, audio, and landmark data for your own dataset. Details to generate the .t7 file can be found in data/AnnScripts/script_AnnVI.py.

  • 👍1
niladridutt commented

Thank you so much for your reply and uploading the scripts!
What do you mean by "synchronized audio, image, pose, landmark, and blink information" in the paper? And how do I get it (for example dlib 68 shape predictor?)? It'll be really helpful if you could explain what is synced images. Thanks!

zhangzjn commented
  • Synchronization means that the image and audio have the same timestamp, so that the audio can be used to control the facial movement.
  • For the landmark detector, you can refer to https://github.com/davisking/dlib or https://www.faceplusplus.com.cn/face-detection
niladridutt commented

Thanks!
Do you have any script that generates the landmarks of the images and cropped images in the format needed by script_AnnVI.py ? It'll be very helpful if you could upload them as well.
Essentially if you have a script that generates landmark files from the video it'll be great

zhangzjn commented

I'm sorry, but you can refer to str_landmark to create the custom dataset that matches script_AnnVI.py.