How to create npy file from existing video for face and eyes movement
nitinmukesh opened this issue · 3 comments
Please excplain, how to generate npy file?
NPY is a binary file format for storing data from the NumPy Python package.
To extract Hubert features for inference of your own audio, please refer to this documentation to obtain the Hubert feature. After extraction, place the feature path to test_hubert_path. We also provide the pre-extracted features for the audios in test_demos/audios at URL for your testing.
The above information can be found on README.
If you have trouble extracting the features and have something urgent, you can send them to my email, liutaw@sjtu.edu.cn. I will help you extract them. Please try to keep the audio clips short and don't send too many. Thanks.
I wil not trouble you during your personal time.
The speed is very very fast which is very good.
I guess there are some issues related to eyes focus and lip movement. I got poor results using hubert method.
If I use custom image (using to create intro talking head video with EchoMimic and got good results) the results are very very bad. I guess if you choose to fix those issues, this will be used widely as it will even work on low VRAM systems.
I am going to do more tests.
FACE1-monalisa.mp4
I think image having face and neck only should work.
PS: And I wasn't trying to scare anyone. ;)
Here is the output from EchoMimic using same image (intro video)
https://www.youtube.com/watch?v=WtHdvSSQlWo