Whether to support sound streaming input audio2face to drive lip shape？
Closed this issue · 9 comments
thank you for your great work，May I ask if this project supports sound streaming into audio2face to drive lip shapes, rather than in the form of voice files?
Hey,
I haven't implemented that yet, however I think it's not that complicated, the headless server of Nvidia supports it.
Feel free to contribute, I would appreciate!
Best regards
感谢无私奉献，能否进一步实现实时流式驱动？
Confirmed. Will work on it. The project will be improved and continued.
Started working on it. In a second step I will also move away from plain requests and build on fastSDK and media-toolkit. Second step will take longer though. I'll release an upgraded module first - can't tell you yet how long it takes, but I expect to finish in a few weeks.
@singelhero @CasonTsai can you give me a little bit more information where your input stream comes from?
Do you expect a live microphone audio input? What are your aims / projects you're working on about?
How would this feature look like in your opinion?
For the beginning I would implement chunking an audio file and feeding it to the stream.
Finished streaming implementation.
@singelhero, @CasonTsai please check feature especially interesting wiht livelink. Please share result. Raise new issue if something is missing or misbehaving.
Next steps: fastSDK
@w4hns1nn yes,
@singelhero @CasonTsai can you give me a little bit more information where your input stream comes from? Do you expect a live microphone audio input? What are your aims / projects you're working on about? How would this feature look like in your opinion?
For the beginning I would implement chunking an audio file and feeding it to the stream.
@singelhero @CasonTsai can you give me a little bit more information where your input stream comes from? Do you expect a live microphone audio input? What are your aims / projects you're working on about? How would this feature look like in your opinion?
For the beginning I would implement chunking an audio file and feeding it to the stream.
yeah ,i want to build a metahuman in unreal.the input stream from microphone audio input or audio data in memory .i found a solution in audio2face official doc,it use grpc in that document.
excuse me,I want to know if there is a way to use audio2face without opening omniverse?
@CasonTsai
I've added the streaming feature which uses gRPC. Check the updated readme for it.
About the second question:
- please check if the headless server also uses omniverse.. py_audio2face runs with the headles..
- I plan something similar, but will be end of the year until i can publish something about
@CasonTsai I've added the streaming feature which uses gRPC. Check the updated readme for it. About the second question:
- please check if the headless server also uses omniverse.. py_audio2face runs with the headles..
- I plan something similar, but will be end of the year until i can publish something about
thanks