DanielSWolf/rhubarb-lip-sync

Phoneme set?

b-aaz opened this issue ยท 12 comments

b-aaz commented

Hi thanks for the grate program it's very nice
I worked with papagayo for a while and it had a folder with phoneme set files Here
And you could make some custom phoneme sets for it
But I searched for the same in the rhubarb files but couldn't find such thing (just found that they are called shape sets๐Ÿ˜ฌ) is such a thing possible in rhubarb?๐Ÿค”

I'm afraid that's not possible with Rhubarb. Rhubarb uses a more complex logic for its animation, which cannot be configured without changing the source code.

b-aaz commented

I'm afraid that's not possible with Rhubarb. Rhubarb uses a more complex logic for its animation, which cannot be configured without changing the source code.

Hi thanks for your answer I found the animationRules.cpp file and I think this file is the "phoneme set" and I sort of figured out how it works

But I tried to compile the source with out editing any files but there was a error at the line 9 of exceptions.cpp in tools folder telling that the rethrow_if_nested(e); wasn't declared in this scope
I tried to remove the line and continue compiling
But there was another error at line 43 of ProgressBar.cpp in tools folder telling that 'sleep_for' isn't a member of std::this_thread
I removed that line too but after the compile was finished rhubarb failed when you gave it file
Is there any workaround for this problem?
My os is arch and updated recently

What version of what C++ compiler are you using?

b-aaz commented

What version of what C++ compiler are you using?

G++ 11.1.0

I'm on holiday right now, researching with my smartphone, which makes things rather awkward. But here goes: Both places you mentioned are wrong indeed. They just happen to work with all compilers I use.

  • The line in exceptions.cpp should be prefixed with std::.
  • ProgressBar.cpp is missing an #include <thread> directive.

I hope this helps. If it does, feel free to send me a PR!

b-aaz commented

Hi I compiled it but
The line 9 of exceptions.cpp didn't work it just said that rethrow_if_nested is not a member of std but I think it wasn't that much important because the ProgressBar.cpp worked and the program is working

And BTW iam not in hurry you can answer this later

It may compile without this line, but the error handling will be broken.

I think the file is missing an #include <exception> directive at the top in addition to the std:: prefix in the affected line.

b-aaz commented

It worked thanks ๐Ÿ˜ƒ

b-aaz commented

Hello again
So I edited Shape.cpp Shape.h and shapeShorthands.h successfully and add 3 new shapes and it compiled till there I edited animationRule.cpp to but I couldn't understand ๐Ÿค” the matrix at line 35 it said that it a matrix that for each has all shapes in ascending order of "effort " but iam not seeing a pattern in the matrix and it dosent compile
I added the new shapes like this
A, B, C, D, E, F, LastBasicShape = F, G, H, I, J, K, X,

effortMatrix needs to be an NxN matrix, where N is the number of mouth shapes. So in order to add one new mouth shape, you'll have to add one more row and one more column.

The rows need to be in the same order as the Shape enum (as indicated by the comments at the start of each line). Within each row, you need to list all N mouth shapes in descending order of visual similarity with the mouth shape to which the row is dedicated.

For instance, let's consider the first row of the matrix. That's the row for the A shape, which is the closed mouth used for sounds like "m" and "b". Trivially, the shape most similar to A is A itself, so the first column contains A. Another mouth shape that looks very similar to A is X, the idle mouth shape, so it's in the second column. The next-similar mouth shape is G, which is almost closed and used for "f" sounds. And so on.

As you can see, adding new mouth shapes is possible but certainly not easy. Maybe I can simplify this process in a future version of Rhubarb. May I ask what your motivation is? Why do you want to add more mouth shapes?

b-aaz commented

Thank you i will try this!
And of course I actually want to make a adobe animate style like phoneme set so I can use my old phoneme set (I don't want to change the phoneme set because it will change style of the animation I think ๐Ÿ˜ฌ)
But the main goal is to spreate B mouth shape in to two so that you can use more exaggerated "th" or "s" mouth shapes like tongue between teeth or etc

I see!