r9y9/pyopenjtalk

How can I get phoneme durations?

n5-suzuki opened this issue · 2 comments

n5-suzuki commented

Hi, r9y9. Thank you for your greatest work!

I am getting phoneme durations by OpenJTalk below shell command.

echo $sentence | open_jtalk -x $jtalk_path/mecab-naist-jdic \
-m $jtalk_path/MMDAgent_Example-1.7/Voice/mei/mei_normal.htsvoice \
-ow wavs/temp.wav -ot logs/$log_name

## Some processing to logs/$log_name

So I also want to get phoneme durations by your pyopenjtalk.
For example,

['0/300000/xx^xx-sil+k=o/A:xx+xx+xx/B:xx-xx_xx/C:xx_xx+xx/D:xx+xx_xx/E:xx_xx!xx_xx-xx/F:xx_xx#xx_xx@xx_xx|xx_xx/G:5_5%0_xx_xx/H:xx_xx/I:xx-xx@xx+xx&xx-xx|xx+xx/J:1_5/K:1+1-5',
'3000000/600000/xx^sil-k+o=N/A:-4+1+5/B:xx-xx_xx/C:09_xx+xx/D:xx+xx_xx/E:xx_xx!xx_xx-xx/F:5_5#0_xx@1_1|1_5/G:xx_xx%xx_xx_xx/H:xx_xx/I:1-5@1+1&1-1|1+5/J:xx_xx/K:1+1-5',
'600000/1000000/sil^k-o+N=n/A:-4+1+5/B:xx-xx_xx/C:09_xx+xx/D:xx+xx_xx/E:xx_xx!xx_xx-xx/F:5_5#0_xx@1_1|1_5/G:xx_xx%xx_xx_xx/H:xx_xx/I:1-5@1+1&1-1|1+5/J:xx_xx/K:1+1-5',
...
...
]

I guess it is probably impossible to get it in its current code, so I will need to make some modifications and re-build it.

Thank you!

r9y9 commented

There's no C interface to get phoneme durations in HTS Engine API (we could get durations from file using https://github.com/r9y9/hts_engine_API/blob/b7e1c8b51787e19ea4376176afd9707c3c9d599a/src/include/HTS_engine.h#L465-L466 though). We would need the following changes:

  • Add a function to hts_engine_API (HTS_Engine_get_label or something)
  • Write a python wrapper like

    pyopenjtalk/pyopenjtalk/htsengine.pyx

    Lines 96 to 103 in 7ee892c

    def get_generated_speech(self):
    """Get generated speech"""
    cdef size_t nsamples = HTS_Engine_get_nsamples(self.engine)
    cdef np.ndarray speech = np.zeros([nsamples], dtype=np.float64)
    cdef size_t index
    for index in range(nsamples):
    speech[index] = HTS_Engine_get_generated_speech(self.engine, index)
    return speech
n5-suzuki commented

@r9y9
Thank you for your reply!

There's no C interface to get phoneme durations in HTS Engine API

I understood . I will try to get durations from HTS_engine and re-build pyopenjtalk!