fynv/ScoreDraft

Some questions about ScoreDraft

melspectrum007 opened this issue · 1 comments

melspectrum007 commented

It's nice job about singing synthesizer. There are some questions about ScoreDraft:
(1) which pitch shifting method using in ScoreDraft?
(2) which time shifting method using in ScoreDraft?
(3) It seems there are some noise in singing voice synthesizer case, so how to reduce noise or solve it?
(4) from the source code, ScoreDraft is concatenated based method, do ScoreDraft have plan use HMM or DNN based method?
(5) What about the future plan of next version?

fynv commented

(1) Which pitch shifting method using in ScoreDraft?
(2) Which time shifting method using in ScoreDraft?
The current solution is mixture of short-time FFT analysis + HNM(harmony noise model)
By stretching and interpolating time domain windows (harmony part) or frequency domain coefficients (noise part), pitch shifting and time shifting can be adjusted independently.
I will write a blog about the details when I got the time..
If you can read Chinese, see below link
https://www.zhihu.com/question/26165668/answer/318431652
I wrote it earlier when ScoreDraft didn't have HNM yet, and consonants are treated the same way as vowels. That is closer to PSOLA in concept.
(3) It seems there are some noise in singing voice synthesizer case, so how to reduce noise or solve it?
Noises are caused by different reasons. Very complicated to solve.
(4) from the source code, ScoreDraft is concatenated based method, do ScoreDraft have plan use HMM or DNN based method?
Not yet.. That involves fundamental restructuring of voicebanks. Not going to do that in predicable future.
(5) What about the future plan of next version?

  1. As you have noticed, there are still some quality issues just using UTAU voicebanks.
    I think that should be the first the solve. And there are some kinds of UTAU voicebanks I've not studied yet.
  2. Instrumental synthesis is also too simple currently. I found an interesting project on GitHub:
    https://github.com/LMMS/lmms
    A quite powerful musical software for Linux. Will study that and improve ScoreDraft. (Add support for SF2, vst etc)
  3. Kanru has opensourced some core algorithms behind moresampler.
    https://github.com/Sleepwalking/libllsm2
    I believe that could be a short-cut to high-quality singing synthesis. Will study that code thoroughly.