GalaxyCong/3Settings

3Settings

We will release the detailed configuration for all experiment settings for the GRID and V2C Animation dataset, like the ".txt" files of reference audio.

• 3Settings

  • 1. Result of Dubbing Setting1
  • 2. Result of Dubbing Setting2 🔥
    • 2.1 The GRID Setting2 Results
    • 2.2 The V2C Animation Setting2 Results
  • 3. Result of Dubbing Setting3 (Zero-Shot Dubbing) 🔥🔥
    • 3.1 Female voice actors dubbing female characters
    • 3.2 Female voice actors dubbing male characters
    • 3.3 Male voice actors dubbing male characters
    • 3.4 Male voice actors dubbing female characters

1. Result of Dubbing Setting1

• Text: “Yes, I’m the baby Jesus”

---

Ground Truth:

0_GT.mp4

👇👇 The Generated result:

1_Fastspeech2.mp4

2_Stylespeech.mp4

3_V2C-Net.mp4

4_Zeroshot-TTS.mp4

5_FaceTTS.mp4

6_HPMDubbing.mp4

Our_StyleDubber.mp4

2. Result of Dubbing Setting2

2.1 The GRID Setting2 Results

(Reference audio is the Same-speaker from other video clips)

• Reference audio:

  Grid_S10.mp4

• Text: “place red with m eight now”

---

Ground Truth:

0_GT.mp4

👇👇 The Generated result:

1_Fastspeech2.mp4

2_Stylespeech.mp4

3_V2C-Net.mp4

4_Zeroshot-TTS.mp4

5_FaceTTS.mp4

6_HPMDubbing.mp4

Our_StyleDubber.mp4

---

2.2 The V2C Animation Setting2 Results

(Reference audio is the Same-speaker from other video clips)

• Reference audio:

  FrozenII@Anna.mp4

• Text: “You are not responsible for their choices, elsa.”

---

Ground Truth:

👇👇 The Generated result:

1_Fastspeech2.mp4

2_Stylespeech.mp4

3_V2C-Net.mp4

4_Zeroshot-TTS.mp4

5_FaceTTS.mp4

6_HPMDubbing.mp4

Our_StyleDubber.mp4

3. Result of Dubbing Setting3

(Zero-shot Test)

3.1 Female voice actors dubbing female characters

• Reference audio:

  Grid-S29.mp4

• Text: “Why did you bring me here?”

• Dubbing_Video_Raw (Providing silence video information)

  Frozen@Elsa.mp4

---

Ground Truth:

⚠️ There are no ground-truth (GT) result, since this test is zero-shot testing for unseen speakers.

👇👇 The Generated result:

1_Fastspeech2.mp4

2_Stylespeech.mp4

3_V2C-Net.mp4

4_Zeroshot-TTS.mp4

5_FaceTTS.mp4

6_HPMDubbing.mp4

Our_StyleDubber.mp4

3.2 Female voice actors dubbing male characters

• Reference audio:

  Grid_S16.mp4

• Text: “I thought you would understand.”

• Dubbing_Video_Raw (Providing silence video information)

  Cloudy@Earl.mp4

---

Ground Truth:

⚠️ There are no ground-truth (GT) result, since this test is zero-shot testing for unseen speakers.

👇👇 The Generated result:

1_Fastspeech2.mp4

2_Stylespeech.mp4

3_V2C-Net.mp4

4_Zeroshot-TTS.mp4

5_FaceTTS.mp4

6_HPMDubbing.mp4

Our_StyleDubber.mp4

3.3 Male voice actors dubbing male characters

• Reference audio:

  Grid_S32.mp4

• Text: “I can't help. I can't help anyone.”

• Dubbing_Video_Raw (Providing silence video information)

  Toy@Buzz.mp4

---

Ground Truth:

⚠️ There are no ground-truth (GT) result, since this test is zero-shot testing for unseen speakers.

👇👇 The Generated result:

1_Fastspeech2.mp4

2_Stylespeech.mp4

3_V2C-Net.mp4

4_Zeroshot-TTS.mp4

5_FaceTTS.mp4

6_HPMDubbing.mp4

Our_StyleDubber.mp4

3.4 Male voice actors dubbing female characters

• Reference audio:

  Grid-S2.mp4

• Text: “She was my whole world.”

• Dubbing_Video_Raw (Providing silence video information)

  ToyII@Jessie_00_0607_00.mp4

---

Ground Truth:

⚠️ There are no ground-truth (GT) result, since this test is zero-shot testing for unseen speakers.

👇👇 The Generated result:

1_Fastspeech2.mp4

2_Stylespeech.mp4

3_V2C-Net.mp4

4_Zeroshot-TTS.mp4

5_FaceTTS.mp4

6_HPMDubbing.mp4

Our_StyleDubber.mp4