ViDove: A Python repository from alex625051

🐦ViDove: End-to-end Video Translation Toolkit

Transcribe and Translate Your Video with a Single Click
Offical Website »

Try Demo · Report Bug · Request Feature

Table of Contents

Release
About The Project
Getting Started
- Installation
Usage
Contributing
License
Contact

Release

[12/20]🔥ViDove V0.1 Released: We are happy to release our initial version of ViDove: End-to-end Video Translation Toolkit.

(back to top)

About The Project

Introducing ViDove, a pioneering video automated machine translation toolkit, meticulously crafted for professional domains. Developed by Pigeon.AI, ViDove promises rapid, precise, and relatable translations, revolutionizing the workflow of subtitle groups and translation professionals. It's an open-source tool, offering unparalleled flexibility, transparency, and security, alongside scalable architecture for customization. Featuring domain adaptation, ViDove effortlessly adjusts to various professional fields, and its end-to-end pipeline turns video links into captioned content with a single click. ViDove is not just a translation tool; it's a bridge connecting content across languages, making video translation more accessible, efficient, and accurate than ever.

Here's why:

End-to-End Pipeline (from video link to captioned video):
- One-Click Deployment: Users can deploy the tool with just one click.
- Video Link to Translated Video: Simply input a video link to generate a translated video with ease.
Domain Adaptation:
- Our pipeline is adaptable to various professional fields (e.g., StarCraft II). Users can easily upload customized dictionaries and fine-tune models based on specific data corpora.
Open Source:
- Our toolkit is entirely open source, and we warmly welcome and look forward to the participation of the broader developer community in the ongoing development of the toolkit.

(back to top)

Main Contributors

Web Dev: Tingyu Su

Getting Started

Installation

Get a OpenAI API Key at https://platform.openai.com/api-keys

Clone the repo

git clone https://github.com/project-kxkg/ViDove.git
cd ViDove

Install Requirments

conda create -n ViDove python=3.10 -y
conda activate ViDove
pip install --upgrade pip
pip install -r requirements.txt

Enter your API in bash
```
export OPENAI_API_KEY="your_api_key" 
```

(back to top)

Usage

Quick Start with Gradio User Interface

python3 entries/app.py

Launch with configs

Start with Youtube Link input:

python3 entries/run.py --link "your_youtube_link"

Start with Video input:

python3 entries/run.py --video_file path/to/video_file

Start with Audio input:

python3 entries/run.py --audio_file path/to/audio_file

Terminal Usage:

usage: run.py [-h] [--link LINK] [--video_file VIDEO_FILE] [--audio_file AUDIO_FILE] [--launch_cfg LAUNCH_CFG] [--task_cfg TASK_CFG]

options:
  -h, --help            show this help message and exit
  --link LINK           youtube video link here
  --video_file VIDEO_FILE
                        local video path here
  --audio_file AUDIO_FILE
                        local audio path here
  --launch_cfg LAUNCH_CFG
                        launch config path
  --task_cfg TASK_CFG   task config path

Configs

Use "--launch_cfg" and "--task_cfg" in run.py to change launch or task configuration

configs/local_launch.yaml

# launch config for local environment
local_dump: ./local_dump # change local dump dir here
environ: local

configs/task_config.yaml

copy and change this config for different configuration

# configuration for each task
source_lang: EN
target_lang: ZH
field: General

# ASR config
ASR:
  ASR_model: whisper
  whisper_config:
    whisper_model: tiny
    method: stable
  
# pre-process module config
pre_process: 
  sentence_form: True
  spell_check: False
  term_correct: True

# Translation module config
translation:
  model: gpt-4
  chunk_size: 1000

# post-process module config
post_process: 
  check_len_and_split: True
  remove_trans_punctuation: True

# output type that user receive
output_type: 
  subtitle: srt
  video: True
  bilingual: True

(back to top)

Contributing

Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also simply open an issue with the tag "enhancement". Don't forget to give the project a star! Thanks again!