/manga-translator

A manga translator built with python

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

Manga Translator

Why ?

  • Boy read manga.
  • Boy reach latest chapter.
  • Chapter not translated.
  • Boy get mad.
  • Boy do code instead of waiting like a normal person.

How ?

  • Yolo for bubble recognition and text segmentation
  • Open CV for bubble and masking
  • PIL for text replacement
  • Api's or neural networks for translation
  • Deepfillv2 for inpainting and bubble cleanup

Progress

  • Bubble inpainting using deepfillv2
  • Bubble recognition (should improve with more training data)
  • Free text recognition (should improve with more training data)
  • Bubble text extraction
  • Bubble masking
  • Bubble text insertion
  • Bubble Text ocr
  • Bubble Text translation
  • Bubble Text hypenation
  • Format and structure dataset
  • Create converters i.e. yolo => coco, my dataset => yolo etc
  • Detection of text color
  • Better free text ocr
  • Better free text translation
  • Improve text resize algorithm, some texts are too small/big
  • Implement lama for cleaning

Models

  • Available in the latest release

Install

  • setup Anaconda
  • clone this repository
  • Open a terimal in the cloned repository
  • Run the following
conda create -n manga-translator python=3.10
conda activate manga-translator
conda install -c conda-forge poetry

poetry install
# For cuda support run "poe force-cuda"
  • Download models to models/modelname (i.e. models/detection.pt)

Usage

UI

poe run-server

CLI

python main.py -f image1.png image2.png

Datasets

Detection

Segmentation

assets/examples

Original Translated
Original Result
Japanese => English
Jujutsu Kaisen
Original Result
Japanese => "Meow"
Solo Leveling
Original Result
Japanese => Clean
The Rising of the Shield Hero
Original Result
Japanese => English
A Certain Scientific Accelerator
Original Result
Japanese => English
One Punch Man
Original Result
Japanese => English
Oshi No Ko

Glossary

  • Bubble: a speech bubble
  • Free text: text found on pages but not in speech bubbles
  • Bubble Text: text within speech bubbles