/YAICON-Ditto

Ditto(Doodle to Image TranslaTiOn)

Primary LanguagePython

DITTO: Doodle to Image Translation Web Project

🎊 YAICON 1st Prize!! 🎊

Members

  • 박지호: AI, BE
  • 박찬혁: PM, BE Lead
  • 안정우: Data, AI
  • 이수민: FE Lead, AI
  • 장윤호: Data, AI
  • 최정우: AI Lead


  • This is ControlNet(Latent Diffusion) Web Application Project
  • Our model generates high quality Image from prompt guided Doodle.
  • To enhance the Doodle to Image performance, we fine-tuned ControlNet using SBU Captions dataset.
  • This is the result of our web

0. Inference Results

0 1
2 3
4 5

1. ControlNet

Train

Our goal was to generate high-quality image from even more doodling sketch.
We trained pretrained ControlNet.

Dataset

To train ControlNet on sketch control, caption, sketch and target image are required.
We used SBU Captions dataset, which is large-scale dataset that contains 860K image-text pairs.

Edge Detection

To obtain Doodle of the target image, we have tested several edge detectors.
We have chose PhotoSketch, which was generating most human-like doodle.

2. Web

HomePage SketchPage ResultPage
1 2 3

How to Run

  1. Create conda virtual environment using environment.yaml
    conda env create --file environment.yaml
  1. Move to cloned folder and start django prototype server
    python3 manage.py runserver --noreload
  1. Access to localhost:8000 and enjoy!