/LandscapePainter

ccClub2022fall final project

Primary LanguagePythonMIT LicenseMIT

tags: ccClub2022fall

Landscape Painter

We provide a web application to transfer the image from a segmentation map to a photorealistic image. In the application, you can choose the SPADE or the PsP encoder with StyleGANv2 to generate your painting. We also provide the docker images and the checkpoint files that can build the application quickly on your own.

By default, we only provide 5 classes to draw. If you want to change the class, you can refer to the ADE20K document.

(Note that the drawing color is equal to the label color in ADE20K)

Table of Contents

Goals

  1. Develop a web application.
  2. Build a demo page. (with painter)
  3. Transfer the image from a segmentation map to a photorealistic image.
  4. Provide docker images and well-trained checkpoint files.

Architecture

Getting Started

Prerequisites

  • Windows10 or Windows11 or Linux
  • Nvidia GPU with driver installed. (The tutorial in here may help you)
  • For Windows user, WSL2 needs to be installed. (The tutorial in here may help you)
  • Docker installed. (The tutorial in here may help you)

Installation

  1. Clone the project.
git clone https://github.com/PoPoPoPony/LandscapePainter.git
  • Note that git should be installed.
  • If you didn't install the git, please refer to https://git-scm.com/

  1. Download the checkpoint files, check here

  2. Unzip the ckpts.zipin the LandscapePainter, which would look like Repository structure.

  3. Run the docker-compose command.

cd LandscapePainter

docker-compose up
  • Note that Docker should be installed. (The tutorial in here may help you)
  • If the frontend and the backend service run successfully, you can start painting!
  • Note that downloading the envirovment(almost 18G) and initializing the models(backend) take time. Go to have a cup of coffee first <3

Nvidia GPU driver Installation

  1. check your status. Open the CMD and enter :
nvidia-smi

You can skip this part if the information of GPU driver shows successfully.

  1. Go here to check your GPU version and download the driver. You may need to reboost your computer.

WSL2 Installation

Before using docker, the WSL2 needs to be prepared.

  1. Allow these features and reboost your computer(「控制台」→「 程式集」→「開啟或關閉 Windows 功能」中，勾選其中兩項功能「Windows 子系統 Linux 版」與「虛擬機器平台」)

  2. Install the WSL

  • Run the PowerShell as the administrator, and enter :
wsl --install -d Ubuntu
  • After setting up the username and password, you can close the window.
  1. Update the WSL to version 2.
  • Open the CMD, and enter :
wsl --set-version Ubuntu 2

  1. (Optional) For Windows 10 user, if you can't update to WSL2, you can download the package and update manually.

  2. Check your WSL environment.

  • Open the CMD, and enter :
wsl -l -v
  • Make sure you have the WSL2 environment.

Docker Installation

  1. Install Docker
  • (For Windows user) While installing the docker, make sure the WSL2 option checked.

  1. (For Windows user) Setting up the WSL backend.

    • After setting up the WSL backend, restart the docker and check the new config applied.

  2. Check the environment using the following command.

docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi
  • If all settings are ready, the following infomation below will be showed.

Repository Structure

Path Description
PsP Modules for training PsP encoder
We only implement the encoder part, others are refer to
Official PsP encoder implmentation
SPADE Modules for SPADE
backend Backend based on FastAPI
frontend Frontend based on Vue and Element-plus3
│ │ ├  Painter.vue Setting up the painter.
If you want to change the class, please modify this file.
posters Images in github readme
stylegan2-pytorch Modules for training StyleGANv2
The code is clone from rosinality's implementation
webcrawler_sophie Web crawler scripts provided by sophie
webcrawler_wen Web crawler scripts provided by wen
docker-compose.yml docker compose file
PsP_ckpt PsP checkpoint file
├  iteration_200000.pt The checkpoint file of PsP encoder. You can set your own pt file after training your PsP encoder.
SPADE_ckpt SPADE checkpoint file
├  epoche007.pt The checkpoint file of SPADE. You can set your own pt file after training your SPADE.

Project Timeline

References