/SketchMetaFace

This repository includes the prototype system of SketchMetaFace.

Primary LanguagePython

SketchMetaFace

This repository includes the prototype system of SketchMetaFace.

SketchMetaFace: A Learning-based Sketching Interface for High-fidelity 3D Character Face Modeling

Zhongjin Luo, Dong Du, Heming Zhu, Yizhou Yu, Xiaoguang Han, Hongbo Fu

Introduction

We present SketchMetaFace, a novel sketching system designed for amateur users to create high-fidelity 3D character faces. With curvature-aware strokes (valley strokes in green and ridge strokes in red), novice users can smoothly customize detailed 3D heads. Note that our system only outputs geometry without texture and texturing is achieved using commercial modeling tools.

| Paper | Project |

Demo

demo1_1.mp4
demo1_2.mp4
demo2_1.mp4
demo2_2.mp4

Usage

This system has been tested with Python 3.8, PyTorch 1.7.1, CUDA 10.2 on Ubuntu 18.04.

  • Installation:

    conda create --name SketchMetaFace -y python=3.8
conda activate SketchMetaFace
pip install -r requirements.txt

  • Start by cloning this repo:

    git clone git@github.com:zhongjinluo/SketchMetaFace.git
cd SketchMetaFace

  • Download pre-compiled user interface and checkpoints for backend algorithms from sketchmetaface_files.zip and then:

    unzip sketchmetaface_files.zip
unzip App.zip
mv sketchmetaface_files/Stage1/Global/* /path-to-repo/Stage1/Global/
mv sketchmetaface_files/Stage2/Preview/* /path-to-repo/Stage2/Preview/
mv sketchmetaface_files/Stage2/Suggestion/* /path-to-repo/Stage2/Suggestion/
mv sketchmetaface_files/Stage2/Local/* /path-to-repo/Stage2/Local/

  • Run the backend servers for two-stage modeling:

    # stage 1
cd /path-to-repo/Stage1/Global/ && bash server.sh

# stage 2
cd /path-to-repo/Stage2/Preview/ && bash server.sh
cd /path-to-repo/Stage2/Suggestion/ && bash server.sh
cd /path-to-repo/Stage2/Local/ && bash server.sh

  • Launch the user interface and enjoy it:

    cd App/ && bash run.sh
    • stage 2 hot key: Y - symmetrize stroke, ↥ - zoom in stroke, ↧ - zoom out stroke, P - predict,

  • If you want to run the backend algorithms on a remote server, you may have to modify App/config.ini.

  • This repo represents the prototype implementation of our paper. Please use this for research and educational purposes only. This is a research prototype system and made public for demonstration purposes. The user interface runs on Ubuntu 18.04 platforms only and may contain some bugs.

  • If you are interested in sketch-based 3D modeling, you can also refer to SimpModeling and Sketch2RaBit.