ExprGen: Learning to Generate 3D Stylized Character Expressions from Humans

paper | database

This repository contains python code to generate facial expressions on 3D MAYA rigs. There are three files in this repository:

savetomaya.py, which reads the rig parameters and their values from a CSV file and sets the parameter values of the 3D rig.
extractfrommaya.py, which extracts rig parameters and their values from a 3D rig and saves them in a CSV file.
renderfrommaya.py, which renders a given time frame from the rig as an image.

An example CSV file, input.csv, is provided in this repository. The format of the CSV files is as follows:

The first row contains the names of the facial parameters of the 3D rig.
Each of the remaining rows contains the values of those parameters for a particular expression.
The first column contains the name of the expression. The expression category is mentioned within the name.

Fill the agreement form linked on the database webpage to get access to the full versions of all the CSV files.

Prerequisites:

Install MAYA. Optionally, you can learn more about MAYA controls here.
Set up system environment for mayapy according to these instructions.

Basic usage:

Download the RAR file from here, extract the .mb (MAYA) file, and save it as Mery.mb in the current folder.
Run the following command from the terminal:

$ mayapy savetomaya.py

This will overwrite the original Mery.mb file (not recommended!). To avoid this, you can rename the file to be saved to Mery_new.mb using:

$ mayapy savetomaya.py -rename

In case the terminal cannot run mayapy directly, you can specify the entire path to mayapy.exe to run it. On MacOS with MAYA version 2018, you can do:

$ /Applications/Autodesk/maya2018/Maya.app/Contents/bin/mayapy savetomaya.py -rename

Full usage:

By default, the python file reads input.csv (input csv file) and Mery.mb (input rig file) from the current directory. Once you have the MAYA files and CSV files for all the four characters in our dataset, you can change the paths to the input files using:

$ mayapy savetomaya.py -icsv /path/to/csv/file -irig /path/to/rig/file -rename

You can then open the newly generated MAYA file in MAYA and see the results. You can also render the current view to get the results in 2D.

In case you animate the rig within MAYA (manipulate the rig parameter values), you can extract the new parameter values using:

$ mayapy extractfrommaya.py -char <character name> -irig /path/to/rig/file

For example, to extract parameter values for character Mery from Meryfile.mb, use:

$ mayapy extractfrommaya.py -char Mery -irig Meryfile.mb

and this will generate a CSV file named Meryfile.csv in the current directory.

To render a particular time frame using a particular camera (see inside the file for these settings), use:

$ mayapy renderfrommaya.py -irig /path/to/rig/file -dirname /path/to/folder/where/image/will/be/saved

Citation:

If you use this dataset, please cite our paper:

@inproceedings{learningstylizedcharacters, 
  author={D. {Aneja} and B. {Chaudhuri} and A. {Colburn} and G. {Faigin} and L. {Shapiro} and B. {Mones}}, 
  booktitle={2018 IEEE Winter Conference on Applications of Computer Vision (WACV)}, 
  title={Learning to Generate 3D Stylized Character Expressions from Humans}, 
  pages={160-169}, 
  year={2018}, 
  month={March}
}