An exploratory face tuning program built to investigate the face space generated by OpenFace's nn4.small2.v1. The program, when socketed in to a dataset similar in construction to VGGFace2 and a spreadsheet of some of those faces openface 128-value representations, will use Satya Mallick's OpenCV face averaging algorithm to display an interpolated face that might appear at a user-determined location in the face space.
This program was built as an extension of a paper I wrote showing that the high dimensional organization of neural network learned face spaces (specifically, this one) is perceptible by humans. That paper is contianed in this repository.
- numpy
- pandas
- scipy
- dlib
- opencv
- matplotlib
- openface (optional)
Clone this repo with
git clone https://github.com/blakemoya/charactercreation
Unfortunately, this program works off of a local copy of VGGFace2, which can be downloaded here but is incredibly large. Beware.
In the character_creation.py main call, there a five configuration variables to set.
- CSV_PATH: this string should be the path to thge csv file where you are storing the openface calculated representations of the faces to be included in the interpolation.
- VGG_TRAIN_PATH: this string should be the path where you are storing the VGGFace2 "train" folder (or similarly constructed dataset).
- NUM_NEIGHBORS: this integer sets the number of nearest neighbors to be included in the image interpolation.
- EXAG_FACTOR: this integer is similar to a weighting of how much more closer faces should matter in the summation as images are averaged.
- NUM_IMAGES: this integer sets the number of images to be averaged in each interpolation.
Once configuration variables are set, run character_creation.py
A panel of the 128 measurements utilized by nn4.small2.v1 appears on the left side of the window, with a viewing panel on the right and a navigation slider at the bottom.
Click on the button with the axis you intend to move on and left click and drage the slider to manipulate that axis' value, or left click to instantly change it.
As you move each slider, that variables value will be reset, and then the active location of the interpolation will be projected back down onto nn4.small2.v1's output space before the inteprolation is generated.
