Suggestions to fix the bugs when you render

[DongChengdongHangZhou]

landmark numpy is missing:

The authors didn't provide the landmark_embedding.npy, and I finally find it on https://github.com/yfeng95/DECA/tree/master/data
you can download it from this weblink

Render the face avatar images:

The authors of this repository made a mistake, when you run the eval_splatting_avatar.py, you should organize the checkpoints as follows:

/output-splatting/last_checkpoint
|---eval_30000
|---image
|---point_cloud
|---config.yaml
|---flame_params.json

To organize the checkpoints, we first download the data from the provided link - Google Drive
For example, I want to evaluate on the data and pretrained checkpoint of obama.
Then we run the following commands:

cd SplattingAvatar
mkdir results
cd results

put your obama.7z under SplattingAvatar/results/obama.7z
unzip the 7z file

7za x  obama.7z
rm -rf obama.7z

move the flame_params.json file, move the /images folder and rename it as /image

mv obama/flame_params.json obama/output-splatting/last_checkpoint/flame_params.json
mv  obama/images obama/output-splatting/last_checkpoint/image

then go back to the folder /SplattingAvatar

Finally, you should run the command as follows:

python eval_splatting_avatar.py --config configs/splatting_avatar.yaml --dat_dir results/obama/output-splatting/last_checkpoint --pc_dir results/obama/output-splatting/last_checkpoint/point_cloud/iteration_30000

You will get the rendered results in the folder:
results/obama/output-splatting/last_checkpoint/point_cloud/iteration_30000/eval/render

if you want to acquire the rendered images with the pose in training dataset:
change the sentence in eval_splatting_avatar.py from

frameset_test = make_frameset_data(config.dataset, split='test')

to

frameset_test = make_frameset_data(config.dataset, split='train')

Render the full-body avatar images:

We use male-3-casual as an example:
download the peopleSnapShot data from PeopleSnapshot, unzip it and you can find a folder named male-3-casual
download the pretrained weights from Google Drive, unzip it and you can find a folder also named male-3-casual

merge the contents in the two male-3-casual folders mentioned above, and put the new male-3-casual folder under the directory results/male-3-casual

then preprocess

python scripts/preprocess_PeopleSnapshot.py --root results --outdir results/processed_male-3 --subject male-3-casual

then move the config.json

mv results/male-3-casual/config.json results/processed_male-3/male-3-casual/config.json

render:

python eval_splatting_avatar.py --config "configs/splatting_avatar.yaml;configs/instant_avatar.yaml" --dat_dir results/processed_male-3/male-3-casual  --pc_dir results/male-3-casual/output-splatting/last_checkpoint/point_cloud/iteration_30000

Find the rendered images in
results/male-3-casual/output-splatting/last_checkpoint/point_cloud/iteration_30000/eval/render

[seablue77]

@DongChengdongHangZhou
Following your steps, it still doesn't work. images is empty.