ShirleyMaxx/VirtualMarker

Questions about Camera Intrinsics and mesh data

Closed this issue · 1 comments

CHENYUHANG-smp commented

Thanks for your great work!

I have two questions:

  1. How you get the Camera Intrinsics of wild datasets (e.g. COCO)

  2. How you get the SMPL parameters of wild datasets, it seems that you just use h36m to supervise the Virtual Markers?

ShirleyMaxx commented

Hi, thank you for your interest, and sorry for the late reply.

  1. For the COCO dataset, we use the annotations provided by I2L-MeshNet. They use SMPLify-X to get the pseudo-GT label including the camera parameters, and SMPL parameters, as mentioned here. For other wild image inputs, we estimate the camera intrinsics according to the image size.

  2. During the model training process, we use the official SMPL GT parameters provided by the H36M dataset and 3DPW dataset to supervise 3D virtual marker estimation. For the MPI-INF-3DHP dataset, SMPL GT labels were not provided, so we only use its 3D pose labels. The COCO dataset and UP-3D dataset are both 2D datasets and their pseudo-SMPL GT parameters are obtained by SMPLify-X and SMPLify, respectively. Since we find that these fitted meshes are not accurate, we only use the 2D projections and supervise the first two dimensions of the virtual marker estimation. We've detailed these datasets in the supplementary.