Issue with 3D Skeleton Visualization: Unnatural Representation

[ShunyuYao]

Hello, thanks for sharing your excellent work and dataset.

As I've been working on visualizing 3D skeletons using the provided dataset, I've encountered some issues. Specifically, the plotted skeletons look unnatural and seems not on the same plane. And the root orientation changes quite abruptly and it appears that the camera of the scene is not fixed. Here is the plotted picture and some of my code, I use SMPL model to visualize the skeletion:

Pictures:

Codes:

for person_name in ['person1', 'person2']:
    global_orient = motions_data[person_name]['root_orient']
    body_pose = motions_data[person_name]['pose_body']
    betas = motions_data[person_name]['betas']
    transl = motions_data[person_name]['trans']

    device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
    global_orient_tensor = torch.from_numpy(global_orient).float().to(device)
    body_pose_tensor = torch.from_numpy(body_pose).float().to(device)
    betas_tensor = torch.from_numpy(betas).float().to(device)
    transl_tensor = torch.from_numpy(transl).float().to(device)

    rotation_matrix = euler_angles_to_matrix(
        body_pose_tensor.view(body_pose_tensor.shape[0], -1, 3), 'XYZ')
    global_orient_mat = euler_angles_to_matrix(global_orient_tensor, 'XYZ').view(global_orient_tensor.shape[0], -1, 3, 3)
    dtype = body_pose_tensor.dtype

    add_rotation_matrix = torch.eye(3, device=device, dtype=dtype).view(
                    1, 1, 3, 3).expand(rotation_matrix.shape[0], 2, -1, -1)
    processd_rot_mat = torch.cat([rotation_matrix, add_rotation_matrix], dim=1)
    betas_tensor_input = betas_tensor.expand(global_orient_mat.shape[0], -1)

    smpl_model = SMPL().eval().to(device)
    out = smpl_model(
            global_orient=global_orient_mat,
            body_pose=processd_rot_mat,
            betas=betas_tensor_input,
            transl=transl_tensor)
    show_idx = 0
    
    if person_name == 'person1':
        plot_skeleton_onePic(
            out['smpl'][show_idx].cpu().detach().numpy(),
            ax,
            'r',
            'Skeleton 1',
            humanact12_kinematic_chain
                            )
    else:
        plot_skeleton_onePic(
            out['smpl'][show_idx].cpu().detach().numpy(),
            ax,
            'b',
            'Skeleton 2',
            humanact12_kinematic_chain
                            )

[ShunyuYao]

Could you give me some suggestions? Thanks a lot.

[tr3e]

Try to use axis-angle representation for all rotations, and the pose parameters are standard SMPL parameters as same as AMASS.

[ShunyuYao]

Thanks! Everything works fine after using the SMPL-H model same to AMASS