Mismatch between number of images and corresponding mano files

[anjugopinath]

I want to use the mano files, but I noticed that there is a mismatch between number of files:

Ex) Within this particular folder, there are 733 images.

But, the number of mano files are only 732 (unpack/arctic_data/data/raw_seqs/s01/box_grab_01.mano.npy):

Since the mano file does not have information about which image each row maps to, instead of discarding the one image for which the mano parameters are missing, I would have to discard all the images in this folder.
I wanted to confirm this is the case or if there was an error when downloading the files.
Thank You!

[zc-alexfan]

There is no error. The mocap files (MANO groundtruth) are usually 1 or 2 frames off from the image files. This is because we need to synchronize the two systems.

The synchronization is specified here where vidx is the index for mocap and image_idx is the image name.

[anjugopinath]

Hi,

Thank You for your response. This answers my question!
I just wanted to confirm the logic with an example:

Say, the number of images are 5 and the number of rows in the corresponding mano file is 4 (0,1,2,3).
And say ioi_offset is 2.
And image indexes start with 1 (1.jpg).
So, for the first image, 1-2=-1 (the last row from the mano file will be picked).
And similarly, for the other images:
2-2=0 -> 0th row
3-2=1 -> 1st row
4-2=2 -> 2nd row
5-2=3 -> 3rd row

[zc-alexfan]

yes, exactly

[anjugopinath]

Thank You for the quick response!

[anjugopinath]

Hi,

For the image
unpack/arctic_data/data/images/s01/ketchup_grab_01/1/00655.jpg:

img_id is : 655 offset is : 1 and so index =655-1= 654
And size of pose parameters is
pose_all_l size is : (654, 45)

so it's trying to index into the 654th location which throws an error since the pose dictionary length is 654 (0...653) and
so the last row is at index 653.

Similarly, this error occurs several times with other images too.

For the folder:
unpack/arctic_data/data/images/s06/notebook_use_01/1
the length of rotation and pose dictionaries are 681:
len rot_l : 681
len pose_l : 681
but the number of images in the folder is 674:
len rgbImages : 674

Is this correct?
Usually, the number of images is one more than the number of rows in the corresponding pose and rotation
dictionaries.

[zc-alexfan]

Currently at cvpr so i have limited capacity. It only happens at a few frames at the end of the sequences. So it won't affect your training.

I will take a look after cvpr

[zc-alexfan]

You can check the image folder and see if the images missing annotation are at the final 1 or 2 frames. If that's the case, then it is fine.

We have two capture systems to synchronize: RGB system, MoCap system. They are off by 1 or 2 frames because we need to take into the account of the camera shutter for the two systems during the capture. So they won't have exactly the same number of frames and off by 1 or 2.

You can reopen the issue if it is not resolved.