Do more objects imply more or less accuracy?

[TommasoBendinelli]

Hello,
I have a doubt regarding how the number of objects affects the overall performance of the network,
Let say I create a network that is able to infer the pose of object A, and that I train also a network that is able to infer the pose of object A,B,C,D. With same data for class A, but of course with novel data for class B,C,D.

Will the network performance be different between the two networks? In other words, is the pose estimation of one object affected by data from other objects?

Best,
Tommaso Bendinelli

[j96w]

Hi, thanks for mention this good question. We empirically found that the first network will perform better on the pose estimation of object A compared to the second network. But it's also easier to become overfitting.

So if you have enough data for a single object and only want to track it, I would suggest using the first network. But we do want to point out this interesting combined training method, which shows us that even between objects will different geometry and texture, they still have some common feature and could be projected to one embedding space. This is basically the motivation of our second research (6-pack) on category-level object pose tracking.