Result on Franka dataset

[Giuse1]

Dear authors,
first of all, thank you for the amazing work.
I am trying to use the model on the Frianka16, but it's not clear to me how to do it.
First, in the dataset it seems that the annotated elevations are not correct, as I see images with elevation 0° having actual elevation greater than 0°.
Second, I would like to generate new views on a circular trajectory around the object with a fixed elevation.
I changed the code about the output poses using the same lines of code relative to the input views. However, the elevation of the produced images seems wrong to me.
This happens with the 6DoF model, while with 4DoF it does not work at all.
Could you please give me some suggestions on how to do correctly give the information about the angles?

Best regards,
Giuseppe

[kxhit]

Hi @Giuse1
Thanks for testing our work!
Yes the Franka dataset is collected by @Marwan99 and the poses are obtained via inverse kinematics. Please check these lines and see how we convert the poses before feeding into EscherNet.

I think both 4DoF and 6DoF models should work if the poses look correct. You can visualise the camera poses for verification.

In fact, for the 6DoF model, as long as your poses are in the same coordinates, there is no need to obtain the canonical pose, you could just input the raw poses and query the target poses you want with respect to the given coordinates.