camera_flip parameter is not respected by transform to optical frame and others

[lreiher]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues.
• This issue is not a question, feature request, or anything other than a bug report directly related to this project.

Description

We have mounted our ZED2i cameras upside-down on our vehicle. Naturally, the ROS driver by default then also gives us upside-down images. Luckily, there is the camera_flip option in common.yaml that automatically flips the images. However, we believe that this messes up the transform tree of the camera. Please see the attached screenshot of Rviz.

You can see images from three different cameras, but let's focus on the one labeled front_center. As expected, the image is upright after setting camera_flip: true. One thing to notice is that the grayscale image (incorrectly labeled front_right) is not flipped, but that is not much of an issue for us.

The greater problems are related to the coordinate frames shown in the 3D view. All shown frame origins belong to the camera labeled front_center. The left frame is the optical center of the right lens. The middle frame is the camera center frame. The right frame is the camera center of the left lens.

Note how the right lens frame of the upside-down forward-facing camera is on the left side. While this corresponds to the right lens if the camera had been mounted upright, the notion of left/right lens should switch if camera_flip is set. I believe that this does happen for the image topics, since the images corresponding to the left lens definitely show more of the world to the left.

The last thing we noticed is that the optical frames are pointing left (x) and up (y). This would have been correct when mounting the camera upside-down but still setting camera_flip: False, but if the images are flipped, the optical frame should also flip with it.

The result with the current frames is that we cannot correctly calibrate between cameras and lidars.

TLDR:

1. Grayscale images are not flipped if camera_flip: true.
2. Notion of left/right camera lens is wrong in the assembly of the transform tree if camera_flip: true.
3. Optical frame is wrong if camera_flip: true.

Steps to Reproduce

1. Mount camera upside-down.
2. Set camera_flip: true.
3. Launch ROS driver.
4. Investigate camera images and transform frames.

Expected Result

All camera images and the transform tree is respecting the camera_flip parameter.

Actual Result

see Description

ZED Camera model

ZED2i

Environment

- ZED SDK v3.8
- ROS Noetic

Anything else?

No response

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment otherwise it will be automatically closed in 5 days