handeye_server error tf2.ExtrapolationException: Lookup would require extrapolation into the future

Hi @marcoesposito1988,

I am running easy_handeye2 on ROS2 Humble with actual hardware. I have made the setup as suggested in the Readme launch files.
However, I see a tf2 extrapolation error as:

[handeye_server-12] tf2.ExtrapolationException: Lookup would require extrapolation into the future. Requested time 1691770862.861832 but the latest data is at time 1691770862.819078, when looking up transform from frame [dx400/base_link] to frame [dx400/ee_gripper_link]

The error narrows down to this function:

easy_handeye2/easy_handeye2/easy_handeye2/handeye_sampler.py

Line 87 in c788191

    
           def _get_transforms(self, time: Optional[rclpy.time.Time] = None) -> easy_handeye2_msgs.msg.Sample:

If I change the time argument to 0 sec in all the lookup_transform functions like below, It works fine. Is there a better fix of this error?

tracking = self.tfBuffer.lookup_transform(self.handeye_parameters.tracking_base_frame,
                                                  self.handeye_parameters.tracking_marker_frame, Time(seconds=0),
                                                  Duration(seconds=10))

Thanks

Hi @pinakjani,

thanks for the report! This is most likely due to [this line], which just hardcodes an offset. It was a quick hack to make it work, and it worked too well because I forgot to replace it with a proper implementation 🙈

The problem with using 0 for both lookup_transform()s is that it will use the last available data for each transform, and it may be substantially different, leading to a bad sampling. In theory this is negligible if the system is not moving, but subtle oscillations may reduce the accuracy of the calibration if the tracking device is extremely accurate.

The proper way would be to replace it with a wait_for_transform() acting on the whole chain (from robot_base to tracking_marker for eye_in_hand, from robot_effector to tracking_marker for eye_on_base) and to use the returned time to do both lookup transforms.

Would you feel like trying it out and creating a PR?

Hi @marcoesposito1988 😃 ,

Thanks for your reply. Yes sure I will give it a try as you suggested and also test it with hardware and put in a PR.

Hi @marcoesposito1988 ,
I added the #3 PR to fix this issue 🔧. Apologies for the delay in this PR.

I am having this problem as well! Glad to see there is some work here I can refer to :)

[handeye_server-2] [ERROR] [1733770289.002378773] [bam_BAMGPU.handeye_server]: Failed to get the tracking transform: Lookup would require extrapolation into the future. Requested time 1733770287.427184 but the latest data is at time 1733770287.243116, when looking up transform from frame [base_link] to frame [charuco_frame]
[handeye_server-2] [ERROR] [1733770292.395803729] [bam_BAMGPU.handeye_server]: Failed to get the tracking transform: Lookup would require extrapolation into the future. Requested time 1733770291.176237 but the latest data is at time 1733770291.043210, when looking up transform from frame [base_link] to frame [charuco_frame]