Accessing IMU and Camera Data with UTC Nanosecond Timestamps

Hello,
I would like to retrieve the IMU data and the left, right, and RGB camera images along with their capture timestamps in UTC nanoseconds, similar to how the MPS SLAM service sets the utc_timestamp_ns field in the closed_loop_trajectory.csv file. The timestamp provided by the get_time_ns(TimeDomain.DEVICE_TIME) method differs from utc_timestamp_ns. It seems to be generated by an internal clock (by the MCU, as I understand).

I also attempted to access the timestamps in this format using the VRS Command Line Tool with the extract-all command but was unsuccessful. I found that ntp_time_enabled in metadata.json (the output of the extract-all command) is set to false. Does this mean that the UTC timestamp was not recorded in the VRS file and only the time generated by the internal device clock is recorded for sensor data? Additionally, the value of capture_timestamp_in_processing_clock_domain_ns is -1.

However, I checked the recording time of the .vrs file, which is June 20 at 16:10 CEST, and compared it with the value of the utc_timestamp_ns in the first entity in closed_loop_trajectory.csv, which is 1718892641002929366, equal to June 20, 2024, at 14:10:41.002929 UTC, confirming the same time.

Could you please guide me on how to achieve UTC timestamps in nanoseconds for sensor data?

Thank you in advance.
Banafshe

We are taking a look to the issue and will come back to you.

Thanks a lot.

Hi,

The utc_timestamp and tracking_timestamp have a 1 to 1 mapping in closed trajectory json file. tracking_timestamp is equvalent to DEVICE_TIME. so you should be able to find the right correspondence.
Here is the information
https://facebookresearch.github.io/projectaria_tools/docs/data_formats/mps/slam/mps_trajectory

Sorry for the confusion.
Please let us know if you have more questions.

Hi,
Thank you for your response and the information provided.‍‍

Based on the details in the link you shared, the tracking_timestamp_us is in microseconds, whereas the output from the ‍‍get_time_ns(TimeDomain.DEVICE_TIME)‍ method is in nanoseconds (generated by an internal clock). Therefore, I attempted to convert the output of this function to microseconds.
I retrieved all images from the VRS file using the following code and saved the files with the device_timestamp and (device_timestamp / 1000) as the filenames:

provider = data_provider.create_vrs_data_provider(vrsfile)
options.set_subsample_rate(stream_id, 1)  # sample every data for camera
iterator = provider.deliver_queued_sensor_data(options)
    for sensor_data in iterator:
        label = provider.get_label_from_stream_id(sensor_data.stream_id())
        sensor_type = sensor_data.sensor_data_type()
        device_timestamp = sensor_data.get_time_ns(TimeDomain.DEVICE_TIME)
        micro_sec = int(device_timestamp / 1000)
        
        # Sensor data obtained by timestamp (nanoseconds) 
        time_domain = TimeDomain.DEVICE_TIME  # query data based on DEVICE_TIME
        time_query_option = TimeQueryOptions.CLOSEST # get data whose time [in TimeDomain] is CLOSEST to query time
        image_data = provider.get_image_data_by_time_ns(stream_id, device_timestamp, time_domain, time_query_option)

Since TimeDomain.DEVICE_TIME (the capture timestamp) does not have a one-to-one correspondence with utc_timestamp_ns or tracking_timestamp_us found in the closed_loop_trajectory.csv file, I tried using the tracking_timestamp_us in the closed_loop_trajectory.csv file and (device_timestamp / 1000) calculated by the code above to find correspondences, but I was unable to find a satisfactory number of matches with the threshold required for my application.

Explaining this matter without getting too complex is challenging. However, I wanted to thank you for your reply.

Best regards,
Banafshe

Hi,
We do have interface to find interpolated pose for the specific device_timestamp that you are requesting.

projectaria_tools/core/python/MpsPyBind.h

Line 601 in 7ca8aaf

&MpsDataProvider::getInterpolatedClosedLoopPose,

Hopefully this helps.
Best,

Awesome, thanks!