Assistance with Loading and Executing real_data.zip.

[nisshimura]

Hello,

First of all, thank you for your fantastic work. I was able to successfully run demo.ipynb. However, I'm having trouble understanding how to load and execute it using the real_data.zip file that you have provided. Could you please guide me on how to proceed? (e.g. which file/command should I use?) I appreciate your work and thank you in advance for your help.

Best regards,

[lzylucy]

Thanks for your interest in our work!

Each folder in real_data.zip contains recording of 1 episode for 1 real world task. replay_buffer.zarr contains information about the robot actions and subgoal labels. You can read the file using this command:meta_data = zarr.open(file_path, 'r'). To access the robot end effector poses for example, you can do robot_eef_pose = meta_data['data/robot_eef_pose']. You may find this thread helpful too #3.

videos/0/0/color contains all the RGB images and videos/0/0/depth contains all the depth images recorded for the episode. You can read them using the following commands:

from imagecodecs import imread

rgb = imread(f'appleInFridge1/videos/0/0/color/{step_idx}.0.0.0')
depth = imread(f'appleInFridge1/videos/0/0/depth/{step_idx}.0.0')

You might need to do pip install imagecodecs. The RGB images and depth images are recorded 30Hz (30 frames per second).

Finally, audio.wav is the audio track recorded. Please let me know if you have more questions!

Best,
Zeyi