Videos are Empty and Broken

[AliEmreNarin]

I'm trying to capture videos during training. My code's runs successfully and i get everything correctly, except for the video. Instead of a video, i get this json:

{"step_id": 5000, "episode_id": 13, "content_type": "video/mp4", "empty": true, "broken": true}

/home/x/anaconda3/envs/eureka/lib/python3.8/site-packages/gym/wrappers/monitoring/video_recorder.py:165: UserWarning: �[33mWARN: Env returned None on render(). Disabling further rendering for video recorder by marking as disabled: path=/home/x/Eureka/eureka/outputs/eureka/2023-12-12_17-55-06/policy-2023-12-12_17-56-32/videos/CartpoleGPT_2023-12-12_17-56-33/rl-video-step-10000.mp4 metadata_path=/home/x/Eureka/eureka/outputs/eureka/2023-12-12_17-55-06/policy-2023-12-12_17-56-32/videos/CartpoleGPT_2023-12-12_17-56-33/rl-video-step-10000.meta.json�[0m
logger.warn(

I'm not sure why this is happening and need help.

[120L021304]

maybe you can try running your code with headless=False, if code runs in headless mode, render() will return None.

[dmtrung14]

I am having the same problem, though I wanted to save the data from play.py from a remote machine. So I added the following lines to play.py:

if SAVE_VIDEO:
            env = gym.wrappers.RecordVideo(
                        env,
                        os.path.join(LEGGED_GYM_ROOT_DIR, 'logs', train_cfg.runner.experiment_name, 'exported', 'videos'),
                        step_trigger=lambda step: step % 10000 == 0, # record the videos every 10000 steps
                        # episode_trigger=lambda ep: ep % 10 == 0, # record the videos every 10 episodes
                        video_length=100,  # for each video record up to 100 steps
                    )

and only get one rl-video-step0.meta.json file with the following content.

{"step_id": 0, "episode_id": 0, "content_type": "video/mp4", "empty": true}

Then rendering works fine on the remote machine, so I don't think it is my code or the play.py file that causes the error. Any tips to fix this?

[paLeziart]

@dmtrung14 Do you have a line like that? UserWarning: WARN: Tried to pass invalid video frame, marking as broken: Your frame has shape (723, 1280, 3), but the VideoRecorder is configured for shape (720, 1280, 3). I had to do an ugly manual edit of my /site-packages/gym/wrappers/monitoring/video_recorder.py because for some reason the captured frame does not have the right amount of pixels, so I added a

if frame.shape[0] != 720:
                frame = frame[:-3, :, :]

Also, it seems recording can mess up if the frequency is too close from the length of the recording, capture_video_freq=1100 capture_video_len=1000 produced no video, but capture_video_freq=1100 capture_video_len=250 worked well.

Finally, you can also check if you have a warning about Vulkan in the output of the terminal. Something like:

vulkan: No DRI3 support detected - required for presentation
Note: you can probably enable DRI3 in your Xorg config

If you have that, check #126 for the solution I just found 👍

[dmtrung14]

@paLeziart Thanks for the kind response. I have none of the symptoms that you had (haven't tried making frequency different from video length though), but I don't see why that would fix my problem.

Additionally, I tried to run the code in the README of this repo inside ./isaacgymenvs and it doesn't work either, so I have no clue what's going wrong.