making video captures (capture_video=True) and viewing the GUI over SSH work in Ubuntu 22.04

[Yasu31]

Description

I was not able to make video captures during the training process, nor view the GUI window over SSH (with X11 forwarding enabled).

I know Ubuntu 22.04 is not officially supported by IsaacGym so it may not be appropriate to post it here, but as I've encountered this issue & already resolved it myself I will post the solution here for future reference.

To Reproduce

system: Ubuntu 22.04

fails to make video captures

run a training with video capture enabled, e.g. python train.py capture_video=True force_render=False, I get the following error and the program terminates:

ValueError: not enough values to unpack (expected 3, got 0)

which occurs at the line h, w, pixfmt = frame_shape at gym/wrappers/monitoring/video_recorder.py (unfortunately I didn't save the full error, and now I can't reproduce the error even if I try to revert the state of the system.

cannot view the GUI over SSH

the client PC was also running 22.04

when I ssh into the machine with isaacgym with the -X option to enable X11 forwarding, when I run any of the programs that create a display (e.g. python train.py) a window appears but is completely black and is frozen.

If I disable the window from opening (e.g. python train.py headless=True) the program runs as usual.

Fixes

Unfortunately, I am not completely sure which one fixed the problem, but after running the following steps the issue was resolved.

1. install all required packages by PyVirtualDisplay
   sudo apt-get install xvfb xserver-xephyr tigervnc-standalone-server x11-utils gnumeric
2. enable X instead of Wayland for Ubuntu 22.04(refer to https://askubuntu.com/questions/1410256/how-do-i-use-x-instead-of-wayland-on-22-04 - edit /etc/gdm3/custom.conf)
3. reboot the system

[Yasu31]

closed as I resolved it myself

[Yasu31]

update: I was able to reproduce the error, pasting the full error message;

Traceback (most recent call last):
  File "train.py", line 156, in launch_rlg_hydra
    runner.run({
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/rl_games/torch_runner.py", line 120, in run
    self.run_train(args)
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/rl_games/torch_runner.py", line 101, in run_train
    agent.train()
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/rl_games/common/a2c_common.py", line 1162, in train
    self.obs = self.env_reset()
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/rl_games/common/a2c_common.py", line 470, in env_reset
    obs = self.vec_env.reset()
  File "/home/yasu/faive-isaac_gym_env/isaacgymenvs/utils/rlgames_utils.py", line 159, in reset
    return self.env.reset()
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/gym/wrappers/record_video.py", line 94, in reset
    self.start_video_recorder()
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/gym/wrappers/record_video.py", line 112, in start_video_recorder
    self.video_recorder.capture_frame()
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/gym/wrappers/monitoring/video_recorder.py", line 191, in capture_frame
    self._encode_image_frame(frame)
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/gym/wrappers/monitoring/video_recorder.py", line 250, in _encode_image_frame
    self.encoder = ImageEncoder(
  File "/home/yasu/venv/rlgpu/lib/python3.8/site-packages/gym/wrappers/monitoring/video_recorder.py", line 382, in __init__
    h, w, pixfmt = frame_shape
ValueError: not enough values to unpack (expected 3, got 0)

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

[MildCloud]

Hi, I am also trying to use isaacGym in ubuntu22.04.
I get these error when installing
>>> from isaacgym import gymutil Importing module 'gym_38' (/home/rob-arc-group/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/rob-arc-group/isaacgym/python/isaacgym/__init__.py", line 5, in <module> from isaacgym import gymapi File "/home/rob-arc-group/isaacgym/python/isaacgym/gymapi.py", line 104, in <module> _import_active_version() File "/home/rob-arc-group/isaacgym/python/isaacgym/gymapi.py", line 63, in _import_active_version module = importlib.import_module(package_path)
Can you provide a guidance about how to fix it?