[Error] [carb.gym.plugin] cudaImportExternalMemory failed on rgbImage buffer with error 999
Opened this issue · 2 comments
Hello, when I run the distillation policy, this error was returned, while everything goes fine during base policy training. Does anyone know how to solve this? Thanks a lot!
I know that it might be the problem of ubuntu version, is there any solution not to downgrade system?QAQ
command: python train.py --exptid 111-11 --resume --resumeid 000-00 --delay --use_camera --no_wandb
Importing module 'gym_38' (/home/user/apps/IsaacGym_Preview_4_Package/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/user/apps/IsaacGym_Preview_4_Package/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.10.0+cu113
Device count 1
/home/user/apps/IsaacGym_Preview_4_Package/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/user/.cache/torch_extensions/py38_cu113 as PyTorch extensions root...
Emitting ninja build file /home/user/.cache/torch_extensions/py38_cu113/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
Setting seed: 1
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Start creating ground...
Converting heightmap to trimesh...
iteration 0 - triangles 3414402 threshold 2.187e-06
iteration 5 - triangles 1776036 threshold 0.00209715
iteration 10 - triangles 458906 threshold 0.0627485
simplified mesh in 2.6172 seconds from 3414402 to 170720 triangles
Created 88159 vertices
Created 170720 triangles
Adding trimesh to simulation...
Trimesh added
Finished creating ground. Time taken 4.11 s
Creating env...
0%| | 0/192 [00:00<?, ?it/s][Error] [carb.gym.plugin] cudaImportExternalMemory failed on rgbImage buffer with error 999
Segmentation fault (core dumped)
I fixed it with downgrade my ubuntu22.04 to 20.04. Seems the only way to fix it.
I fixed it with downgrade my ubuntu22.04 to 20.04. Seems the only way to fix it.
add this in your code or export the environment
os.environ['VK_ICD_FILENAMES'] ='/usr/share/vulkan/icd.d/nvidia_icd.json'
it works for me ubuntu22.04 conda python3.8