med-air/DEX

How to display the robot training GUI in PyBullet

Closed this issue · 8 comments

arduino183454 commented

Hello, I reproduced your code, but I can't see the robot during training. If I want to display the robot training process( PyBullet GUI ) , How should I modify this code? Thank you very much.

arduino183454 commented 
    I try to modifie  this code, but an error was reported. Looking forward to your help, thank you very much.
    self.train_env = gym.make(self.cfg.task, render_mode='human')
    self.eval_env = gym.make(self.cfg.task, render_mode='human')
TaoHuang13 commented

Hello. The visualization of training has been integrated into WandB. You can turn on WandB as instructed in README to see that.

It seems your code is correct, may I know more details about the error? (I guess you should only render train_env with 'human', because render 'human' with 2 envs may result in errors)

  • 👍1
  • 🚀1
arduino183454 commented

Thank you very much for your answer，the following is the error when running.
Error executing job with overrides: ['seed=3', 'task_name=NeedlePick-v0']
Traceback (most recent call last):
File "train.py", line 174, in main
exp = Experiment(cfg)
File "train.py", line 22, in init
self.setup(cfg)
File "train.py", line 42, in setup
self.eval_env = gym.make(self.cfg.task_name, render_mode='human')
File "/home/pan/anaconda3/envs/herdemo/lib/python3.8/site-packages/gym/envs/registration.py", line 597, in make
env = cls(**_kwargs)
File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/tasks/psm_env.py", line 61, in init
super(PsmEnv, self).init(render_mode)
File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/gym/surrol_env.py", line 39, in init
self.cid = p.connect(p.GUI)
pybullet.error: Only one local in-process GUI/GUI_SERVER connection allowed. Use DIRECT connection mode or start a separate GUI physics server (ExampleBrowser, App_SharedMemoryPhysics_GUI, App_SharedMemoryPhysics_VR) and connect over SHARED_MEMORY, UDP or TCP instead.

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
Exception ignored in: <function SurRoLEnv.del at 0x7f01d80ec040>
Traceback (most recent call last):
File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/gym/surrol_env.py", line 238, in del
self.close()
File "/home/pan/强化学习应用/hindsight-experience-replay-with-demo/surrol/gym/surrol_env.py", line 142, in close
if self.cid >= 0:
AttributeError: 'NeedlePick' object has no attribute 'cid'
numActiveThreads = 0
stopping threads
Thread with taskId 0 exiting
Thread TERMINATED
destroy semaphore
semaphore destroyed
destroy main semaphore
main semaphore destroyed
finished
numActiveThreads = 0
btShutDownExampleBrowser stopping threads
Thread with taskId 0 exiting
Thread TERMINATED
destroy semaphore
semaphore destroyed
destroy main semaphore
main semaphore destroyed

  • 🎉1
  • 🚀1
arduino183454 commented

During the training process, I cannot see the robot. How should I modify the code? Thank you very much. In this code "https://github.com/TaoHuang13/hindsight-experience-replay-with-demo" I encountered the same problem when run code "sh scripts/run_herdemo.sh " . Thank you very much.

  • 👍1
  • 🚀1
arduino183454 commented

Hello, when I run "python3 train.py task=NeedlePick-v0 agent=dex use_wb=True" I can see the robot in the software use_wb.

If I don't use Wb，run "python3 train.py task=NeedlePick-v0 agent =dex use_wb=False”, In this case how to see robots in Pybullet. How to modify this code? Thank you very much。

  • ❤️1
  • 🚀1
TaoHuang13 commented

Hi. Could you try self.train_env = gym.make(self.cfg.task, render_mode='human'); self.eval_env = gym.make(self.cfg.task, render_mode='rgb_array')?

  • 👍1
  • ❤️1
arduino183454 commented

I try self.train_env = gym.make(self.cfg.task, render_mode='human'); self.eval_env = gym.make(self.cfg.task, render_mode='rgb_array'), but it doesn't seem to work, the error is reported as follows. Thanks a lot.

(dex) pan@pan-Lenovo-Legion-Y9000K2021H:~/强化学习应用/hindsight-experience-replh-demo/DEX-main$ python3 train.py task=NeedlePick-v0 agent=sac use_wb=Fals
e
pybullet build time: Nov 28 2023 23:51:11
startThreads creating 1 threads.
starting thread 0
started thread 0
argc=2
argv[0] = --unused
argv[1] = --start_demo_name=Physics Server
ExampleBrowserThreadFunc started
X11 functions dynamically loaded using dlopen/dlsym OK!
X11 functions dynamically loaded using dlopen/dlsym OK!
Creating context
Created GL 3.3 context
Direct GLX rendering context obtained
Making context current
GL_VENDOR=NVIDIA Corporation
GL_RENDERER=NVIDIA GeForce RTX 3070 Laptop GPU/PCIe/SSE2
GL_VERSION=3.3.0 NVIDIA 530.41.03
GL_SHADING_LANGUAGE_VERSION=3.30 NVIDIA via Cg compiler
pthread_getconcurrency()=0
Version = 3.3.0 NVIDIA 530.41.03
Vendor = NVIDIA Corporation
Renderer = NVIDIA GeForce RTX 3070 Laptop GPU/PCIe/SSE2
b3Printf: Selected demo: Physics Server
startThreads creating 1 threads.
starting thread 0
started thread 0
MotionThreadFunc thread started
ven = NVIDIA Corporation
ven = NVIDIA Corporation
/home/pan/anaconda3/envs/dex/lib/python3.8/site-packages/gym/logger.py:30: UserWarning: WARN: Box bound precision lowered by casting to float32
warnings.warn(colorize('%s: %s'%('WARN', msg % args), 'yellow'))
EGL device choice: -1 of 3.
Loaded EGL 1.5 after reload.
Failed to make context current (eglError: 12288)

  • 👍1
  • 🚀1
arduino183454 commented

Thank you very much for your patient reply. I will take a closer look at the code. Thank you for your help.

  • 👍2
  • 🚀1