No controllers work in the simulator

[Hatt3n]

Hello!

I am using Ubuntu 18.04 with Python 3.8.0, and managed to use the flip-controller with the QUBE 2 hardware succesfully. However, when I try to use it with the simulator (by running python tests/test.py --env QubeSwingupEnv --controller flip -s -r), it seems like the controller just immediately swings the arm all the way to the left or right, ending the episode. The same is seen when using other controllers such as "sw".

The simulator still seems to work on its own, though; I can train RL agents against it and get successful results. But there seems to be a deviation between the simulator and hardware due to the controllers not working.

Am I doing something wrong? Could it be that I am using incorrect versions of the required Python modules?

Thanks in advance!
Best Regards

[polzounov]

The fact that it swings all the way to the left or the right is due to the differences between the real system and the simulated physics. The simulation is a bit too 'perfect', and the real system has a few other forces acting on it that the simulation is missing (one example is forces from the cable that attaches the base to the pendulum arm).

The fact that the episode ends is something I need to fix, for now you can comment out lines 77 and 78 in tests/test.py and that will solve it for you.

[Hatt3n]

I see, thank you for the clarifications and your help! And thanks for developing this OpenAI driver, it has been extremely useful.

Best Regards

[polzounov]

Closing issue for now, feel free to reopen if there's something else