sim_to_real problem

[jheyeo222]

Hello, I am a student. I have trained a robot within the Isaac Gym simulator using the Isaac Gym's RL game library. Now, I am looking to apply this trained model to a real robot.

For this, I need to replicate the neural network architecture used in Isaac Gym's RL game library on the robot and load the .pth file.

However, implementing the neural network architecture exactly as it is in Isaac Gym's RL game library seems overly complex. Is there a way to accurately reproduce the same neural network architecture from Isaac Gym's RL game library for real robot implementation?
Alternatively, are there other methods to effectively apply this trained model to a real robot?

[locoxsoco]

Have you got any updates? I found this thread useful refering to DeXtreme research paper

[jheyeo222]

Have you got any updates? I found this thread useful refering to DeXtreme research paper

"Thank you, I hope everything goes well for you."

[AIWintermuteAI]

rl_games thing IS overly complex, but looking at the forward method in rl_games/algos_torch/models.py provides some insight of what is going on there - i.e. how the inputs are passed into NN and what is output and how it is post-processed.
I'm still working it out, but it looks like for the example I'm running ModelA2CContinuousLogStd is used: the action values are sampled from a normal distribution with mean and standard obtained from NN, like this

            mu, logstd, value, states = self.a2c_network(input_dict)
            distr = torch.distributions.Normal(mu, sigma, validate_args=False)
            selected_action = distr.sample()

These values are then possibly re-scaled (that is according to your sim code) and then applied to a robot in the sim.
@gavrielstate do you have any feedback on this? Perhaps there is some guide or example? So far I manage to save the trained model as .onnx (to run inference in a more portable way) and now looking into post-processing the results.

[AIWintermuteAI]

Okay, more breadcrumbs for anyone who stumbles upon this issue
https://github.com/Denys88/rl_games#quickstart-colab-in-the-cloud
It is pretty much what is needed - export model as .onnx, get the output, feed it into other env (or a robot). The only problem is that I likely don't have or don't want to use numpy (will on the model on MCU)...

[ynarang]

Hi all,

We released a repo recently for deploying a policy trained in Isaac Gym onto a Franka robot. It might provide another useful point of reference: https://github.com/NVLabs/industreallib

Best,
Yash

[jheyeo222]

Hi all,

We released a repo recently for deploying a policy trained in Isaac Gym onto a Franka robot. It might provide another useful point of reference: https://github.com/NVLabs/industreallib

Best, Yash

Thank you for your assistance. I wish you well in all your endeavors.

[eferreirafilho]

Any new findings on this? @jheyeo222 @locoxsoco
Many thanks!