`openrlbenchmark` integration

[vwxyzjn]

Hi all, this is very cool stuff. I especially like that there is an MBPO implementation.

Would you be interested in using wandb to contribute experiment runs to openrlbenchmark utilities? It provides more plots and analysis options and could make comparing with other libraries much easier. See openrlbenchmark/openrlbenchmark#22 for an example.

To use openrlbenchmark, all you need to do is to track metrics in wandb with the x-axis being global_step. If you use tensorboard, you just need to turn on the wandb tensorboard integration (example). Then you can use openrlbenchmark to pull runs from wandb and generate plots like in openrlbenchmark/openrlbenchmark#22