Farama-Foundation/Minari

[Proposal] Porting Antmazes to Minari.

RajGhugare19 opened this issue · 3 comments

RajGhugare19 commented

Proposal

I think it would be a great idea to port Antmaze data originally proposed in D4RL, to Minari.

Motivation

Many offline RL papers use antmaze. Many papers will even skip pointmaze and only report numbers on antmaze because the latter is more difficult. I think adding this dataset and environment with Minari's clean and easier to use code will boost Minari's usage amongst offline RL researchers.

Pitch

  1. Use the data collecting policy from D4RL to re-collect data in Minari. I don't think the policy can be used directly, because the ant model for the original D4RL (mujoco-py antmaze) was different. I found that you could just divide the control output of the D4RL data collecting policy by some constant factor (like 5 / 10), it seems to work on gymansium antmaze.
  2. If point (1)' fix is not appropriate, we would need to train a new data collecting policy for the antmaze and use it to collect new data.

Checklist

  • [ YES ] I have checked that there is no similar issue in the repo (required)
skylooop commented

Upvotting your proposal. Minari list remote shows that there are several ant maze datasets. However, seems like they are not the same as in d4rl, or I'm missing something?

  • 👍1
rodrigodelazcano commented

Hey @RajGhugare19, as @skylooop mentioned we have uploaded datasets for AntMaze but it won't show up in the docs until we merge this PR: #155

Our approach for generating the datasets has been number 2. We decided to train our own goal reaching policy for the ant since we found that the d4rl policy didn't perform very well in our environment. Also, when replaying the d4rl datasets the ant's behavior is not very reliable as for what it is stated in the paper. I haven't tried dividing the control output though...

Any way, our new datasets achieve a success rate greater than 85% for all of the trajectories in every ant dataset. We've tried to resemble the data collection process as much as possible to the original in d4rl. You can have a look at the scripts here: https://github.com/rodrigodelazcano/d4rl-minari-dataset-generation/blob/main/scripts/antmaze/create_antmaze_dataset.py

We still have to add more documentation to the scripts for reproducibility. In the meantime let us know if you have anymore questions, I'll be happy to answer them :)

  • 👍1
RajGhugare19 commented

Thanks for your reply!