pixel obs
haoyu-x opened this issue · 5 comments
Hi @keiohta , thanks for sharing such a great project!
I'm wondering if I want to set observation to be image-based using tf2rl, do you have any advice?
I'm not sure what you want to do but I guess Atari example with DQN might help you.
https://github.com/keiohta/tf2rl/blob/master/examples/run_dqn_atari.py
hi @keiohta, sorry for the late reply.
basically, I'd like to use GAIfO+SAC on image-based dm_control and robosuite tasks.
What I'm trying to say is that does tf2rl have any support of image-based RL/IL. or let's say can I pull a request, for example, SAC+AE working on the image-based dm_conttrol suite. https://sites.google.com/view/sac-ae/home. also something like CURL. https://arxiv.org/abs/2004.04136
example of IL, the experiment of the original GAIfO paper is also based on image-based tasks, it uses CNN policy and CNN discriminator. also, VAIL explored video imitation in their experiments.
if there could be a CNN-GAIfO or VAE-GAIfO-SAC. that would be great!
thanks!
Haoyu
Hi @haoyu-x , sorry I completely forgot to answer your question...
Current tf2rl does not officially support image-based RL/IL (I mean I don't provide no example of such scripts under examples), but we've already implemented the codes to do that.
For example, I think you can solve an easy image-based RL (let's say Pendulum-v0) using SAC + VAE, or do IRL in similar way.
I strongly suggest you use VAE instead of directly inputting image to your policy. Refs are:
- Improving Sample Efficiency in Model-Free Reinforcement Learning from Images
- Learning to drive in a day
Of course it would be awesome if you can send the PR that supports CURL or other image-based RL/IL!
I close this issue since it seems to be resolved.
Please reopen this issue if you find other problem related to this.