[Feature Request] Implement TQC for the example algorithms
maxweissenbacher opened this issue · 6 comments
Motivation
I suggest adding an implementation of TQC to the examples. I suggest adding this as a request in the 'call for distributions' stack. I would be happy to take on the implementation.
Solution
Add a performant, clear and minimal implementation of TQC to the examples. I would base the implementation on the already existing implementation of SAC in the examples. This seems reasonable due to the overall similar structure of the algorithms. In the Stable-Baselines3 library implementation of TQC, the implementation is likewise based on their corresponding implementation of SAC.
Checklist
- I have checked that there is no similar issue in the repo (required)
- Add a request to the call for contributions.
- Adapt the implementation of SAC from the examples to TQC
- Benchmark?
Hi @maxweissenbacher
Adding TQC would be fantastic and greatly appreciated!
Any help we can provide, just let us know. I'm adding this to the call for contributions straight away!
Hi @vmoens, that's great, thank you! I'll get started right away and come back here in case I need assistance (more likely than not).
I cannot seem to get the logger to work for me. By running the (unchanged) training script for the SAC example, trying to execute theget_logger function gives me the following error:
wandb: WARNING Path sac_logging/wandb/ wasn't writable, using system temp directory.
Any suggestions?
Another (very silly) question... this is my first open source contrib - how do I contribute the code to the github? Do I need to fork the repository first, and submit a pull request?
For the logger it seems you lack write access in the folder where you're executing the code.
For the OSS contribution:
- make a fork of torchrl on your username
- write code in your separate branch locally on your machine
- when you're at a stage where you feel the code is mature and/or you want to show it to us to get feeeback, commit the changes on that branch and push it to your fork. Then create a PR from that branch towards main on the PyTorch org.
You will need to format the code etc for which I invite you to read the CONTRIBUTE.md file in torchrl.
Thanks again for your help!
Thank you for all this info. I have submitted a pull request now, hopefully I did it right!
I couldn't get the logging to work, so I implemented a custom logger to use on my local machine instead - however the logging is just as in the other example files in the pull request, so it should hopefully work as expected for you.