Moving benchmarks to avalanche
AntonioCarta opened this issue · 6 comments
I think Avalanche is ready to integrate the CRL benchmarks.
@NickLucche do you agree with moving the benchmarks in avalanche? Once they are in the main repository, they would be considered "stable" and possibly receive more attention. There are a lot of people using Avalanche just for the benchmarks that may not know about avalanche-rl.
Good job with new API, it looks like a much simpler abstraction over the core concept of streams, I'll try to create a PR in avalanche asap integrating the work I did on CRL benchmarks.
Done here ContinualAI/avalanche#984
Shouldn't we remove them now from Avalanche-RL?
I agree, we shouldn't duplicate features. avalanche-rl should use avalanche benchmarks now.
Yep, in order to do so we need to add this #8 tho, currently we can't just import the latest avalanche version due to the major changes in its structure since I last backported it
Re-opening with a possible solution implemented here https://github.com/ContinualAI/avalanche-rl/tree/new-avalanche-version.
It involves a workaround to keep the old avalanche BaseStrategy around before fully switching to the Templates model.
It uses the newly implemented benchmarks in avalanche (up to date with the master branch) and BasePlugin/BaseLogger.