Free run sharing
joaquinvanschoren opened this issue · 0 comments
Using OpenML tasks ensures reproducibility, but at the same time it can be limiting, since you can't share runs post-hoc, just after you discovered something nice, the way most experiments are.
Maybe we should allow uploading runs post-hoc, when models are trained even if they didn't start from an OpenML task.
So, say you have a sklearn or PyTorch model trained, allow uploading it by not linking it to an existing task (because you can't), but perhaps with an ad-hoc task described manually, or by extracting all the info we can from the ML environment.
It would give people a way to easily share experiments and models with each other with still 'pretty acceptable reproducibility'.
People can always filter those out if they don't trust them enough.
Thoughts?