openml/OpenML

Free run sharing

joaquinvanschoren opened this issue · 0 comments

Using OpenML tasks ensures reproducibility, but at the same time it can be limiting, since you can't share runs post-hoc, just after you discovered something nice, the way most experiments are.

Maybe we should allow uploading runs post-hoc, when models are trained even if they didn't start from an OpenML task.

So, say you have a sklearn or PyTorch model trained, allow uploading it by not linking it to an existing task (because you can't), but perhaps with an ad-hoc task described manually, or by extracting all the info we can from the ML environment.

It would give people a way to easily share experiments and models with each other with still 'pretty acceptable reproducibility'.

People can always filter those out if they don't trust them enough.

Thoughts?