Train, test split on OpenML data

[pplonski]

I would like to run my AutoML package on the same train/test split of OpenML datasets. Could you please give some instructions on how to obtain exact split for train/test?

[Innixma]

Hi @pplonski,

The easiest way to run the OpenML datasets would be to use automlbenchmark as done in our paper: https://github.com/openml/automlbenchmark

By running through automlbenchmark, it will automatically get the same train/test splits as in our paper, as they are predefined.

All you need to do is create a framework wrapper for your AutoML package in automlbenchmark, and it should work for you.

Here is the issue open regarding adding AutoGluon officially to automlbenchmark, which you could consider going a similar path: openml/automlbenchmark#93

Hope this helps!

[pplonski]

Thank you Nick!