asappresearch/flambe

Out-of-the-box experiments with standard datasets

Opened this issue · 1 comments

Is your feature request related to a problem? Please describe.
For most research, new models/algorithms/etc are tested on specific "standard" datasets.

Describe the solution you'd like
flambe-integrated datasets that could be used by, e.g., having the following config.yaml entry: dataset: !flambe.MultiNLI

Additionally, it would be great to even have this functionality over entire suits of datasets, such as: dataset: !flambe.GLUE_ExperimentSuite

We have some of those already for text classification and language modeling (see flambe.nlp). But I agree we should keep expanding!