google-research/FLAN

T0 FS OPT - unable to process.

AadSah opened this issue · 2 comments

AadSah commented

Hi @shayne-longpre, I have been using @SirNeural's script here to generate the data. I have been able to generate all the files, except t0_fs_opt data. Every time I run the script, the process gets killed (with a prompt "Killed", and no error message) after some time. Also, the same data is unavailable in the data processed by @SirNeural in the huggingface repo. Did you face such issues earlier? Any help? :)

Thanks!

shayne-longpre commented

@AadSah Hmmm I'm not sure why that would be, other than it is the biggest submixture and longest to generate. One way to make it work is to process 1/4 of the datasets at a time. For instance, you could manually subset this list (once for each quarter), generate each quarter of the submixture, then combine them manually at the end.

shayne-longpre commented

You can also now manually download the T0 submixture (and the others) -- see the new README! :)

  • 👍1