google-research/FLAN

niv2 few-shots are returning single shot examples

TheExGenesis opened this issue · 2 comments

@lehougoogle I applied the latest changes and running

task = seqio.get_mixture_or_task(f"tfds_natural_instructions_template_0to10_x_shot")
dataset = task.source.get_dataset("train")
next(dataset.as_numpy_iterator())

is outputting

{'definition': b'In this task, given a sentence in the English language, your task is to convert it into the Filipino language.',
 'id': b'task559-1221f105f0cf40bb97291692f5eda329',
 'input': b'"Out of concern for the interpreters and their families\' security as well as the security of the Danish base in Iraq, the Defence Ministry has chosen to inform the public after the interpreters and others had left Iraq," the Danish Defence Ministry said in a statement.',
 'output': b'"Para sa kapakanan ng mga ng mga tagapagsalin at seguridad ng kanilang mga pamilya pati narin ang seguridad ng mga taga denmark na nakabase sa Iraq, Pinili ng Ministro ng Depensa na ipaalam sa publiko matapos makaalis sa Iraq ang mga tagapagsalin at iba pa," nasabi ng Danish Defence Ministry sa isang pahayag.',
 'source': b'asian_language_treebank',
 'task_name': b'task559_alt_translation_en_fi'}

@TheExGenesis I ran this in the run_example.py and am seeing few shot examples.

selected_mixture = seqio.get_mixture_or_task('niv2_fsopt')

I tried again and it worked, sorry.