ValueError: For decoder-only generation, one must pass `input_ids

[zhouliang-yu]

Hi! Thx for sharing the amazing code repo.

First of all I've tried the trial on gpt2 finetuning with textbox, eveything is good, works pretty smooth.
However, I came across an error when I tried to use prompt tuning for gpt-2.
I set the efficient training hyper-parameters at overall.yaml to be

efficient_methods: ['prompt-tuning']
efficient_kwargs: {'prompt_length': 100}
efficient_unfreeze_model: False

Then when prompt-tuning it returns the following error:

generating:   0%|          | 0/7 [00:00<?, ?it/s]
generating:   0%|          | 0/7 [00:00<?, ?it/s]
24 Nov 21:13    ERROR Traceback (most recent call last):
  File "/weka-jd/prod/public/permanent/group_yangyaodong/yuzhouliang/workspaces/teeter/textbox/utils/dashboard.py", line 323, in new_experiment
    yield True
  File "/weka-jd/prod/public/permanent/group_yangyaodong/yuzhouliang/workspaces/teeter/textbox/quick_start/experiment.py", line 130, in run
    self._do_train_and_valid()
  File "/weka-jd/prod/public/permanent/group_yangyaodong/yuzhouliang/workspaces/teeter/textbox/quick_start/experiment.py", line 105, in _do_train_and_valid
    self.valid_result = self.trainer.fit(train_data, valid_data)
  File "/weka-jd/prod/public/permanent/group_yangyaodong/yuzhouliang/workspaces/teeter/textbox/trainer/trainer.py", line 453, in fit
    self.stopped |= self._valid(valid_data, 'epoch')
  File "/opt/hf_venvs/python3.8/202111/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/weka-jd/prod/public/permanent/group_yangyaodong/yuzhouliang/workspaces/teeter/textbox/trainer/trainer.py", line 297, in _valid
    valid_results = self.evaluate(valid_data, is_valid=True)
  File "/opt/hf_venvs/python3.8/202111/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/weka-jd/prod/public/permanent/group_yangyaodong/yuzhouliang/workspaces/teeter/textbox/trainer/trainer.py", line 526, in evaluate
    generated = self.accelerator.unwrap_model(self.model).generate(batch_data, self.accelerator)
  File "/weka-jd/prod/public/permanent/group_yangyaodong/yuzhouliang/workspaces/teeter/textbox/model/abstract_model.py", line 90, in generate
    sample_outputs = accelerator.unwrap_model(self.model).generate(**inputs, **self.generation_kwargs)
  File "/opt/hf_venvs/python3.8/202111/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "/opt/hf_venvs/python3.8/202111/lib/python3.8/site-packages/transformers/generation_utils.py", line 934, in generate
    raise ValueError("For decoder-only generation, one must pass `input_ids`.")
ValueError: For decoder-only generation, one must pass `input_ids`

mainly ValueError: For decoder-only generation, one must pass input_ids`` at transformers/generation_utils.py when validating

How can I solve this problem?

[StevenTang1998]

Sorry, we have not yet supported GPT-2 for prompt tuning. We have been working to solve it.

Since it is not supported to pass prefix embeds for decoder-only models.

[zhouliang-yu]

ok I got it