Cannot run stackllama example

[fecet]

There is no more _get_submoudles in peft per huggingface/peft#279, so https://github.com/lvwerra/trl/blob/main/examples/stack_llama/scripts/merge_peft_adapter.py cannot use directly.

Also are we expected to use this script to merge both reward model and sft model? At first glance, we should change forcausallm to forsequenceclassfication for reward model, correct?

[vvasily]

Use this:

parent, target, target_name = _get_submodules(model.base_model.model, key)

[vpegasus]

Use this:

parent, target, target_name = _get_submodules(model.base_model.model, key)
make it complete:
at the beginning of the script, add:

from peft.utils import _get_submodules

then
change the line (line 56):

parent, target, target_name = model.base_model._get_submodules(key)

to

parent, target, target_name = _get_submodules(model.base_model.model, key)