关于使用指令中不同词性的特征
Closed this issue · 2 comments
xiongfy commented
您好，想请问一下在预训练reverie时有使用到词性划分的特征吗，我在pretrain_src的代码中好像没有找到和PickSpecificWords()相关的部分
CrystalSixone commented
Hi! As mentioned in the paper, we didn't use specific action-level and object-level words during pre-training since we found that they could result in leakage for masked language modeling (MLM). They are added in the fine-tuning stage.
xiongfy commented
好的好的，感谢回复