Does LLaMA-2-7B-32K just continue pretrain after LLaMA-2-7B?
Opened this issue · 0 comments
DavideHe commented
from togethercomputer/LLaMA-2-7B-32K,I found the mode just chage the rope to PI .
Does LLaMA-2-7B-32K just continue pretrain after LLaMA-2-7B?