OpenMOSS/AnyGPT

hi，when will the pre-train related codes&scripts be released?

Closed this issue · 4 comments

XL2248 commented

I see you have released the mmpretrain file. When will you plan to release all the code and scripts? I believe this will benefit the community a lot and gain much researchers' interest in your great work like MOSS. Besides, how many A100 GPUs were used in the pre-training stage, and how much time did it cost?

JunZhan2000 commented

In fact, you only need to use the code for training LLM. I will sort out the code and release it in the next two weeks.
The pre-training took 96 80G A100 for a week, and the instruction fine-tuning was much less.

  • 👍5
XL2248 commented

Hi, are the codes and scripts ready to release?

  • 👍3
JunZhan2000 commented

Hi, are the codes and scripts ready to release?

We have uploaded the relevant scripts for reference

JunZhan2000 commented

Hello, we provide some training data samples and related descriptions, please refer to https://github.com/OpenMOSS/AnyGPT?tab=readme-ov-file#pretraining-and-sft