Incorrect value for phi-1

Question

Incorrect value for phi-1

rokset3 opened this issue a year ago · 2 comments

phi-1 is finetuned version of phi-1-base on CodeExercises dataset, phi-1-base pretrained is 29% on HumanEval
https://arxiv.org/pdf/2306.11644.pdf

Could you please take a look on Table 4. from your paper https://arxiv.org/pdf/2311.07989.pdf.
In my humble opinion, phi-1 should be reported as an instruction-tuned model, and phi-1-base results should be reported separately

Answer 1 · 2024-01-31T08:47:05.000Z

@rokset3 You are right! Great thanks! We will correct this in our next update of paper.

Answer 2 · 2024-01-31T09:30:02.000Z

@ss41979310 I am wrong actually. After double checking the papers, Phi-1 has not been instruction tuned. It was finetuned on some more code data T_T.