Could not obtain official perplexity using bloom_eval()
xingyueye opened this issue · 0 comments
xingyueye commented
Hi, I ran the bloom.py using fp16 to test the perplexity (PPL) of BLOOM on Wikitext-2, PTB, and C4 datasets. The results are 11.79 / 20.14 / 17.68, which is worse than the official results of 11.37/19.40/14.13.