Phi-3 model performance on MeteorLake GPU

Question

Phi-3 model performance on MeteorLake GPU

bopeng1234 opened this issue a month ago · 2 comments

Hi,

we are running benchmark on 3b models, using all-in-one script, models are phi3-4k, phi3-128k and starcoder2-3b.

Environments:
ThinkBookX2024
Ultra9-185H
Windows11pro-23H2
32GB LPDDR5x8400MHz
Arc Driver: 31.0.101.5445
ipex-llm[xpu] 2.1.0b20240506

Questions here:
have we done the performance comparison for Phi 3 4k/128k and starcoder2-3b

Answer 1 · 2024-05-09T01:24:50.000Z

we have fixed the performance gap between phi-3-mini-4k and phi-3-mini-128k, please try latest ipex-llm (
2.1.0b20240508)

and we are still working to improve phi-3-mini performance

Answer 2 · 2024-05-10T01:21:35.000Z

we have made a big performance improvement for phi-3-mini (4k and 128k) in latest version, please upgrade to latest ipex-llm to try it (2.1.0b20240509)