Can wanda speedup LLM inference performance?

[ifromeast]

I have tested a llama-13b model, the pruned model's latency is almost the same as the raw model.

[Eric-mingjie]

Wanda is a weight pruning method developed for LLMs. The speedup guarantee and conditions are the same as the standard magnitude pruning approach. For example, unstructured sparsity would need specialized hardware for practical speedup.