Profile Llama3 Attention Heads

[griff4692]

See this section of the writeup.

This issue involves implementing the paper below into gpt-fast

• Model tells you what to discard: Adaptive kv cache compression for llms

and doing a sanity check to see if the profiled attention heads are accurate for llama-3