AnswerDotAI/cold-compress

Issues

• SnapKV

  #42 opened 15 days ago by SimJeg
  2

• Does the repo support quantization methods? Does the repo support kv merge methods?

  #40 opened 17 days ago by foreverpiano
  5

• torch dependency results in error

  #41 opened 18 days ago by maxjeblick
  1

• It seems the compression doesn't work and compression ratio is always =0

  #39 opened 19 days ago by foreverpiano
  5

• Implement ThinK

  #36 opened a month ago by griff4692
  0

• Implement PyramidInfer

  #35 opened a month ago by griff4692
  0

• Implement InfLLM

  #30 opened 2 months ago by griff4692
  0

• MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention

  #29 opened 2 months ago by griff4692
  0

• LLama3 GIST

  #2 opened 2 months ago by griff4692
  0

• Compute Heavy Hitters KV-Cache Eviction Policy

  #3 opened 2 months ago by griff4692
  0

• Profile Llama3 Attention Heads

  #4 opened 2 months ago by griff4692
  0

• Implement Evaluations (Decide on datasets and benchmark initial methods)

  #8 opened 2 months ago by griff4692
  0

• Record Model Speed in evals

  #6 opened 3 months ago by griff4692
  0

• Record memory consumption

  #7 opened 3 months ago by griff4692
  0

• Benchmark prompt summarizers with frontier LLMs (GPT-4o / Opus)

  #1 opened 3 months ago by griff4692
  1

• Experiment with Fixed Global Tokens

  #5 opened 3 months ago by griff4692
  1