FasterDecoding/Medusa

Issues

• PPL compute

  #106 opened a month ago by yuyangxie96
  0

• Token-wise the same generalization?

  #99 opened a month ago by Ageliss
  2

• Some questions about sampling strategy

  #63 opened 7 months ago by qianxiao1111
  2

• Using Medusa with Whisper

  #100 opened 2 months ago by AvivSham
  2

• Containerization with Dockerfile to setup medusa

  #104 opened a month ago by gangooteli
  0

• Conversation roles must alternate user/assistant/user/assistant/

  #102 opened a month ago by gangooteli
  0

• How to use the finetuned mistal model for inference with Medusa

  #75 opened 5 months ago by pradeepdev-1995
  7

• Medusa Training Loss

  #95 opened 2 months ago by TomYang-TZ
  5

• [bug] fix preprocess function

  #101 opened 2 months ago by xiezipeng-ML
  0

• ImportError: cannot import name 'is_flash_attn_available' from 'transformers.utils'

  #98 opened 2 months ago by imneov
  1

• Is there no way to inference without training?

  #77 opened 4 months ago by MoOo2mini
  3

• Is there a bug in gen_model_answer_baseline.py?

  #96 opened 2 months ago by qspang
  1

• train medusa stage-2

  #94 opened 2 months ago by smartliuhw
  1

• mistral.json

  #93 opened 2 months ago by Git-L1
  0

• which dataset should i use when training medusa heads with llama2 7b

  #92 opened 2 months ago by tu2022
  0

• Training Medusa heads

  #70 opened 5 months ago by mmilunovic-mdcs
  5

• Why medusa-2 train llama2 with no such great improvement?

  #85 opened 4 months ago by MeJerry215
  2

• Cant it support chatgllm?

  #91 opened 3 months ago by PeterXiaTian
  0

• HYDRA support?

  #90 opened 3 months ago by arunpatala
  0

• Misleading Name LLM Name MEDUSA

  #89 opened 3 months ago by Pittconnect
  0

• about Medusa mask details

  #88 opened 3 months ago by dhcode-cpp
  0

• release medusa-llm v1.0

  #84 opened 4 months ago by zhyncs
  1

• [Dynamic Batching] Concerns about whether features are not supported using Medusa

  #82 opened 4 months ago by Ageliss
  0

• Encounter an CUDA error when set Medusa head

  #81 opened 4 months ago by 1649759610
  0

• Why the speed up of Medusa 1 on vicuna changed?

  #79 opened 4 months ago by niyunsheng
  2

• deepspeed support

  #78 opened 4 months ago by jiangix-paper
  0

• medusa-2 HF repo has no 'medusa_num_heads' in config

  #76 opened 5 months ago by HaebinShin
  1

• CUBLAS_STATUS_EXECUTION_FAILED when training Medusa Head with base model set to Llama2 7B

  #45 opened 5 months ago by void-main
  7

• Medusa 1 and 2 speed up

  #73 opened 5 months ago by LotuSrc
  2

• OSError

  #69 opened 5 months ago by qspang
  3

• About changing LLM from LLAMA to LLAMA-2

  #68 opened 5 months ago by dydrkfl06
  2

• AttributeError: 'LlamaForCausalLM' object has no attribute 'medusa_head'

  #60 opened 5 months ago by blwaji
  2

• how did you construct the sparse tree architecture

  #67 opened 5 months ago by pengfeiwu1999
  2

• Will using this method result in inconsistent output results?

  #56 opened 5 months ago by niyunsheng
  7

• Sparse candidate generation confusion

  #64 opened 5 months ago by zankner
  6

• Install the package with the console script ?

  #50 opened 5 months ago by devrimcavusoglu
  1

• name not exist "from medusa.model.medusa_choices import medusa_choices"

  #48 opened 5 months ago by JianbangZ
  4

• [New feature] More sampling schemes

  #39 opened 5 months ago by Jokoe66
  2

• Question about Heads warmup

  #74 opened 5 months ago by eloooooon
  1

• vLLM support

  #41 opened 9 months ago by MichaelJayW
  11

• Clarifications on Models + Batch Size

  #66 opened 6 months ago by RonanKMcGovern
  5

• Can I make an AWQ quantization?

  #65 opened 6 months ago by RonanKMcGovern
  1

• TypeError: __init__() got an unexpected keyword argument 'medusa_num_heads'

  #55 opened 7 months ago by HackGiter
  3

• Results for different configs

  #62 opened 7 months ago by zankner
  8

• How to load finetune checkpoint files directly？

  #61 opened 7 months ago by qianxiao1111
  0

• AttributeError: 'LlamaForCausalLM' object has no attribute 'medusa_head'

  #59 opened 8 months ago by blwaji
  0

• FasterTransformer support

  #57 opened 8 months ago by niyunsheng
  1

• [Feature Request] Qwen model support

  #52 opened 9 months ago by JianbangZ
  1

• How to test latency between medusa & baseline

  #49 opened 9 months ago by YixinSong-e
  3

• errors occurred when running simple_gradio_interface.py

  #51 opened 9 months ago by MeWannaSleep
  2