Infini-AI-Lab/Sequoia

Is there any benchmark that compares Sequoia against vanilla speculative decoding?

KexinFeng opened this issue · 2 comments

KexinFeng commented

Hi,

Thanks for the great work!

I'm wondering if there is any benchmark that compares Sequoia against vanilla speculative decoding?

preminstrel commented

Once you get your acceptance rate, I think you can directly calculate the theoretical best gamma and speedup for vanilla speculative decoding.

dreaming-panda commented

Hello, you can use 4-chain, 8-chain in L40_growmaps as growmaps, which will produce "tree structures" for vanilla speculative decoding. Thank you!

  • 👍1