bytedance/lightseq

Is llama inference available now?

frankxyy opened this issue · 1 comments

frankxyy commented

Hi, I am wondering if llama inference now is available, thank you.

cg3dland commented

Any updates?

Is the llama2 inference is available too?

If the llama2 inference is available now, may I have a sample code which show to use lightseq to accelerate the llama2 inference?