Xin-Zhou-smu/FalconStreaming

Falcon40B and 7B (Instruct) with streaming, top-k, and beam search

Jupyter Notebook

FalconStreaming

Falcon40B and 7B (Instruct) with streaming, top-k, and beam search 4bit

You must change the runtime type for Colab to run both 7B and 40B!

Screen Shot 2023-06-06 at 2 02 18 PM

Adapted from Serin32's starter code on huggingface: https://huggingface.co/tiiuae/falcon-40b/discussions/38