Falcon40B and 7B (Instruct) with streaming, top-k, and beam search 4bit
![Screen Shot 2023-06-06 at 2 02 18 PM](https://private-user-images.githubusercontent.com/24441720/243787026-3275a0da-e39d-4233-bcf1-0e9c56daef40.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM3NTY4MDcsIm5iZiI6MTcyMzc1NjUwNywicGF0aCI6Ii8yNDQ0MTcyMC8yNDM3ODcwMjYtMzI3NWEwZGEtZTM5ZC00MjMzLWJjZjEtMGU5YzU2ZGFlZjQwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE1VDIxMTUwN1omWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWUxMTYxNzZlNGJiYjA5OWI3ZDBhMTI3YWI0YTQ2NjQyZGIxMGMyMTlmNDZjNGM0NTJjZGIzYmNiZDRmMjQ1NjQmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.pywquRL8ACkvFoMvLAK7wY8-H2zC2V6aj7B8p3k_Y5c)
Adapted from Serin32's starter code on huggingface: https://huggingface.co/tiiuae/falcon-40b/discussions/38
Falcon40B and 7B (Instruct) with streaming, top-k, and beam search
Jupyter Notebook
Falcon40B and 7B (Instruct) with streaming, top-k, and beam search 4bit
Adapted from Serin32's starter code on huggingface: https://huggingface.co/tiiuae/falcon-40b/discussions/38