Falcon40B and 7B (Instruct) with streaming, top-k, and beam search 4bit
![Screen Shot 2023-06-06 at 2 02 18 PM](https://private-user-images.githubusercontent.com/24441720/243787026-3275a0da-e39d-4233-bcf1-0e9c56daef40.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjQ1OTA4MTIsIm5iZiI6MTcyNDU5MDUxMiwicGF0aCI6Ii8yNDQ0MTcyMC8yNDM3ODcwMjYtMzI3NWEwZGEtZTM5ZC00MjMzLWJjZjEtMGU5YzU2ZGFlZjQwLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MjUlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODI1VDEyNTUxMlomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTA4MzM5YmI4NGYzNmY2OTZjNjZkNTdmN2MxZGQzOGExMDVmZDFlNzRlOTMwNTc1NGFjNDEyNTJhMTI1NTQ5YjYmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.RCpvw-N6baaFhknrhiAQm-kj0FjpPirQTWch77P_8xU)
Adapted from Serin32's starter code on huggingface: https://huggingface.co/tiiuae/falcon-40b/discussions/38
Falcon40B and 7B (Instruct) with streaming, top-k, and beam search
Jupyter Notebook
Falcon40B and 7B (Instruct) with streaming, top-k, and beam search 4bit
Adapted from Serin32's starter code on huggingface: https://huggingface.co/tiiuae/falcon-40b/discussions/38