PotatoSpudowski/fastLLaMa
fastLLaMa: An experimental high-performance framework for running Decoder-only LLMs with 4-bit quantization in Python using a C/C++ backend.
CMIT
Issues
- 0
GGUF and/or LLama-3 support?
#86 opened by BHX2 - 10
- 3
how to load model in webui ?
#82 opened by valdesguefa - 0
Webui UX issue on mobile
#84 opened by stduhpf - 0
Port llama.cpp openCL support to fastllama?
#83 opened by stduhpf - 1
Make running fastLLaMa on windows simple!
#43 opened by PotatoSpudowski - 1
- 2
Deciding the Schema for the protocol between webUI and webSocket Server
#77 opened by PotatoSpudowski - 0
- 0
Implement the WebSocket Server
#79 opened by PotatoSpudowski - 21
Pip support testing
#55 opened by PotatoSpudowski - 1
Designing the UI
#75 opened by PotatoSpudowski - 2
Pip uninstall not removing the package
#74 opened by stduhpf - 4
How install on Windows?
#27 opened by soleimanian - 2
TypeError: Model.generate() got an unexpected keyword argument 'stop_word'
#73 opened by vootshiclone - 3
- 7
Feature suggestions!
#38 opened by PotatoSpudowski - 1
- 8
from build.fastllama import Model, ModelKind ModuleNotFoundError: No module named 'build.fastllama'
#56 opened by lucasjinreal - 4
n_ctx argument is ignored
#29 opened by stduhpf - 4
When stop words are reached, they get ingested, but are not forwarded to streaming_fn.
#62 opened by stduhpf - 5
convert-pth-to-ggml.py expects 2 parts for ALPACA-LORA-13B, but it has only one
#57 opened by stduhpf - 5
Cannot build this
#51 opened by bratislav - 2
Passing arguments such as -ins
#41 opened by regstuff - 12
- 17
AVX2 performance issue
#21 opened by Showdown76py - 6
Bad Magic error
#58 opened by Th3F0x-code - 1
Lora adaptor support
#49 opened by PotatoSpudowski - 1
Cmake Error
#50 opened by notmehul - 5
Is Alpaca 13B and 30B tested?
#12 opened by imranraad07 - 18
Unicode characters break tokenizer
#24 opened by stduhpf - 14
- 3
function wrap for getting the embedding
#36 opened by complexly - 11
Add posibility to choose python version for module or make it independent from version
#16 opened by yevhen-kalyna - 1
Fix multiple relative pointer transform
#15 opened by amitsingh19975 - 2
Return Log Probs in Output
#6 opened by PotatoSpudowski - 9
- 5
Error when using setup.py
#42 opened by SumYin - 2
Make prompt ingestion faster!
#2 opened by PotatoSpudowski - 1
Still slow on AVX2 CPUs
#37 opened by Showdown76py - 1
Does not support Python 3.11
#25 opened by jhud - 8
Error at ./build.sh
#18 opened by Niellai - 1
Error using build.sh
#19 opened by poohzaza166 - 22
Getting Error with make command
#8 opened by imranraad07 - 1
Example doc has incorrect repo
#7 opened by raldebsi - 1
Stop words is buggy!
#1 opened by PotatoSpudowski - 2
- 2