[FEATURE] ADD Jamba Support
TechxGenus opened this issue · 0 comments
TechxGenus commented
Is your feature request related to a problem? Please describe.
Jamba is a state-of-the-art, hybrid SSM-Transformer LLM. It delivers throughput gains over traditional Transformer-based models, while outperforming or matching the leading models of its size class on most common benchmarks.
Describe the solution you'd like
https://huggingface.co/ai21labs/Jamba-v0.1
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.