keras-team/keras-nlp

Any plans for more Llama type models?

Opened this issue · 1 comments

Is there a plan to support more models similar to llama, which only need simple modifications or even no modifications to the existing llama backbone to achieve compatibility?

for example
yi model :https://huggingface.co/01-ai This model is exactly the same as llama.
qwen model :https://huggingface.co/Qwen This model only need to modify three attention bias

Are there plans to support even larger llama models, such as the llama 70B model?

I plan to add larger LLaMA presets in KerasNLP. No plans for Yi or Quen yet.