This repository contains the code and models for "Tamil-Llama", a project focused on enhancing the performance of language models for the Tamil language. It builds upon the open-source LLaMA model, introducing additional Tamil tokens and employing the LoRA methodology for efficient training. Please read the technical report for more details.
Technical Report: https://arxiv.org/abs/2311.05845
| Model | Type | Data | Base Model | # Params | Download Links |
|---|---|---|---|---|---|
| Tamil LLaMA 7B Base | Base model | 12GB | LLaMA 7B | 7B | HF Hub |
| Tamil LLaMA 13B Base | Base model | 4GB | LLaMA 13B | 13B | HF Hub |
| Tamil LLaMA 7B Instruct | Instruction following model | 145k instructions | Tamil LLaMA 7B Base | 7B | HF Hub |
| Tamil LLaMA 13B Instruct | Instruction following model | 145k instructions | Tamil LLaMA 13B Base | 13B | HF Hub |
The repository includes a Tamil-translated version of the Alpaca dataset and a subset of the OpenOrca dataset, which are used for instruction fine-tuning and evaluation.
Tamil Alpaca: abhinand/tamil-alpaca
Tamil Alpaca Orca: abhinand/tamil-alpaca-orca
Tamil LLaMA Eval: abhinand/tamil-llama-eval
Prompt Template Without Input
{system_prompt}
### Instruction:
{instruction or query}
### Response:
{response}
Prompt Template With Input
{system_prompt}
### Instruction:
{instruction or query}
### Input:
{input}
### Response:
{response}
It's important to note that the models have not undergone detoxification. Therefore, while they possess impressive linguistic capabilities, there is a possibility for them to generate content that could be deemed harmful or offensive. We urge users to exercise discretion and supervise the model's outputs closely, especially in public or sensitive applications.
We welcome contributions to this project. If you have suggestions or improvements, please open an issue or a pull request.
This project is licensed under the GNU GPL v3.0 license - see the LICENSE.md file for details.
If you use this model or the Tamil-Llama dataset in your research, please cite:
@misc{balachandran2023tamilllama,
title={Tamil-Llama: A New Tamil Language Model Based on Llama 2},
author={Abhinand Balachandran},
year={2023},
eprint={2311.05845},
archivePrefix={arXiv},
primaryClass={cs.CL}
}For any queries regarding the codebase or research, please reach out to Abhinand Balachandran at abhinandb.ml@gmail.com.