/Indian-LawyerGPT

Fine-Tuning Falcon-7B, LLAMA 2 with QLoRA to create an advanced AI model with a profound understanding of the Indian legal context.

Primary LanguageJupyter Notebook

Indian Law AI: Fine-Tuning Falcon-7B & LLAMA 2 Language Models

Welcome to our exciting project where we are adapting two cutting-edge language models, Falcon-7B & LLAMA 2, to become proficient in Indian law.

Overview

Our adventure began with a modest 150 Q&As on Indian law. Now, we're charging ahead with an impressive dataset of 3300 instructions! This AI legal project combines:

  • Falcon-7B & LLAMA 2: State-of-the-art language models, prepped and ready for legal training.
  • PEFT & QLoRA: The dream duo for memory-efficient and high-performance model fine-tuning.
  • Our Dataset: Comprehensive Indian law knowledge, spanning constitutional law, civil rights, and more!

Dataset Creation

Dataset Creation (3)

Dive into our Dataset

Our dataset is designed with four key features: instruction, input, output, and prompt. Crafted to shape our models into AI law experts! Dataset on Hugging Face : https://huggingface.co/datasets/nisaar/Constitution_Of_India_Instruction_Set https://huggingface.co/datasets/nisaar/Articles_Constitution_3300_Instruction_Set https://huggingface.co/datasets/nisaar/LLAMA2_Legal_Dataset_4.4k_Instructions

Fine Tuning Process

Fine Tuning

Track the Progress

Get a front-row seat to the training progress with TensorBoard. Kickstart it, navigate to the provided localhost link, and witness the models learn: