This repository is dedicated to the task of mass summarization using large language models (LLMs), particularly focusing on leveraging the capabilities of llama models. The primary goal is to facilitate large-scale dataset cleaning and summarization to enhance data understanding and usability.
The Mass_Summarization repository contains notebooks that demonstrate the use of llama2 models for summarizing large datasets. These notebooks are designed to be used with Google Colab and Runpod, making it easy for users to run and modify the code according to their needs.
- LLAMA2_13B_Summarizer.ipynb: Demonstrates how to perform mass summarization using the llama2 13B model on Google Colab.
- LLAMA_2_70B_Information_Extractor.ipynb: Shows the use of the llama2 70B model for information extraction on Runpod.