Utilizing Hugging Face Transformers and Blurr, I streamlined data scraping and text summarization. Three specialized models were integrated seamlessly, allowing for accurate summaries. Deployment on Hugging Face Spaces ensured easy sharing and accessibility. This project showcases the efficiency of modern NLP for succinct text summarization.
Article Wordcloud
Highlights Wordcloud
Most Common Words in Highlights
Most Common Words in Article
Highlights Text Length
Article Text Length
I've collected 8176 articles and their highlights from the Daily Mail website. The goal is to train a text summarization model that can generate brief summaries for given articles. By utilizing advanced techniques and libraries like Hugging Face's Transformers, along with tools like Blurr and Fastai, I aim to create an efficient summarization system. The aim is to develop a solution that automatically produces concise and informative summaries, aiding readers in understanding the key points of the articles quickly.
I have trained three models for text summarization: distilbart-cnn-6-6, distilbart-cnn-12-6, and facebook/bart-large-cnn. Out of these, I've deployed the distilbart-cnn-12-6 model, which is a transformer model from Hugging Face.For implementing text summarization, I utilized the Blurr library along with Fastai. All the notebooks detailing the training and deployment process for these models are available here.Feel free to explore the notebooks to understand the methodologies and techniques employed in training and deploying these models for text summarization.
distilbart-cnn-6-6
| accuracy | precision | recall | f1 |
|---|---|---|---|
| 0.91 | 0.90 | 0.97 | 0.93 |
distilbart-cnn-12-6
| accuracy | precision | recall | f1 |
|---|---|---|---|
| 0.90 | 0.89 | 0.97 | 0.93 |
facebook/bart-large-cnn
| accuracy | precision | recall | f1 |
|---|---|---|---|
| 0.90 | 0.89 | 0.97 | 0.93 |
The text summarization model has been deployed on the HuggingFace Spaces Gradio App. You can access the implementation either by going to the deployment folder or directly through the provided link to the application.
I've developed a Flask application specifically for text summarization. It's designed to take input text and generate a condensed summary. You're invited to explore the Flask branch to delve into the implementation details further. To try out the application in real-time, simply visit the live website accessible through the provided link.
