mohamedehab00/A-Hybrid-Arabic-Text-Summarization-Approach-based-on-Transformers

In this paper, we proposed a sequential hybrid model based on a transformer to summarize Arabic articles. We used two approaches of summarization to make our model. The First is the extractive approach which depends on the most important sentences from the articles to be the summary, so we used Deep Learning techniques specifically transformers such as AraBert to make our summary, The second is abstractive, and this approach is similar to human summarization, which means that it can use some words which have the same meaning but different from the original text. We apply this kind of summary using MT5 Arabic pre-trained transformer model. We sequentially applied these two summarization approaches to building our A3SUT hybrid model. The output of the extractive module is fed into the abstractive module. We enhanced the summary’s quality to be closer to the human summary by applying this approach. We tested our model on the ESAC dataset and evaluated the extractive summary using the Rouge score technique; we got a precision of 0.5348 and a recall of 0.5515, and an f1 score of 0.4932 and the evaluation of the abstractive model is evaluated by user satisfaction. We add some features to our summary to make it more understandable by applying the metadata generation task” data about data” and classification. By applying metadata generation, we add facilities to our summary, identification, and summary organization. Metadata provides essential contextual details, as not all summaries are self-describing. Also, classify the original text to determine the summary topic before reading. We acquire 97.5% accuracy by using Support Vector Machine (SVM) and trained it using NADA corpus.

A Hybrid Arabic Text Summarization Approach based on Transformers

Abstract :

In this paper, we proposed a sequential hybrid model based on a transformer to summarize Arabic articles. We used two approaches of summarization to make our model. The First is the extractive approach which depends on the most important sentences from the articles to be the summary, so we used Deep Learning techniques specifically transformers such as AraBert to make our summary, The second is abstractive, and this approach is similar to human summarization, which means that it can use some words which have the same meaning but different from the original text. We apply this kind of summary using MT5 Arabic pre-trained transformer model. We sequentially applied these two summarization approaches to building our A3SUT hybrid model. The output of the extractive module is fed into the abstractive module. We enhanced the summary’s quality to be closer to the human summary by applying this approach. We add some features to our summary to make it more understandable by applying the metadata generation task "data about data" and classification. By applying metadata generation, we add facilities to our summary, identification, and summary organization.

IEEE Paper Link : IEEE Paper

Proposed Approach :

Our Approach Diagram

Data Provided in Data Folder :

• EASC ( Extractive Summarization Dataset )
• NADA Corpus ( Text Classification Dataset )

Provided Models :

• Classical SVM Model with a Grid Search Optimzation Approach
• Extractive Bert Model : Model Source
• Abstractive mt5 Model : Model Source
• Meta Data Extraction Model : Model Source

How to run the project :

1. Run the TextClassificationModel.ipynb notebook

We run this notebook ( only one time ) to train the Text Classification model and convert it to a pickle file to ensure not to train the model each time we use it.

2. Run the MainProject.ipynb notebook

This notebook is the main notebook which loads all of the models (Text Classification model, Extractive Bert model, Abstractive mt5 model, A3SUT (our hybrid model) and Meta Data Extraction model). All outputs are in the Models output section (the last section in MainProject.ipynb).

Thanks to all project contributors :

Contributer Name	LinkedIn	GitHub	E-mail
Mohamed Ehab			Send Email
Ameen Reda			Send Email
Ismail Ahmed			Send Email
John Adel			Send Email
Mostafa Kafafy			Send Email
Nada Salah			Send Email
Assoc. Prof. Ghada Khoriba			Send Email
Assoc. Prof. Enssaf Hussien			Send Email

Feel Free to Contact us ❤️