Part 1: ParsiNLU dataset Classification (Textual Entailment)
In this question, we intend to use Transformers in text application. One of the applications in the Natural Language Processing studied is Textual entailment. Textual entailment is a classification that can be used to understand the relationship between two sentences. In this assignment there are 3 relationships:
- Entailment: In this case, the first sentence proves that the second sentence is correct.
- Contradiction: There is contradiction between the two sentences.
- Neutral: The sentences are not related to each other.
In this task we are to use contextualized word embedding like BERT.
The main topics discussed in this part are:
- The need of preprocessing data
- Implementing a neural network to classify the data using
XLM-RoBERTa. This model is available in Hugging Face. - Implementing a neural network to classify the data using
ParsBERT
One of the datasets published by the ParsiNLU article is related to the Textual Entailment task. This dataset is available here: https://huggingface.co/datasets/persiannlp/parsinlu_entailment
The main topics discussed in this part are:
- Implementing a neural network to classify the English data using
BERT. - Implementing a neural network to classify the Persian data using
ParsBERT. - Implementing a neural network to classify the Persian & English data together using
XLM-RoBERTa.
The dataset is in Question2_Data folder given. This dataset has three columns: source (English text), targets (translated Persian text) and category (including three tags: Quran, Bible, Mizan (collection of literary masterpieces translated into English)).
According to this paper, in zero-shot cross-lingual transfer, a supervised NLP task trained on a corpus in one language is directly applicable to another language without any additional training.
According to this paper, some NLP tasks are not applicable to most human languages due to the lack of annotated training data for various NLP tasks. Cross-lingual transfer learning (CLTL) is a viable method for building NLP models for a low-resource target language by leveraging labeled data from other (source) languages which have more labeled data.
I can mention that sometimes it is better to have a large labeled data in another language than training the model with small data with the same language as test data.
In this part, I train the model on English data and test it on Persian data.
The dataset is in Question2_Data folder given. This dataset has three columns: source (English text), targets (translated Persian text) and category (including three tags: Quran, Bible, Mizan (collection of literary masterpieces translated into English)).
Report is available here.