Code for the paper from IFSA (International Fuzzy Systems Association) 2023.
Large pre-trained models such as BERT and RoBERTa are currently used as the state-of-the-art foundation for Natural Language Processing classification tasks. Fuzzy Fingerprints can be used as a classification layer in such models to improve the interpretability of the classification results, while reducing model complexity and without a significant loss in performance. In this work, we exploit a recent framework that combines Fuzzy Fingerprints with large pre-trained models. By employing a Genetic Algorithm, we further optimize these fingerprints to obtain a hybrid classification model that has the potential to achieve similar performance and become more interpretable while less complex than traditional large pre-trained classifiers. Our experiments show that with a smaller fingerprint size and thus more easily interpretable, the optimization improves the base work with a larger fingerprint size and achieves competing results with state-of-the-art approaches.