this repository contains bunch of experiences on MedNli dataset. remember to upload the dataset files (in parquet format) before running the notebooks.
Here it is the ways that i experimented for textual inference on Mednli dataset, my best model had 77% accuracy which has under 10% difference with SOTA accuracy.
SciFive is the state of the art for medical language inference, this language model achieved 86% accuracy on MedNli dataset. you can use this notebook to evaluate its performance
for textual inference the most simple idea that comes to mind is to extract premise and hypothesis word embedding concatenating them and within a simple feed forward layer do the classification task. it can achieve about 68% accuracy on MedNli dataset. read this article and see this notebook for further informations.
another idea is to give premise and hypothesis within a start token to a recurrent neural network module (LSTM for example) and then use the last hidden layer of the module for classification. see this notebook or this notebook for further informations.
inspired by this article we used an attention module to further contextualize the embeddings it achieved about 69% accuracy on MedNli dataset. see this notebook or this notebook for further informations.
inspired by this article Ive implemented a neural network to do the inference based on interaction space see this notebook for further informations.
using the famous attention is all you need paper encoder we enhanced contexualizing of premise and hypothesis and then taking hadamard product of embeddings we do a classification task, it achieved 75% accuracy on MedNli dataset. this notebook have done this using Early stoping, this one have done it without Early stoping and this one have done it with freezing word embedding layer.
using previous way we did some Augmentation on datas using text attack library see this notebook for further informations.
having a neural network that do the inference for us we used another neural network to decide whether this network is sure about its decison or not, then for the examples that the neural network is unsure about it we can use another agent.
the idea is to give the contextualized embedding of the transformers to a Recurrent neural network, see this notebook for further informations.
inspired by this and this articles we enhanced our accuracy to 76% by combining symbolic and deep AI, see this notebook for implementing it using the selector network. and see this notebook for implementing it without the selector network.
thanks to this implemention of transformers, i used it with slight changes in my code.