This repository contains the code for out of the box ready to use zero-shot classifiers among different tasks, such as Topic Labelling or Relation Extraction. It is built on top of 🤗 HuggingFace Transformers library, so you are free to choose among hundreds of models. You can either, use a dataset specific classifier or define one yourself with just labels descriptions or templates! The repository contains the code for the following publications:
- 📄 Ask2Transformers - Zero Shot Domain Labelling with Pretrained Transformers accepted in GWC2021.
- 📄 (Coming soon) Label Verbalization and Entailment for Effective Zero- and Few-Shot Relation Extraction accepted in EMNLP2021
Follow the links to see some examples of how to use the library on each task.
- Topic classification evaluated on BabelDomains (Camacho- Collados and Navigli, 2017) dataset.
- Relation classification evaluated on TACRED (Zhang et al., 2017) dataset.
By using Pip (check the last release)
pip install a2tOr by clonning the repository
git clone https://github.com/osainz59/Ask2Transformers.git
cd Ask2Transformers
python -m pip install .By default, roberta-large-mnli checkpoint is used to perform the inference. You can try different models to perform the zero-shot classification, but they need to be finetuned on a NLI task and be compatible with the AutoModelForSequenceClassification class from Transformers. For example:
roberta-large-mnlijoeddav/xlm-roberta-large-xnlifacebook/bart-large-mnlimicrosoft/deberta-v2-xlarge-mnli
Coming soon: t5-large like generative models support.
Cite this paper if you want to cite stuff related to Relation Extraction, etc.
Coming soon.Cite this paper if you want to cite stuff related with the library or topic labelling (A2TDomains or our paper results).
@inproceedings{sainz-rigau-2021-ask2transformers,
title = "{A}sk2{T}ransformers: Zero-Shot Domain labelling with Pretrained Language Models",
author = "Sainz, Oscar and
Rigau, German",
booktitle = "Proceedings of the 11th Global Wordnet Conference",
month = jan,
year = "2021",
address = "University of South Africa (UNISA)",
publisher = "Global Wordnet Association",
url = "https://www.aclweb.org/anthology/2021.gwc-1.6",
pages = "44--52",
abstract = "In this paper we present a system that exploits different pre-trained Language Models for assigning domain labels to WordNet synsets without any kind of supervision. Furthermore, the system is not restricted to use a particular set of domain labels. We exploit the knowledge encoded within different off-the-shelf pre-trained Language Models and task formulations to infer the domain label of a particular WordNet definition. The proposed zero-shot system achieves a new state-of-the-art on the English dataset used in the evaluation.",
}