/allennlp-models

Officially supported AllenNLP models

Primary LanguagePythonApache License 2.0Apache-2.0




Officially supported AllenNLP models.

Build PyPI License Codecov


❗️ To file an issue, please open a ticket on allenai/allennlp and tag it with "Models". ❗️

In this README

About

This repository contains the components - such as DatasetReader, Model, and Predictor classes - for applying AllenNLP to a wide variety of NLP tasks. It also provides an easy way to download and use pre-trained models that were trained with these components.

Tasks and components

This is an overview of the tasks supported by the AllenNLP Models library along with the corresponding components provided, organized by category. For a more comprehensive overview, see the AllenNLP Models documentation or the Paperswithcode page.

  • Classification

    Classification tasks involve predicting one or more labels from a predefined set to assign to each input. Examples include Sentiment Analysis, where the labels might be {"positive", "negative", "neutral"}, and Binary Question Answering, where the labels are {True, False}.

    🛠 Components provided: Dataset readers for various datasets, including BoolQ and SST, as well as a Biattentive Classification Network model.

  • Coreference Resolution

    Coreference resolution tasks require finding all of the expressions in a text that refer to common entities.

    See nlp.stanford.edu/projects/coref for more details.

    🛠 Components provided: A general Coref model and several dataset readers.

  • Generation

    This is a broad category for tasks such as Summarization that involve generating unstructered and often variable-length text.

    🛠 Components provided: Several Seq2Seq models such a Bart, CopyNet, and a general Composed Seq2Seq, along with corresponding dataset readers.

  • Language Modeling

    Language modeling tasks involve learning a probability distribution over sequences of tokens.

    🛠 Components provided: Several language model implementations, such as a Masked LM and a Next Token LM.

  • Multiple Choice

    Multiple choice tasks require selecting a correct choice among alternatives, where the set of choices may be different for each input. This differs from classification where the set of choices is predefined and fixed across all inputs.

    🛠 Components provided: A transformer-based multiple choice model and a handful of dataset readers for specific datasets.

  • Pair Classification

    Pair classification is another broad category that contains tasks such as Textual Entailment, which is to determine whether, for a pair of sentences, the facts in the first sentence imply the facts in the second.

    🛠 Components provided: Dataset readers for several datasets, including SNLI and Quora Paraphrase.

  • Reading Comprehension

    Reading comprehension tasks involve answering questions about a passage of text to show that the system understands the passage.

    🛠 Components provided: Models such as BiDAF and a transformer-based QA model, as well as readers for datasets such as DROP, QuAC, and SQuAD.

  • Structured Prediction

    Structured prediction includes tasks such as Semantic Role Labeling (SRL), which is for determining the latent predicate argument structure of a sentence and providing representations that can answer basic questions about sentence meaning, including who did what to whom, etc.

    🛠 Components provided: Dataset readers for Penn Tree Bank, OntoNotes, etc., and several models including one for SRL and a very general graph parser.

  • Sequence Tagging

    Sequence tagging tasks include Named Entity Recognition (NER) and Fine-grained NER.

    🛠 Components provided: A Conditional Random Field model and dataset readers for datasets such as CoNLL-2000, CoNLL-2003, CCGbank, and OntoNotes.

  • Text + Vision

    This is a catch-all category for any text + vision multi-modal tasks such Visual Question Answering (VQA), the task of generating a answer in response to a natural language question about the contents of an image.

    🛠 Components provided: Several models such as a ViLBERT model for VQA and one for Visual Entailment, along with corresponding dataset readers.

Pre-trained models

Every pretrained model in AllenNLP Models has a corresponding ModelCard in the allennlp_models/modelcards/ folder. Many of these models are also hosted on the AllenNLP Demo and the AllenNLP Project Gallery.

To programmatically list the available models, you can run the following from a Python session:

>>> from allennlp_models import pretrained
>>> print(pretrained.get_pretrained_models())

The output is a dictionary that maps the model IDs to their ModelCard:

{'structured-prediction-srl-bert': <allennlp.common.model_card.ModelCard object at 0x14a705a30>, ...}

You can load a Predictor for any of these models with the pretrained.load_predictor() helper. For example:

>>> pretrained.load_predictor("mc-roberta-swag")

Here is a list of pre-trained models currently available.

Installing

From PyPI

allennlp-models is available on PyPI. To install with pip, just run

pip install allennlp-models

Note that the allennlp-models package is tied to the allennlp core package. Therefore when you install the models package you will get the corresponding version of allennlp (if you haven't already installed allennlp). For example,

pip install allennlp-models==2.2.0
pip freeze | grep allennlp
# > allennlp==2.2.0
# > allennlp-models==2.2.0

From source

If you intend to install the models package from source, then you probably also want to install allennlp from source. Once you have allennlp installed, run the following within the same Python environment:

git clone https://github.com/allenai/allennlp-models.git
cd allennlp-models
ALLENNLP_VERSION_OVERRIDE='allennlp' pip install -e .
pip install -r dev-requirements.txt

The ALLENNLP_VERSION_OVERRIDE environment variable ensures that the allennlp dependency is unpinned so that your local install of allennlp will be sufficient. If, however, you haven't installed allennlp yet and don't want to manage a local install, just omit this environment variable and allennlp will be installed from the main branch on GitHub.

Both allennlp and allennlp-models are developed and tested side-by-side, so they should be kept up-to-date with each other. If you look at the GitHub Actions workflow for allennlp-models, it's always tested against the main branch of allennlp. Similarly, allennlp is always tested against the main branch of allennlp-models.

Using Docker

Docker provides a virtual machine with everything set up to run AllenNLP-- whether you will leverage a GPU or just run on a CPU. Docker provides more isolation and consistency, and also makes it easy to distribute your environment to a compute cluster.

Once you have installed Docker you can either use a prebuilt image from a release or build an image locally with any version of allennlp and allennlp-models.

If you have GPUs available, you also need to install the nvidia-docker runtime.

To build an image locally from a specific release, run

docker build \
    --build-arg RELEASE=1.2.2 \
    --build-arg CUDA=10.2 \
    -t allennlp/models - < Dockerfile.release

Just replace the RELEASE and CUDA build args with what you need. You can check the available tags on Docker Hub to see which CUDA versions are available for a given RELEASE.

Alternatively, you can build against specific commits of allennlp and allennlp-models with

docker build \
    --build-arg ALLENNLP_COMMIT=d823a2591e94912a6315e429d0fe0ee2efb4b3ee \
    --build-arg ALLENNLP_MODELS_COMMIT=01bc777e0d89387f03037d398cd967390716daf1 \
    --build-arg CUDA=10.2 \
    -t allennlp/models - < Dockerfile.commit

Just change the ALLENNLP_COMMIT / ALLENNLP_MODELS_COMMIT and CUDA build args to the desired commit SHAs and CUDA versions, respectively.

Once you've built your image, you can run it like this:

mkdir -p $HOME/.allennlp/
docker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/models

Note: the --gpus all is only valid if you've installed the nvidia-docker runtime.