NeMo_Question_Answering

This tutorial was adopted from the Nemo User Guide.

Key Features

NVIDIA NeMo is a conversational AI toolkit built for researchers working on automatic speech recognition (ASR), text-to-speech synthesis (TTS), large language models (LLMs), and natural language processing (NLP).
The primary objective of NeMo is to help researchers from industry and academia to reuse prior work (code and pretrained models) and make it easier to create new conversational AI models.

State of the Art pretrained NeMo models are freely available on HuggingFace Hub and NVIDIA NGC.
These models can be used to transcribe audio, synthesize speech, or translate text in just a few lines of code.
They have extensive tutorials that can be run on Google Colab.

Feel free to run the first two model training sessions using the BERT and BART models.
If you require an A100 GPU, you can complete the remaining exercises.
If you require an RTX 2080 Ti GPU (11GB memory), you may encounter an out-of-memory issue. Running the GPT-2 model will need approximately 13 GB, and the S2S BART model for MS-4. MARCO will need approximately 12 GB. One way to address this is by reducing the batch size to fit the training on a single RTX GPU.
If you are interested in running with multiple GPUs, more technical information can be found in the PyTorch Lightning documentation: MODEL PARALLEL GPU TRAINING.