/LLM-WikipediaQA

Document Q&A on Wikipedia articles using LLMs

Primary LanguageJupyter NotebookMIT LicenseMIT

Document Q&A on Wikipedia articles

Run document Q&A on Wikipedia articles. Use Wikipedia-API to search/retrieve/beautify Wikipedia articles, LangChain for the Q&A framework, and OpenAI & HuggingFace models for embeddings and LLMs. The meat of the code is in WikipediaQA.py.

For the accompanying blog post, see https://georgesung.github.io/ai/llm-qa-eval-wikipedia/

Architecture

Search and index Wikipedia article

arch

Q&A on article

arch

Instructions

Batch runs

For a batch run over different LLMs and embedding models, you can run the notebook WikipediaQA_batch_runs.ipynb in your own compute instance, or run the same notebook on Colab:

Open In Colab

Interactive app

To run an interactive Gradio app, do the following:

  • pip install -r requirements.txt
  • If you're using OpenAI ada embeddings and/or GPT 3.5, then cp template.env .env, and edit .env to include your OpenAI API key
  • python gradio_app.py

Results with different LLMs and embeddings

For detailed results and analysis, see the full blog post here