OBQnA

An OpenBook Question 'n' Answer System

Introduction

OBQnA is a high-level OO Python package which aims to provide an easy and intuitive way of creating a OpenBook Question ‘n’ Answer system.

The package parses PDF files using Apache Tika, splits the corpus into passages and calculates their corresponding Dense vector representation exploiting a Transformer NLP model. For each question asked, the system performs a Dense Passage Retrieval, using an efficient similarity search library (Faiss, ScaNN or Annoy) and extracts the answer from the retrieved passages.

Install

To install simply do pip install -r requirements.tx

note: If you want to use GPU please install CUDA

Python code example

We are using J. R. R. Tolkien's Lord Of The Rings Trilogy and the Hobbit for the following example.

For more detailed explanation please read the Documentation.

Parsing PDFs and performing some basic text cleaning

from obqna.process import PDFParser, Passages
from obqna.qa import QuestionAnswering

parser = PDFParser("../books/") # Path of PDFs
books = parser.parse()
books = parser.clean(books)

Splitting the corpus into passages

passages = Passages()
corpus = passages.df2passages(books)

Calculate the vector represantation of each passage and store the corresponding indices

searcher_type = "scann" # other choices: "faiss", "annoy"
qna = QuestionAnswering(searcher_type)
qna.prepare(corpus)

Ask questions

questions = [
    "Who is Galadriel?",
    "Who is Isildur?",
    "Who is Boromir's father?",
    "Who is Aragorn?",
    "Was the ring destroyed?",
    "What language is on the One Ring inscription?"
]

for question in questions:
    print(f"Question: {question}: ")
    results = qna.ask(question)
    print(f"Answer: {results['answer']}")
    print(10*'-')

Question: Who is Galadriel?: 
Answer: The Lady of Lorien
----------
Question: Who is Isildur?: 
Answer: Elendils son
----------
Question: Who is Boromir's father?: 
Answer: Lord Denethor
----------
Question: Who is Aragorn?: 
Answer: Heir of Isildur
----------
Question: Was the ring destroyed?: 
Answer: it perished from the world in the ruin of his first realm
----------
Question: What language is on the One Ring inscription?: 
Answer: Black Speech
----------

nikosNalmpantis/OBQnA