ADAM -- Question Answering System

A question answering system that extracts answers from Wikipedia to questions posed in natural language. Inspired by IBM Watson and START. We are currently focused on improving the accuracy of the extracted answers. Follow the creator's blog at shirishkadam.com for updates on progress.

Getting Started

Elasticsearch is being used to store and index the scrapped and parsed texts from Wikipedia. Elasticsearch 7.X installation guide can be found at Elasticsearch Documentation. You might have to start the elasticsearch search service.

$ git clone https://github.com/5hirish/adam_qas.git
$ cd adam_qas
$ pip install -r requirements.txt
$ python -m qas.adam -vv "When was linux kernel version 4.0 released ?"

Note: The above installation downloads the best-matching default english language model for spaCy. But to improve the model's accuracy you can install other models too. Read more at spaCy docs.

$ python -m spacy download en_core_web_md

Running with Docker

$ git clone https://github.com/5hirish/adam_qas.git
$ cd adam_qas
$ docker-compose up

Now both conntainers are up and running. Next step is to enter in the python container and run Adam:

$ docker exec -it $(docker ps -a -q  --filter ancestor=adam_qas_adam) bash
$ python -m qas.adam -vv "When was linux kernel version 4.0 released ?"

References

Find more in depth documentation about the system with its research paper and system architecture here

Requirements

Python Package dependencies listed in requirements.txt Upgrading Elasticsearch 6.X:

Rolling Update 6.2 to 6.8 > ref
Rolling Update 6.8 to 7.1 > ref

Features

Extract information from Wikipedia
Classify questions with regular expression (default)
Classify questions with a SVM (optional)
Vector space model used for answer extraction
Rank candidate answers
Merge top 5 answers into one response

Current Project State ?

GitHub Issue #36: Invalid Answers

TODO

Replace Wikipedia APIs with custom scraper
Storing extracted data in database (elasticsearch)
SQLite test input data storage
Anaphora resolution in both questions and answers
Machine learning query constructor rather than rule-based
Improve vector space language model for answer extraction

Contributions

Please see our contributing documentation for some tips on getting started.

Maintainers

@5hirish - Shirish Kadam