whoisltd/qr4cqa

[VietAI Reseach] Impliment Question Rewriting for Conversational Question Answering

Question Rewriting for Conversational Question Answering

[VietAI Reseach]

Implement Question Rewriting for Conversational Question Answering

For Development

Support Python 3.8+

Install virtualenv

pip install virtualenv

Create virtualenv

virtualenv -p python3.8 venv

Activate virtualenv

# for linux:
source venv/bin/activate
# for windows:
venv\Scripts\activate

Installation

Install Pytorch + CUDA Toolkit

Install Pytorch in this link compatible with your CUDA.

pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113

Option 1: Install libraries

pip install -r requirements.txt

Update submodules if it needs.

bash init.sh

Option 2: Install step-by-step

Install libraries

pip install underthesea seqeval datasets sentencepiece

git clone https://github.com/huggingface/transformers --branch v4.27.0 --single-branch
cd transformers
pip install -e .

Process data

python process_data.py

Training

Change configuration in src/bin/train.sh

bash src/bin/train.sh