Recommand using Python 3.6.9.
Also available with Python 3.8.6 (as on csil).
Install dependencies
$ pip install catboost==0.24.3 krovetz==1.0.2
Copy necessary data file into project directory
-
TREC disk4-5 (must be in valid xml format)
Unzip csil file/cs/sandbox/faculty/tyang/290N/trec-disk4-5_processed.zip
-
Query title
From hw1-starter:title-queries.301-450
-
Relevance score
From hw1-starter:qrels.trec6-8.nocr
-
Generate inverted index
Trec45 data -> indexer.py -> inverted index
$ python indexer.py --trec trec-disk4-5_processed.xml
-
Generate feature data for training
inverted index, query title, relevance score -> feature.py -> feature dataset
$ python feature.py --query title-queries.301-450 --relevance qrels.trec6-8.nocr
-
Train models and evaluate
feature dataset -> tree.py
$ python tree.py
Compute BM25 as baseline
feature dataset -> tfidf.py
$ python tfidf.py
Argument | Description | Default |
---|---|---|
indexer.py | ||
--trec | TREC disk 4-5 file | trec-disk4-5_processed.xml |
--invertedindex | inverted index file to output | invertedindex.pickle |
feature.py | ||
--invertedindex | inverted index file from indexer.py | invertedindex.pickle |
--query | queries title file | title-queries.301-450 |
--relevance | relevance file of doc-query pair | qrels.trec6-8.nocr |
--dataset | dataset file to output | dataset.pickle |
tfidf.py | ||
--dataset | dataset file from feature.py | dataset.pickle |
tree.py | ||
--dataset | dataset file from feature.py | dataset.pickle |
--iteration | (int) number of repetition for average result | 1 |
--fold | (int) k-fold validation | 4 |
Format:
$ python xxx.py --arg1name arg1value --arg2name arg2value ...