python3 faq_tests.py [--pretrained huggingface_model_name] [--questions question_dataset_path] [--answers answer_dataset_path] [--cmtime disp_time_seconds] [--save_dir output_directory] [--filename results_filename] [--cm] [--verb] [--trust_remote_code]
-
pretrained: Path to the model from the Huggingface website (in format username/modelname)
-
questions: Path to spreadsheet file with questions
-
answers: Path to spreadsheet file with answers
-
model_path: Path to the model to be evaluated
-
cmtime: Cofusion matrix display duration in seconds
-
cm: Show confusion matrices during evaluation
-
verb: Print incorrectly matched pairs in the following format:
Querry question : Incorrectly matched question (answer)
-
save_dir: Directory, where results should be saved (Default: ./output/)
-
filename: Name of the file with evaluation results. (Default: Accuracies.log)
-
trust_remote_code: Allow remote code execution during model initialization
python3 confusion_inspector.py [--pretrained huggingface_model_name] [--questions question_dataset_path] [--trust_remote_code]
Shows a similarity heatmap between all dataset questions. Click on a specific pixel in the heatmap to print the corresponding pair of questions and their similarity value in the terminal.