/upv_faq_transformers

upv_faq test for transformers models.

Primary LanguagePython

UPV FAQ Semantic Text Similarity Test

Dependencies:

Usage:

python3 faq_tests.py [--pretrained huggingface_model_name] [--questions question_dataset_path] [--answers answer_dataset_path] [--cmtime disp_time_seconds] [--save_dir output_directory] [--filename results_filename] [--cm] [--verb] [--trust_remote_code]

Arguments:

  • pretrained: Path to the model from the Huggingface website (in format username/modelname)

  • questions: Path to spreadsheet file with questions

  • answers: Path to spreadsheet file with answers

  • model_path: Path to the model to be evaluated

  • cmtime: Cofusion matrix display duration in seconds

  • cm: Show confusion matrices during evaluation

  • verb: Print incorrectly matched pairs in the following format:

    Querry question : Incorrectly matched question (answer)

  • save_dir: Directory, where results should be saved (Default: ./output/)

  • filename: Name of the file with evaluation results. (Default: Accuracies.log)

  • trust_remote_code: Allow remote code execution during model initialization

Confusion inspector:

python3 confusion_inspector.py [--pretrained huggingface_model_name] [--questions question_dataset_path] [--trust_remote_code]

Shows a similarity heatmap between all dataset questions. Click on a specific pixel in the heatmap to print the corresponding pair of questions and their similarity value in the terminal.