Eli, Avani, Pangbo, and Hilly's project for LING 573 at the University of Washington.
- Make sure you have Conda installed.
- To install Conda, enter the following commands:
wget https://repo.anaconda.com/archive/Anaconda3-2021.11-Linux-x86 64.shsh Anaconda3-2021.11-Linux-x86 64.sh
- Issue the following command from the root of this repo to fine-tune the models and print the results: condor_submit cmd/D4.cmd
- The script will activate a Conda environment in the shared folder
/projects/assigned/2122_ling573_elibales/env/ - Note: D4.cmd only runs the adaptation portion. D4_all.cmd runs both the primary and adaptation task.
- The script will activate a Conda environment in the shared folder
The accuracy and F1 scores of our ensemble classification model can be found under:
- src/results/D4/primary/evaltest/D4_scores.out
- src/results/D4/primary/devtest/D4_scores.out
- src/results/D4/adaptation/devtest/D4_scores.out
- src/results/D4/adaptation/devtest/D4_scores.out
-
Primary:
f1: {'f1': 0.9627391742195367}
accuracy: {'accuracy': 0.9538077403245943}
-
Adaptation:
f1: {'f1': 0.6054421768707482}
accuracy: {'accuracy': 0.5303643724696356}
The top 3 scores for the Hahackathon shared task humor controversy subtask (our adaptation task) were:
- accuracy: 0.5089 | f1: 0.6299
- accuracy 0.4699 | f1: 0.6270
- accuracy: 0.4553 | f1: 0.6249
project
│
└─── src/ contains the source code and data used for this project.
│
└─── results/ contains the results of our system.
│
└─── doc/ contains documentation of our system.
│
│ │
│ └─── /archive contains unused files.
│ │
│ └─── /data contains the data used to train and evaluate the model.
│ │
│ └─── /configs contains JSON files used to configure the models.
│ │
│ └─── /exetuables contains bash scripts to run the models.
│
└─── outputs/ contains the system outputs.
│ │
│ └─── /Dx the results of each deliverable.
│
└─── cmd/ contains the HTCondor job submission files.
│
└─── bin/ a recycling bin for unused files.