metrics.seat: Implementations of SEAT, and variations of SEAT by Lauscher et al. (2021) and Tan et al. (2019).metrics.disco: Implementation of DisCo.metrics.lpbs: Implementation of the log probability bias score.
-The notebooks folder contains subfolders LM_correlations, sublist correlations, winobias
- LM_correlations: contains correlation results on how different template types and embedding types correlate based on scores from different language models.
- sublist_correlations: contains correlation results on how different template types and embedding types correlate based on scores from different sublist of jobs.
- correlation_analysis*.ipynb in each folder contains the results from the correlation experiments
- The codes used for Winobias experiments are directly adopted from Manela et al. (2021) https://github.com/12kleingordon34/NLP_masters_project