Implementations and wrapper of bias scores for text embeddings.
All requirements are listed in requirements.txt.
In the notebooks, we use a wrapper for huggingface models. You can install the lib from source:
https://github.com/UBI-AGML-NLP/Embeddings
Download this repository and install via pip install .
A minimalistic example for the usage of implemented bias scores can be found in example.ipynb.
So far, we implemented geometrical bias scores that measure the association of embeddings with protected groups. Other scores and bias test will follow.
Papers that introduced the scores are linked.