Create Targeted Minimal Pair "Stress-Tests" for Sensitivity to Social Groups

Question

Create Targeted Minimal Pair "Stress-Tests" for Sensitivity to Social Groups

epavlick opened this issue 3 years ago · 2 comments

Coordinate with Meg Mitchell about this

Answer 1 · 2021-08-23T13:49:26.000Z

Hi,

I had recently created gender sensitivity tests that cover binary and non-binary genders (in English and Chinese), and name sensitivity test that is geographically diverse.

Seems very related and would love to join efforts! I guess the contrast sets collected in above pointers can be combined with context sentences from established datasets such as CrowS-Pairs and StereoSet, to create stress-tests?

Answer 2 · 2021-08-23T21:51:27.000Z

This is great! We are working on exactly this within the Evaluation Bias Fairness Social Impact subgroup. Come join the channel and you can start iterating with us. =)