/How-Reasonable-are-Common-Sense-Reasoning-Tasks

Datasets to use the evaluation protocol for the Winograd Schema Challenge defined in the EMNLP2019 paper "How Reasonable are Common-Sense Reasoning Tasks: A Case-Study on the Winograd Schema Challenge and SWAG".

How Reasonable are Common-Sense Reasoning Tasks?

This repository contains two datasets : the original WSC with the switched version of each sentence whenever the process does not obscure the sentence or affect the rationale used to resolve the target pronoun and the original WSC sentences labeled as associative or non-associative.

Files

  • WSC_switchable_label.json contains the original sentence sentence, a binary label which indicates if the two candidates in the sentence can be switched is_switchable, and the switched version of the sentence sentence_switched.

  • WSC_associative_label.json contains the orginal sentence sentence and a binary label is_associative which indicates if one candidate antecedent associates strongly with the clause containing the pronoun.