When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it

Sebastian Schuster and Tal Linzen
In Proceedings of NAACL 2022. [preprint]

Contents:

  • analysis: R scripts for analyzing human and model results, plot generation scripts
  • config: Config files for model experiments
  • experiments: HTML code for human experiments.
  • generation_analysis: Results and analysis code for "Likely continuations" analysis
  • human_data: Anonomyzed data from human experiments
  • models: Code for running perplexity computations and sampling model generations
  • results: Outputs of model experiments
  • scripts: Various scripts to process stimuli and model experiment results
  • stimuli: Experimental stimuli