/duelingBandits

Primary LanguagePython

duelingBandits

Date and code for the paper: Xinyi Yan, Chengxi Luo, Charles L. A. Clarke, Nick Craswell, and Ellen M. Voohees, and Pablo Castells. 2022. Human Preferences as Dueling Bandits. 45th International ACM SIGIR Conference on Research and Development in Information Retrieval.

Additional code may be found at https://github.com/claclark/preferences

This paper is one in a series of related papers. See https://github.com/claclark/Compatibility

The data release includes a third finalization phase (not included in the paper):

  • questions.tsv: Questions used for judging (slightly edited from original TREC questions)
  • judgments.txt: Crowdsourced preference judgments
  • Set-I.qrels: qrels based on first finalization phase
  • Set-II.qrels: qrels based on second finalization phase
  • Set-III.qrels: qrels based on third finalization phase
  • Combined-I+II.qrels: qrels created by combining first and second finalization phases
  • Combined-I+II+III.qrels: qrels created combining first, second, and third finalization phases