This repository presents the code of THUIR team in COLIEE2023 Task 1 and Task 2. Details can be found in the relevant papers.
In task 1 Legal Case Retrieval, THUIR won the championship paper. In task 2 Legal Case Entailment, THUIR received the third place paper.
This legal case competition focuses on two aspects of legal information processing related to a database of predominantly Federal Court of Canada case laws, provided by Compass Law.
The legal case retrieval task involves reading a new case Q, and extracting supporting cases S1, S2, ... Sn for the decision of Q from the entire case law corpus. Through the document, we will call the supporting cases for the decision of a new case 'noticed cases'.
| Team | Submission | Precision | Recall | F1 score |
|---|---|---|---|---|
| THUIR | thuirrun2 | 0.2379 | 0.4063 | 0.3001 |
| THUIR | thuirrun3 | 0.2173 | 0.4389 | 0.2907 |
| IITDLI | iitdli_task1_run3 | 0.2447 | 0.3481 | 0.2874 |
| THUIR | thuirrun1 | 0.2186 | 0.3782 | 0.2771 |
| NOWJ | nowj.d-ensemble | 0.2263 | 0.3527 | 0.2757 |
This task involves the identification of a paragraph from existing cases that entails the decision of a new case.
Given a decision Q of a new case and a relevant case R, a specific paragraph that entails the decision Q needs to be identified. We confirmed that the answer paragraph can not be identified merely by information retrieval techniques using some examples. Because the case R is a relevant case to Q, many paragraphs in R can be relevant to Q regardless of entailment.
This task requires one to identify a paragraph which entails the decision of Q, so a specific entailment method is required which compares the meaning of each paragraph in R and Q in this task.
| Team | Submission | Precision | Recall | F1 score |
|---|---|---|---|---|
| CAPTAIN | mt5l-ed | 0.7870 | 0.7083 | 0.7456 |
| CAPTAIN | mt5l-ed4 | 0.7864 | 0.6750 | 0.7265 |
| THUIR | thuir-monot5 | 0.7900 | 0.6583 | 0.7182 |
| CAPTAIN | mt5l-e2 | 0.7596 | 0.6583 | 0.7054 |
| THUIR | thuir-ensemble_2 | 0.7315 | 0.6583 | 0.6930 |
Please visit COLIEE 2023 to apply for the whole dataset.
@misc{li2023sailer,
title={SAILER: Structure-aware Pre-trained Language Model for Legal Case Retrieval},
author={Haitao Li and Qingyao Ai and Jia Chen and Qian Dong and Yueyue Wu and Yiqun Liu and Chong Chen and Qi Tian},
year={2023},
eprint={2304.11370},
archivePrefix={arXiv},
primaryClass={cs.IR}
}
@misc{li2023thuircoliee,
title={THUIR@COLIEE 2023: Incorporating Structural Knowledge into Pre-trained Language Models for Legal Case Retrieval},
author={Haitao Li and Weihang Su and Changyue Wang and Yueyue Wu and Qingyao Ai and Yiqun Liu},
year={2023},
eprint={2305.06812},
archivePrefix={arXiv},
primaryClass={cs.IR}
}
@misc{li2023thuircoliee,
title={THUIR@COLIEE 2023: More Parameters and Legal Knowledge for Legal Case Entailment},
author={Haitao Li and Changyue Wang and Weihang Su and Yueyue Wu and Qingyao Ai and Yiqun Liu},
year={2023},
eprint={2305.06817},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
If you find our work useful, please do not save your star!
If you have any questions, please email liht22@mails.tsinghua.edu.cn