/SemEval-2016

SemEval-2016 Task 9: Chinese Semantic Dependency Parsing

Primary LanguagePython

SemEval-2016 Task 9 (Chinese Semantic Dependency Parsing) Datasets

For more information: http://alt.qcri.org/semeval2016/task9/

Thanks to all participants!

Statistics

This task provides two distinguished corpora in appreciable quantity respectively in the domain of NEWS and TEXTBOOKS (from primary school textbooks).

Set NEWS TEXTBOOKS
train 8301 10754
validation 534 1535
test 1233 3073

Systems

Fifteen organizations were registered to participate in this task. Finally, five systems were received from three organizations. The main evaluation results are listed below, see the task description paper for detailed description and results.

Results on NEWS corpus:

System LF UF NLF NUF
IHS-RD-Belarus 59.06 77.64 40.84 60.20
OCLSP (lbpg) 57.22 74.93 45.57 58.03
OCLSP (lbpgs) 57.81 75.54 41.56 54.34
OCLSP (lbpg75) 57.78 75.40 48.89 58.28
OSU-CHGCG 55.69 73.72 49.23 60.71

Results on TEXTBOOKS corpus:

System LF UF NLF NUF
IHS-RD-Belarus 68.59 82.41 50.57 64.58
OCLSP (lbpg) 65.54 79.39 51.75 63.21
OCLSP (lbpgs) 66.21 79.85 47.79 55.51
OCLSP (lbpg75) 66.38 79.91 57.51 63.87
OSU-CHGCG 65.17 78.83 54.70 65.71

Reference

If you wish to use this data in your research, please cite:

@InProceedings{che-EtAl:2016:SemEval,
	author    = {Che, Wanxiang  and  Shao, Yanqiu  and  Liu, Ting  and  Ding, Yu},
	title     = {SemEval-2016 Task 9: Chinese Semantic Dependency Parsing},
	booktitle = {Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)},
	year      = {2016},
	publisher = {Association for Computational Linguistics},
	pages     = {1074-1080},
}

SemEval-2016 Task 9 Evaluation Committee