Urdu Word Segmentation
This repository contains code & dataset for Urdu word segmentation as described in paper Urdu Word Segmentation using Conditional Random Fields (CRFs).
Requirement(s)
It is implemented in python and requires scikit-learn and python-crfsuite.
Dataset
A manually annotated corpus of approximately 111,000 tokens is available for download.
Reference(s)
If you use this tool in any of your work, please cite below paper.
Urdu Word Segmentation using Conditional Random Fields (CRFs)
@InProceedings{C18-1217,
author = "Bin Zia, Haris
and Raza, Agha Ali
and Athar, Awais",
title = "Urdu Word Segmentation using Conditional Random Fields (CRFs)",
booktitle = "Proceedings of the 27th International Conference on Computational Linguistics",
year = "2018",
publisher = "Association for Computational Linguistics",
pages = "2562--2569",
location = "Santa Fe, New Mexico, USA",
url = "http://aclweb.org/anthology/C18-1217"
}
License(s)
Copyright (c) 2018 CSaLT, ITU
Code licensed under the MIT License: http://opensource.org/licenses/MIT Data licensed under CC-BY 4.0: https://creativecommons.org/licenses/by/4.0/