PyThaiNLP is a Python package for text processing and linguistic analysis, similar to NLTK with a focus on the Thai language.
PyThaiNLP เป็นไลบารีภาษาไพทอนสำหรับประมวลผลภาษาธรรมชาติ คล้ายกับ NLTK โดยเน้นภาษาไทย ดูรายละเอียดภาษาไทยได้ที่ README_TH.MD
News
Now, You can contact with or ask any questions of the PyThaiNLP team.
Version | Description | Status |
---|---|---|
4.0 | Stable | Change Log |
dev |
Release Candidate for 5.0 | Change Log |
- PyThaiNLP 2 requires Python 3.7+. Python 2.7 users can use PyThaiNLP 1.6. See 2.0 change log | Upgrading from 1.7 | Upgrading ThaiNER from 1.7
- PyThaiNLP Get Started notebook | API document | Tutorials
- Official website | PyPI | Facebook page
- Who uses PyThaiNLP?
- Model cards - for technical details, caveats, and ethical considerations of the models developed and used in PyThaiNLP
PyThaiNLP provides standard NLP functions for Thai, for example part-of-speech tagging, linguistic unit segmentation (syllable, word, or sentence). Some of these functions are also available via the command-line interface.
List of Features
- Convenient character and word classes, like Thai consonants (
pythainlp.thai_consonants
), vowels (pythainlp.thai_vowels
), digits (pythainlp.thai_digits
), and stop words (pythainlp.corpus.thai_stopwords
) -- comparable to constants likestring.letters
,string.digits
, andstring.punctuation
- Thai linguistic unit segmentation/tokenization, including sentence (
sent_tokenize
), word (word_tokenize
), and subword segmentations based on Thai Character Cluster (subword_tokenize
) - Thai part-of-speech tagging (
pos_tag
) - Thai spelling suggestion and correction (
spell
andcorrect
) - Thai transliteration (
transliterate
) - Thai soundex (
soundex
) with three engines (lk82
,udom83
,metasound
) - Thai collation (sorted by dictionary order) (
collate
) - Read out number to Thai words (
bahttext
,num_to_thaiword
) - Thai datetime formatting (
thai_strftime
) - Thai-English keyboard misswitched fix (
eng_to_thai
,thai_to_eng
) - Command-line interface for basic functions, like tokenization and POS tagging (run
thainlp
in your shell)
pip install --upgrade pythainlp
This will install the latest stable release of PyThaiNLP.
Install different releases:
- Stable release:
pip install --upgrade pythainlp
- Pre-release (nearly ready):
pip install --upgrade --pre pythainlp
- Development (likely to break things):
pip install https://github.com/PyThaiNLP/pythainlp/archive/dev.zip
Some functionalities, like Thai WordNet, may require extra packages. To install those requirements, specify a set of [name]
immediately after pythainlp
:
pip install pythainlp[extra1,extra2,...]
List of possible extras
full
(install everything)attacut
(to support attacut, a fast and accurate tokenizer)benchmarks
(for word tokenization benchmarking)icu
(for ICU, International Components for Unicode, support in transliteration and tokenization)ipa
(for IPA, International Phonetic Alphabet, support in transliteration)ml
(to support ULMFiT models for classification)thai2fit
(for Thai word vector)thai2rom
(for machine-learnt romanization)wordnet
(for Thai WordNet API)
For dependency details, look at the extras
variable in setup.py
.
- Some additional data, like word lists and language models, may be automatically downloaded during runtime.
- PyThaiNLP caches these data under the directory
~/pythainlp-data
by default. - The data directory can be changed by specifying the environment variable
PYTHAINLP_DATA_DIR
. - See the data catalog (
db.json
) at https://github.com/PyThaiNLP/pythainlp-corpus
Some of PyThaiNLP functionalities can be used via command line with the thainlp
command.
For example, to display a catalog of datasets:
thainlp data catalog
To show how to use:
thainlp help
License | |
---|---|
PyThaiNLP source codes and notebooks | Apache Software License 2.0 |
Corpora, datasets, and documentations created by PyThaiNLP | Creative Commons Zero 1.0 Universal Public Domain Dedication License (CC0) |
Language models created by PyThaiNLP | Creative Commons Attribution 4.0 International Public License (CC-by) |
Other corpora and models that may be included in PyThaiNLP | See Corpus License |
- Please fork and create a pull request :)
- For style guides and other information, including references to algorithms we use, please refer to our contributing page.
You can read INTHEWILD.md.
If you use PyThaiNLP
in your project or publication, please cite the library as follows:
Wannaphong Phatthiyaphaibun, Korakot Chaovavanich, Charin Polpanumas, Arthit Suriyawongkul, Lalita Lowphansirikul, & Pattarawat Chormai. (2016, Jun 27). PyThaiNLP: Thai Natural Language Processing in Python. Zenodo. http://doi.org/10.5281/zenodo.3519354
or by BibTeX entry:
@misc{pythainlp,
title = "{P}y{T}hai{NLP}: {T}hai Natural Language Processing in {P}ython",
author = "Phatthiyaphaibun, Wannaphong and
Chaovavanich, Korakot and
Polpanumas, Charin and
Suriyawongkul, Arthit and
Lowphansirikul, Lalita and
Chormai, Pattarawat",
month = jun,
year = "2016",
doi = {10.5281/zenodo.3519354},
publisher = {Zenodo},
url = {http://doi.org/10.5281/zenodo.3519354}
}
Our NLP-OSS 2023 paper:
Wannaphong Phatthiyaphaibun, Korakot Chaovavanich, Charin Polpanumas, Arthit Suriyawongkul, Lalita Lowphansirikul, Pattarawat Chormai, Peerat Limkonchotiwat, Thanathip Suntorntip, and Can Udomcharoenchaikit. 2023. PyThaiNLP: Thai Natural Language Processing in Python. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023), pages 25–36, Singapore, Singapore. Empirical Methods in Natural Language Processing.
and its BibTeX entry:
@inproceedings{phatthiyaphaibun-etal-2023-pythainlp,
title = "{P}y{T}hai{NLP}: {T}hai Natural Language Processing in {P}ython",
author = "Phatthiyaphaibun, Wannaphong and
Chaovavanich, Korakot and
Polpanumas, Charin and
Suriyawongkul, Arthit and
Lowphansirikul, Lalita and
Chormai, Pattarawat and
Limkonchotiwat, Peerat and
Suntorntip, Thanathip and
Udomcharoenchaikit, Can",
editor = "Tan, Liling and
Milajevs, Dmitrijs and
Chauhan, Geeticka and
Gwinnup, Jeremy and
Rippeth, Elijah",
booktitle = "Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)",
month = dec,
year = "2023",
address = "Singapore, Singapore",
publisher = "Empirical Methods in Natural Language Processing",
url = "https://aclanthology.org/2023.nlposs-1.4",
pages = "25--36",
abstract = "We present PyThaiNLP, a free and open-source natural language processing (NLP) library for Thai language implemented in Python. It provides a wide range of software, models, and datasets for Thai language. We first provide a brief historical context of tools for Thai language prior to the development of PyThaiNLP. We then outline the functionalities it provided as well as datasets and pre-trained language models. We later summarize its development milestones and discuss our experience during its development. We conclude by demonstrating how industrial and research communities utilize PyThaiNLP in their work. The library is freely available at https://github.com/pythainlp/pythainlp.",
}
Logo | Description |
---|---|
Since 2019, our contributors Korakot Chaovavanich and Lalita Lowphansirikul have been supported by VISTEC-depa Thailand Artificial Intelligence Research Institute. | |
We get support of free Mac Mini M1 from MacStadium for running CI builds. |