/tamil-sentence-tokenizer

A sentence tokenizer NLP tool for the Tamil language

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

tamil-sentence-tokenizer

A sentence tokenizer NLP tool for the Tamil language

Sentence Tokenizer

Command-line utility to perform sentence tokenization on a given Tamil corpus text file.

Usage

python sentence_tokenizer.py <input_file>

For help

python sentence_tokenizer.py -h

Features

  1. No preprocessing needed
  2. Works on any OS which supports Python 3
  3. Handles input file of any size