/NLP-Toolkit

Some useful NLP tools and their implementations.

Primary LanguagePython

NLP-Toolkit

This is an explonatary repository which contains some useful NLP tools and their implementations.

Functions

  • Plots

    • WordCloud
    • Heaps Law
    • Zipfs Law

  • Language Models

    • MLE
    • KneserNeyInterpolated

  • word2vec (gensim)

    • CBOW
    • skip-gram

Setup

1.Download the source code.

virutalenv venv

or

python -m virtualenv venv

2.Lauch the below command in order to download the requirements.

pip install -r requirements.txt

3.Check for the examples folder in order to learn how to use the existing modules.

Sample Datasets

  • T_sample5000.pkl
  • T_sample20000.pkl

Acknowledgements

Many thanks to Reyyan Yeniterzi for providing us the data set with function definitions.