/basic-nlp

Repository for basic applied NLP tasks such as learning how to use pre-trained word vectors for predicting analogies between pairs of words, and how to build a crude language model to perform translation between languages. Also includes commonly used NLTK methods reference.

Primary LanguageJupyter Notebook

Basic NLP methods for reference

Repository for basic applied NLP tasks for an introduction to:

  1. NLTK notes: Basic NLTK reference, containing excerpts from the NLTK book
  2. NLTK - Comparing pronoun and modal verb language use: Basic codes for counting pronoun and modal verb usage in text, and how to use NLTK to do this faster.
  3. Zipf's Law: Zipf's law states that given a large sample of words used, the frequency of any word is inversely proportional to its rank in the frequency table. So word number n has a frequency proportional to 1/n. This file explores this empirical law practically.
  4. Analogy Prediction: Using pre-trained word vectors for predicting analogies between word pairs based on relations.
  5. Language Model for machine translation between English, French, Italian and Spanish: Building a crude character level bigram language model to perform translation between languages such as English, French, Italian and Spanish.