/nlp-preprocess

Primary LanguagePython

nlp-preprocess

Basic fucntions that help with data preprocessing, visualization and vectors

preprocess.py

Removes stop words, puncuation, makes everything lowercase, uses regex (removes digits, special characters, etc.), can also stem and lemmatize sentences and provide parts of speech tags

word2vec.py

Uses the Word2Vec module to vectorize words and get similarity, common words and vectors of words

visualize.py

Uses WordCloud module to make visuals of given text

Modules Used

  • Word2Vec
  • matplotlib
  • WordCloud
  • nltk
  • re
  • Optional (Web Scraping for Data):
    • bs4
    • requests