/Text-Clustering

Text-Clustering

Primary LanguageJupyter Notebook

Text-Clustering

image

Main Topic

  • Feature Extraction
  • Modeling
  • Visualization
  • Evaluation

Intoduction

Appling clustering methods such as EM, K-Means, Hierarchal Clustering, and LDA in order to find the similar groups (clusters) within the following books from Gutenberg

  • austen-emma: Romance – Fiction
  • bible-kjv: Religious
  • edgeworth-parents: children- fiction
  • chesterton-brown: mystery
  • whitman-leaves: Drama

Dataset

Take five different samples of Gutenberg digital books, which are all of five different genres and of five different authors, that are semantically different. Separate and set aside unbiased random partitions for training

Methodology

  • Feature Engineering

    • Bag Of Words
    • Term Frequency - Inverse Document Frequency
    • Word2Vec
    • LDA

  • Modeling

    • K- means
    • EM
    • hierarchical Clustering
    • LDA
    • XGBClassifier
    • pycaret

  • Evaluation

    • Using silhouette
    • Using Cohen's Kappa

Conclusion

  • In conclusion, the champion model is the EM with TF-IDF with 2 clusters and coherence =0.24 and BIC, AIC =0

contact

imagehttps://www.linkedin.com/in/%D9%90%D9%90alaa-elkhashap/