text-analysis

reddit scraper, simple text summarizer

summarizer

for counting word frequency:
- remove everything except letters
- count frequency of words (tokenized, stemmed), except stopwords
- normalize frequency values
tokenize text into sentences
sentence -> list of stemmed words
for each sentence:
- sum up the frequency values for all frequent words -> score
- CHANGES TO TRY:
  - score = score + 2 * (score / word count)
  - score = score / word count