susanli2016/Machine-Learning-with-Python

Error in 'topic_modeling_Gensim.ipynb'

hudmb opened this issue · 0 comments

hudmb commented

Hi,

I have tried to run 'topic_modeling_Gensim.ipynb' and I get this error at this stage in the notebook. Can anyone help?: -

import random
text_data = []
with open('dataset.csv') as f:
    for line in f:
        tokens = prepare_text_for_lda(line)
        if random.random() > .99:
            print(tokens)
            text_data.append(tokens)
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-54-7369a1356984> in <module>()
      3 with open('dataset.csv') as f:
      4     for line in f:
----> 5         tokens = prepare_text_for_lda(line)
      6         if random.random() > .99:
      7             print(tokens)

<ipython-input-51-4f0710beb9ee> in prepare_text_for_lda(text)
      1 def prepare_text_for_lda(text):
----> 2     tokens = tokenize(text)
      3     tokens = [token for token in tokens if len(token) > 4]
      4     tokens = [token for token in tokens if token not in en_stop]
      5     tokens = [get_lemma(token) for token in tokens]

<ipython-input-45-f5c7dc83eb04> in tokenize(text)
      3 def tokenize(text):
      4     lda_tokens = []
----> 5     tokens = parser(text)
      6     for token in tokens:
      7         if token.orth_.isspace():

NameError: name 'parser' is not defined