nltk/nltk

NLTK thinks `turn` is a noun when it shoud be a verb.

alf1e opened this issue · 1 comments

According to NLTK turn in the sentence turn on the lights is a noun but in the sentence turn off the lights its a verb.

In both of these cases it should be a verb.

Here is my code:

pos_tag = nltk.pos_tag(tokens)
simplified_tags = [(word, nltk.map_tag('en-ptb', 'universal', tag)) for word, tag in pos_tag]
statement.tags = simplified_tags

Any help is appreciated. Thanks

The NLTK POS tagging model, like all NLP models, is not perfect. You can also try the spaCy POS tagging models, perhaps they work better for you!