/named-entity-recognition

NER for Twitter data using Viterbi Decoding, CRF and feature engineering in Python

Primary LanguagePython

named-entity-recognition

  • The objective of this project was to perform supervised NER for Twitter data using Viterbi decoding, CRF and feature engineering.
  • Designed dozens of features using lexicons, POS tags, and N-Grams
  • Utilized BIO-encoded data to train the Perceptron of the Conditional Random Field (CRF) tagger
  • Implemented Viterbi Decoding using Dynamic Programming for the sequence tagging part of CRF tagger.
  • Used CONLL evaluation to evaluate the model and reported F1 scores.