/WordEmbedding_Attention

Word embedding with context vector calculated by attention mechanism

Primary LanguagePythonGNU General Public License v3.0GPL-3.0

WordEmbedding_Attention

CBoW word embedding or MSE approach for it with context vector calculated by attention mechanism

TODOs:

  • Modify model architecture for higher efficiency(may be change to torch.nn.transformer in the future)
  • Add GPU support(Done)
  • Fine tuning(NO GPU anymore hence no)