/NLP_Resource

This is reading list and comments for topics in NLP area

Resource

Statistical Machine Translation:

StatMT.org

CVPR 2019 about interpretable model update Feb/02/2919

  1. Exploiting Kernel Sparsity and Entropy for Interpretable CNN Compression

  2. Interpreting CNNs via Decision Trees

    1. related work:
      CNN visualization:
      1.gradient-based 2. up-convolutional nets feature map 3.semanctics of CNN -> object, parts, scenes, texture, materials, color.
      CNN diagnosis:
      1.diagnosis a pre-trained CNN

  3. End-To-End Interpretable Neural Motion Planner

  4. Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks

Neural Architecture Search with Reinforcement Learning ICLR2017 Nov/20/2019

  1. Goal: Using RNN to design CNN and RNN architecture with Reinforcement Learning
  2. Reward is accuracy on test dataset.
  3. using Signmod to choose settings from sets.
  4. Heigh lights 1. variable length and structure. get some novel structures. 2. design structure automatically.

[Natrual language understanding]:

NLU = NLP + IR ?

Named Entity Recognition:

Question Answering

Latest NLP models:

update:Sep.2.2019

ERNIE 2.0: A CONTINUAL PRE-TRAINING FRAMEWORK FOR LANGUAGE UNDERSTANDING

GPT-2

BERT

ELMo

word embedding:

1.word2vec 2.GloVe 3.Fasttext

subword embedding:

  1. Byte Pair Encoding (BPE)
  2. WordPiece
  3. Unigram Language Model

GLUE Tasks:

1.The Corpus of Linguistic Acceptability

Text Generation:

  • Machine Translation

Text Classification:

  • single-sentence classification
  • pairwise text classification
  • pairwise text similarity
  • relevance ranking

SuperGLUE Task：

#ERNIE 2.0: A CONTINUAL PRE-TRAINING FRAMEWORK FOR LANGUAGE UNDERSTANDING