macabdul9/papers

This repository contains the papers that I read with highlighted text.

Papers

This repository contains list of the high impact that I have read, currently reading and intend to read.

0. Transformers

• Done

  • Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention (Aug 2020)
  • Big Bird: Transformers for Longer Sequences (July 2020)
  • Multi-Head Attention: Collaborate Instead of Concatenate (June 2020)
  • Dialogue Transformers (May 2020)
  • Efficient Attention: Attention with Linear Complexities (Jan 2020)
  • Attention Is All You Need (June 2017)
  • NEURAL MACHINE TRANSLATION BY JOINTLY LEARNING TO ALIGN AND TRANSLATE (May 2016)

• To Do

  • PAY ATTENTION WHEN REQUIRED (Sep 2019)
  • PAY ATTENTION WHEN REQUIRED (Sep 2019)
  • Are Sixteen Heads Really Better than One? (May 2019)

1. Domain Adaptation

• Done

  • SENTIX: A Sentiment-Aware Pre-Trained Model forCross-Domain Sentiment Analysis (Nov 2020)
  • Transformer Based Multi-Source Domain Adaptation (Nov 2020)
  • Multi-Source Domain Adaptation with Mixture of Experts (Nov 2018)
  • Neural Unsupervised Domain Adaptation in NLP—A Survey (Oct 2020)
  • Domain Divergences: a Survey and Empirical Analysis (Oct 2020)
  • Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks (July 2020)
  • TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP (Dec 2019)
  • To Annotate or Not? Predicting Performance Drop under Domain Shift (Nov 2019)
  • What does BERT learn about the structure of language? (Aug 2019)
  • Domain-Adversarial Training of Neural Networks (May 2015)
  • Learning Transferable Features with Deep Adaptation Networks (May 2015)

• To Do

  • Evaluating Lottery Tickets Under Distributional Shifts (Nov 2019)
  • Investigating Transferability in Pretrained Language Models (Nov 2020)
  • KinGDOM: Knowledge-Guided DOMain adaptation for sentiment analysis (May 2020)
  • Pretrained Transformers Improve Out-of-Distribution Robustness (April 2020)
  • What Happens To BERT Embeddings During Fine-tuning? (April 2020)
  • CyCADA: Cycle-Consistent Adversarial Domain Adaptation (Nov 2017)
  • How Transferable are Neural Networks in NLP Applications? (Mar 2019)

2. Pretraining, Transfer Learning, Representation Learning

• Done

  • Distilling Knowledge Learned in BERT for Text Generation (Nov 2019)
  • DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter (Oct 2019)
  • Towards Language Agnostic Universal Representations (Sep 2019)
  • What does BERT learn about the structure of language? (Aug 2019)
  • RoBERTa: A Robustly Optimized BERT Pretraining Approach (July 2019)
  • XLNet: Generalized Autoregressive Pretraining for Language Understanding (June 2019)
  • Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned (May 2019)
  • BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (Oct 2018)
  • Improving Language Understandingby Generative Pre-Training

• To Do

  • Do Massively Pretrained Language Models Make Better Storytellers? (Sep 2019)
  • GPT2: Language Models are Unsupervised Multitask Learners (2019)

3. Conversation AI

3.1 Dialogue Act, Intent and Emotion

• Done

  • Towards Emotion-aided Multi-modal Dialogue Act Classification (July 2020)
  • Contextual Dialogue Act Classification for Open-Domain Conversational Agents (May 2020)
  • An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction (Sep 2019)
  • Dialogue Act Classification with Context-Aware Self-Attention (May 2019)
  • BERT for Joint Intent Classification and Slot Filling (Feb 2019)
  • Context-Aware Self-Attention Networks (Feb 2019)
  • A Context-based Approach for Dialogue Act Recognition using Simple Recurrent Neural Networks (May 2018)
  • Utterance Intent Classification of a Spoken Dialogue System with Efficiently Untied Recursive Autoencoders (Aug 2017)

• To Do

  • Modeling Dialogue in Conversational Cognitive Health Screening Interviews (May 2020)

3.2 Dialogue Generation

• Done

  • Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversations (Aug 2020)
  • DIALOGPT : Large-Scale Generative Pre-trainingfor Conversational Response Generation (May 2020)
  • A Simple Language Model for Task-Oriented Dialogue (May 2020)
  • CAiRE: An Empathetic Neural Chatbot (April 2020)
  • Neural Assistant: Joint Action Prediction, Response Generation, and Latent Knowledge Reasoning (Oct 2019)
  • Do Neural Dialog Systems Use the Conversation History Effectively? An Empirical Study (July 2019)
  • What makes a good conversation? How controllable attributes affect human judgments (April 2019)
  • Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset (Nov 2018)
  • Toward Continual Learning for Conversational Agents (Jan 2018)
  • Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial (May 2017)
  • Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models (April 2016)
  • A Neural Conversational Model (Jul 2015)

• To Do

  • PONE: A Novel Automatic Evaluation Metric for Open-Domain Generative Dialogue Systems (April 2020)
  • Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial (2017)

4. Summarization, Translation Paraphrasing and Other Seq2Seq

• Done

  • Unsupervised Paraphrasing by Simulated Annealing (July 2020)
  • Pretraining-Based Natural Language Generation for Text Summarization (Nov 2019)
  • MASS: Masked Sequence to Sequence Pre-training for Language Generation (May 2019)

• To Do

  • Very Deep Transformers for Neural Machine Translation (Aug 2020)

5. Miscellaneous

• Done

  • When BERT Plays the Lottery, All Tickets Are Winning (Nov 2020)
  • Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask (Dec 2019)
  • The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (May 2019)
  • An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Oct 2020)
  • Are Transformers universal approximators of sequence-to-sequence functions? (Dec 2019)

• To Do

  • Beyond Accuracy: Behavioral Testing of NLP Models with CheckList (May 2020)
  • Are distributional representations ready for the real world? Evaluating word vectors for grounded perceptual meaning (May 2017)
  • Understanding Attention for Text Classification (July 2020)