/Lifelong-Learning

Lifelong-Learning

Literuture Review

Continual Lifelong Learning with Neural Networks:A Review

Regularization Approaches

  • Overcoming catastrophic forgetting in neural networks (EWC) (PNAS2017) [paper] [code] [code]
  • Continual Learning Through Synaptic Intelligence (ICML2017) [paper] [code]

Memorey-based Approaches

  • iCaRL: Incremental Classifier and Representation Learning (CVPR2017) [paper] [code]
  • Gradient Episodic Memory for Continual Learning (NIPS2017) [paper] [code]
  • Memory Aware Synapses: Learning what (not) to forget (ECCV2018) [paper] [code]
  • Efficient Lifelong Learning with A-GEM (ICLR2019) [paper] [code]

Architecture Approaches

  • Progressive Neural Networks [paper]
  • Lifelong Learning with Dynamically Expandable Networks (ICLR2018) [paper]
  • Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting (ICML2019) [paper]

Incremental/Lifelong Learning Metrics

  • Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence (ECCV2018)[paper]
  • Learning a Unified Classifier Incrementally via Rebalancing (CVPR2019) [paper]

Network Pruning

  • TensorFlow Model Optimization Toolkit — Pruning API [Source][code]

  • The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (ICLR2019)[paper]

  • Sparse Transfer Learning via Winning LotteryTickets (CoRR2019)[paper]