saneshashank/ResearchPapers

curated list of selected research papers for AI/ML/NLP

ResearchPapers

curated list of selected research papers for AI/ML/NLP

Boosting

• XGBoost: http://learningsys.org/papers/LearningSys_2015_paper_32.pdf
• CatBoost: https://arxiv.org/abs/1706.09516
• LightGBM: https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf

NLP:

• Discriminative Vs Generative Classifiers: http://papers.nips.cc/paper/2020-on-discriminative-vs-generative-classifiers-a-comparison-of-logistic-regression-and-naive-bayes.pdf (practical tip: generative (like naive bayes) work better on smaller training data, while for larger training data discriminative work better.)
• Semi CRFs: http://www.cs.cmu.edu/~wcohen/postscript/semiCRF.pdf

Word Embeddings

• Original paper of word2vec by Mikolov et al.: https://arxiv.org/pdf/1301.3781.pdf

• fastText word2vec: https://arxiv.org/pdf/1607.01759.pdf

• GloVe: Global Vectors for Word Representation: https://nlp.stanford.edu/pubs/glove.pdf

• StarSpace Embed All The Things!: https://arxiv.org/pdf/1709.03856.pdf

• SentoVec using SIF (Smooth Inverse Frequency) and Word2Vec: https://openreview.net/pdf?id=SyK00v5xx

• the Universal Language Model Fine-tuning (ULMFiT) approach. The paper introducing it showed that this extra stage of fine-tuning the language model, prior to transfer learning to a classification task, resulted in significantly better predictions:https://arxiv.org/pdf/1801.06146.pdf

• Universal Sentence Encoder: https://arxiv.org/abs/1803.11175

• Entity Embeddings of Categorical Variables:Entity embedding not only reduces memory usage and speeds up neural networks compared with one-hot encoding, but more importantly by mapping similar values close to each other in the embedding space it reveals the intrinsic properties of the categorical variables. : https://arxiv.org/abs/1604.06737

Deep Learning

• Receptive field of single neurons in cat's striate cortex (motivation for CNN):https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1363130/pdf/jphysiol01298-0128.pdf

• AlexNet paper for ImageNet: https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

• VGGNet paper for ImageNet: https://arxiv.org/pdf/1409.1556.pdf

• GoogleNet: https://arxiv.org/pdf/1409.4842.pdf

• ResNet: https://arxiv.org/pdf/1512.03385.pdf

• An Analysis of Deep Learning Models for Practical Applications: https://arxiv.org/pdf/1605.07678.pdf

• Capsule Networks: https://arxiv.org/pdf/1710.09829.pdf

• Road crack detection using deep convolutional neural network: https://www.researchgate.net/publication/305850872_Road_crack_detection_using_deep_convolutional_neural_network

• A DISCIPLINED APPROACH TO NEURAL NETWORK HYPER-PARAMETERS: https://arxiv.org/pdf/1803.09820.pdf

• Delving Deep into Rectifiers:Surpassing Human-Level Performance on ImageNet Classification: https://arxiv.org/pdf/1502.01852.pdf

• Spatial Pyramid Pooling (The new network structure, called SPP-net, can generate a fixed-length representation regardless of image size/scale.): https://arxiv.org/abs/1406.4729

• Self-Attention Generative Adversarial Networks: https://arxiv.org/abs/1805.08318

• Face Aging with Conditional GAN's: https://arxiv.org/pdf/1702.01983.pdf

• Mutual Deep Learning: https://arxiv.org/pdf/1706.00384.pdf

• Discovering Symbolic Models from Deep Learning with Inductive Biases (We develop a general approach to distill symbolic representations of a learned deep model by introducing strong inductive biases.): https://arxiv.org/pdf/2006.11287.pdf

• Symbolic Deep Learning: https://github.com/MilesCranmer/symbolic_deep_learning

Reinforcement Learning

• Double-Q Learning: https://papers.nips.cc/paper/3964-double-q-learning.pdf

• Playing Atari with Deep Reinforcement Learning: https://arxiv.org/pdf/1312.5602.pdf

• Deep RL for Text Summarization: https://arxiv.org/pdf/1810.06667.pdf

GNN

• A Practical Guide to Graph Neural Networks (This tutorial exposes the power and novelty of GNNs to the average deep learning enthusiast by collating and presenting details on the motivations, concepts, mathematics, and applications of the most common types of GNNs.) : https://arxiv.org/pdf/2010.05234.pdf

Auto ML

• Auto sklearn - NIPS paper: https://ml.informatik.uni-freiburg.de/papers/15-NIPS-auto-sklearn-preprint.pdf
• Bayesian Optimizer : https://arxiv.org/pdf/1807.02811.pdf
• Auto-Keras : https://arxiv.org/pdf/1806.10282.pdf

ICLR 2022

• Fast Model Editing at Scale: https://arxiv.org/pdf/2110.11309.pdf
• Trends in ML @ ICLR 2022: https://relational.ai/blog/trends-in-machine-learning-iclr-2022
• Video: https://www.youtube.com/watch?v=k_KTnJsGYS0