legolas140/competitive-data-science

How to Win a Data Science Competition: Learn from Top Kagglers

Recap of main ML algorithms

• Explanation of Random Forest
• Explanation/Demonstration of Gradient Boosting
• Example of kNN

Additional Tools

• Example from sklearn with different decision surfaces
• Arbitrary order factorization machines
• Blog "datas-frame" (contains posts about effective Pandas usage)

Feature preprocessing and generation with respect to models

Feature preprocessing

• Preprocessing in Sklearn
• Andrew NG about gradient descent and feature scaling
• Feature Scaling and the effect of standardization for machine learning algorithms

Feature generation

• Discover Feature Engineering, How to Engineer Features and How to Get Good at It
• Discussion of feature engineering on Quora

Feature extraction from text and images

Bag of words

• Feature extraction from text with Sklearn
• More examples of using Sklearn

Word2vec

• Tutorial to Word2vec
• Tutorial to word2vec usage
• Text Classification With Word2Vec
• Introduction to Word Embedding Models with Word2Vec

NLP Libraties

• NLTK, spaCy, TextBlob

Pretrained models

• Using pretrained models in Keras
• Image classification with a pre-trained deep neural network

Finetuning

• How to Retrain Inception's Final Layer for New Categories in Tensorflow
• Fine-tuning Deep Learning Models in Keras

Exploratory data analysis

• Biclustering algorithms for sorting corrplots

Validation

• Validation in Sklearn
• Advices on validation in a competition

Data leakage

• Perfect score script by Oleg Trott - used to probe leaderboard
• Page about data leakages on Kaggle

Metrics optimization

Classification

• Evaluation Metrics for Classification Problems: Quick Examples + References
• Decision Trees: “Gini” vs. “Entropy” criteria
• Understanding ROC curves

Ranking

• Learning to Rank using Gradient Descent - original paper about pairwise method for AUC optimization
• Overview of further developments of RankNet
• RankLib (implemtations for the 2 papers from above)
• Learning to Rank Overview

Clustering

• Evaluation metrics for clustering

Hyperparameter tuning

• Tuning the hyper-parameters of an estimator (sklearn)
• Optimizing hyperparameters with hyperopt
• Complete Guide to Parameter Tuning in Gradient Boosting (GBM) in Python

Tips and tricks

• Far0n's framework for Kaggle competitions "kaggletils"
• https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/

Advanced features

Matrix Factorization:

• Overview of Matrix Decomposition methods (sklearn)

t-SNE:

• Multicore t-SNE implementation
• Comparison of Manifold Learning methods (sklearn)
• How to Use t-SNE Effectively (distill.pub blog)
• tSNE homepage (Laurens van der Maaten)
• Example: tSNE with different perplexities (sklearn)

Interactions:

• Facebook Research's paper about extracting categorical features from trees
• Example: Feature transformations with ensembles of trees (sklearn)

Ensembling

• Kaggle ensembling guide at MLWave.com (overview of approaches)
• StackNet — a computational, scalable and analytical meta modelling framework (by KazAnova)
• Heamy — a set of useful tools for competitive data science (including ensembling)