Our team ranked 83rd place (TOP 10%) in the Recursion Cellular Image Classification 2019 on Kaggle platform. This repository consists of code and configs that we used to train our best models. The solution is powered by awesome PyTorch, Ignite, Cadene models, EfficientNet and Albumentations libraries.
models- folder with our models (EfficientNet-B2, EfficientNet-B5, DenseNet201, SeResNext101_32x4d)utils- folder with helper scripts
From the beginning, efficientnet outperformed other models. Using allowed to use bigger batch size - speeded up training and inference. Other models (like ResNet, ResNext or Inception) worked worse for us.
Models used in the final submission:
- EfficientNet-B5 (best single model): AMSoftmaxLoss(margin_type='cos'), RAdam, ExponentialLR, freezing all layers without last linear on 1 epoch
- DenseNet201 (similar score with effnet-b5): AMSoftmaxLoss(margin_type='cos'), Adam, ExponentialLR, freezing all layers without last linear on 1 epoch
- EfficientNet-B2: CrossEntropyLoss, Adam, ExponentialLR, without freezing
- SeResNext101_32x4d: AMSoftmaxLoss(margin_type='cos'), RAdam, OneCycleLR, without freezing, fastai library
From Albumentations library: We tried Horizontalflip, VerticalFlip, RandomRotate90, ShiftScaleRotate, Cutout, RandomBrightnessContrast but models with augmentations were not included in the final solution.
First 3 models were trained using Ignite library and the last one with FastAi, all of them work on top of PyTorch.
Each model trained on 6 channel images using the entire dataset and them finetuning for each cell type. Adam/RAdam were used for training as optimizers and ExponentialLR/OneCycleLR as loss functions.
We used 1x* Tesla V40 (8gb), Kaggle Kernels, sometimes Gcloud SDK (8x* Tesla v100 (16gb) or 4x* Tesla v4 (16gb) with fp16)