/kaggle-jigsaw-toxicity-classification

Toxicity classification challenge. (14th place)

Primary LanguagePython

kaggle-jigsaw-toxicity-classification

Part of 14th place solution to Jigsaw Unintended Bias in Toxicity Classification Challenge. (slightly modified by refactoring) Our solution is published in here.

  • Public LB: 12th (0.94655)
  • Private LB: 14th (0.94628)

Prerequisite

  • Pull PyTorch image from NVIDIA GPU CLOUD (NGC) 
    docker login nvcr.io
docker image pull nvcr.io/nvidia/pytorch:19.04-py3

Usage

BERT-Base and BERT-Large

# NOTE: Apex ver. 0.1 is already installed in this image
docker container run -it --name=bert --runtime=nvidia --ipc=host -v $PWD:/workspace/jigsaw nvcr.io/nvidia/pytorch:19.04-py3

cd /workspace/jigsaw/src

pip install pytorch_pretrained_bert==0.6.2
pip install fastprogress

# train BERT model
python train_bert_base_full.py
python train_bert_large_full.py

GPT-2

docker container run -it --name=gpt2 --runtime=nvidia --ipc=host -v $PWD:/workspace/jigsaw nvcr.io/nvidia/pytorch:19.04-py3

cd /workspace/jigsaw/src

pip install git+https://github.com/pronkinnikita/pytorch-pretrained-BERT
pip install fastprogress

# train GPT-2 model
python train_gpt2_full.py

Author

Masaaki Hirotsu / Kaggle: @mhiro2