Codes for some projects
thrift -gen py sample.thrift
mv -vi ./gen-py/sample .
protoc --python_out=. ./sample.
python write_pb2.py address_book.pb
python server.py
python client.py
[1] Thrift Tutorial: https://thrift-tutorial.readthedocs.io/en/latest/usage-example.html
[2] Protocol Buffer Tutorial: https://developers.google.com/protocol-buffers/docs/pythontutorial
Bert-Base: 0.714
, Bert-Large-WWM: 0.712
, RoBERTa-Base: 0.713
, RoBERTs-Base-SQuAD2: 0.711
, RoBERTa-Large-MNLI: 0.715
python vocab_dl.py
python prepare.py -f
python train_model.py bert-base-uncased --finetune --lr 1.2e-4
python train_model.py bert-large-uncased-whole-word-masking-finetuned-squad --finetune --lr 1e-4
python train_model.py roberta-base --finetune --lr 1.4e-4
[1] Kaggle Twitter Sentiment Extraction Competition: https://www.kaggle.com/c/tweet-sentiment-extraction
[2] Rank 1 Solution: Post, Notebook 1, Notebook 2
[3] Other Solution: Dataset Preprocessing Magic
[4] BERT Fine Tune: Pretrained Model Inventory, Overview, Constructing Auxiliary Sentence
[5] Hugging Face Tokenizer: Overview, Quick Start
[6] Multi-Sample Dropout for Accelerated Training and Better Generalization: Paper
Use PyTorch Ligntning framework to fine tune a pretrained ResNet34
model, achieved 99.17%
accuracy on test dataset.
. perequisite.sh
python prepare.py -r 0.8 -s 1898
python train_model.py --freeze --freezelr 1e-3 --gpus 1
python train_model.py --findlr --ckfile {checkpoint file name} --gpus 1
python train_model.py --finetune --ckfile {checkpoint file name} --finetunelr 1e-4 --gpus 1
python train_model.py --test --ckfile {checkpoint file name}
[1] PyTorch Lightning Documentation: https://pytorch-lightning.readthedocs.io/en/latest/
[2] Finetune Torchvision Model: https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html
[3] Cyclical Learning Rates for Training Neural Networks: https://arxiv.org/pdf/1506.01186.pdf