/InsightFace_TF

Insight Face on TensorFlow

Primary LanguagePython

Insight Face in TensorFlow

Tasks

  • mxnet dataset to tfrecords
  • backbone network architectures [vgg16, vgg19, resnet]
  • backbone network architectures [resnet-se, resnext]
  • LResNet50E-IR
  • LResNet100E-IR
  • Additive Angular Margin Loss
  • CosineFace Loss
  • train network code
  • add validate during training
  • multi-gpu training
  • evaluate code

Training Logs

epoch 0, total_step 20, total loss is 107.34 , inference loss is 80.60, weight deacy loss is 26.74, training accuracy is 0.000000, time 38.373 samples/sec
epoch 0, total_step 40, total loss is 109.65 , inference loss is 77.31, weight deacy loss is 32.34, training accuracy is 0.000000, time 38.281 samples/sec
epoch 0, total_step 60, total loss is 114.86 , inference loss is 82.29, weight deacy loss is 32.57, training accuracy is 0.000000, time 37.687 samples/sec
epoch 0, total_step 80, total loss is 104.92 , inference loss is 72.77, weight deacy loss is 32.15, training accuracy is 0.000000, time 38.402 samples/sec
epoch 0, total_step 100, total loss is 101.66 , inference loss is 69.99, weight deacy loss is 31.67, training accuracy is 0.000000, time 38.235 samples/sec
epoch 0, total_step 120, total loss is 101.70 , inference loss is 70.54, weight deacy loss is 31.16, training accuracy is 0.000000, time 37.822 samples/sec
epoch 0, total_step 140, total loss is 102.23 , inference loss is 71.61, weight deacy loss is 30.63, training accuracy is 0.000000, time 38.308 samples/sec
epoch 0, total_step 160, total loss is 103.26 , inference loss is 73.17, weight deacy loss is 30.08, training accuracy is 0.000000, time 38.054 samples/sec
epoch 0, total_step 180, total loss is 98.61 , inference loss is 69.07, weight deacy loss is 29.54, training accuracy is 0.000000, time 38.198 samples/sec
epoch 0, total_step 200, total loss is 95.20 , inference loss is 66.16, weight deacy loss is 29.04, training accuracy is 0.000000, time 38.217 samples/sec

Training Tips(Continual updates)

  • If you can't use large batch size(>128), you should use small learning rate
  • If you can't use large batch size(>128), you can try batch renormalization(file L_Resnet_E_IR_RBN.py)
  • If use multiple gpus, you should keep at least 16 images each gpu.
  • Try Group Normalization, you can use the code L_Resnet_E_IR_GBN.py

Training models (Continual updates)

model A
model name depth normalization layer batch size total_steps download
model A 50 group normalization 16 1060k model a
accuracy
dbname accuracy
lfw 0.9897
cfp_ff 0.9876
cfp_fp 0.84357
age_db30 0.914
model B
model name depth normalization layer batch size total_steps download
model B 50 batch normalization 16 1100k model_b
accuracy
dbname accuracy
lfw 0.9933
cfp_ff 0.99357
cfp_fp 0.8766
age_db30 0.9342

Requirements

  1. TensorFlow 1.4 1.6
  2. TensorLayer 1.7
  3. cuda8&cudnn6 or cuda9&cudnn7
  4. Python3

Max Batch Size Test

Environment
GPU cuda cudnn TensorFlow TensorLayer Maxnet Gluon
Titan xp 9.0 7.0 1.6 1.7 1.1.0 1.1.0
Results
DL Tools Max BatchSize(without bn and prelu) Max BatchSize(with bn only) Max BatchSize(with prelu only) Max BatchSize(with bn and prelu)
TensorLayer (8000, 9000) (5000, 6000) (3000, 4000) (2000, 3000)
Mxnet (40000, 50000) (20000, 30000) (20000, 30000) (10000, 20000)
Gluon (7000, 8000) (3000, 4000) no official method None

(8000, 9000) : 8000 without OOM, 9000 OOM Error

Test Code
TensorLayer Maxnet Gluon
tensorlayer_batchsize_test.py mxnet_batchsize_test.py gluon_batchsize_test.py

pretrained model download link

References

  1. InsightFace mxnet
  2. InsightFace : Additive Angular Margin Loss for Deep Face Recognition
  3. Group Normalization
  4. tensorlayer_vgg16
  5. tensorlayer_vgg19
  6. tf_slim
  7. Deep Residual Learning for Image Recognition
  8. Very Deep Convolutional Networks For Large-Scale Image Recognition
  9. Squeeze-and-Excitation Networks