Having difficulties on selecting bound box for shape_predictor
Closed this issue · 7 comments
The task is to find 68 landmarks (shape_predictor_68_face_landmarks.dat) of the face on the image on iOS.
We've used the methods above:
- Using dlib hog detector + shape_predictor gives good quality but very slow in part of the detector. (Might be issued with lib compilation, but we've haven't found slowness root yet)
- Using iOS detector + shape_predictor gives good quality and speed, but on some images doesn't find landmarks, for similarity good face detection. (Almost all bad examples works well in p.1) Landmarking does not work properly similarly on iOS in Python, for the iOS detector bounding box.
iOS detector:
dlib detector:
In our research, we've found that shape_predictor quality tightly connected with bounding box position and size. If we move the hog detector for 5-6% shape_predictor doesn't work as expected. The same thing with the size of the bounding box, for some cases it works well, for some gives bad results. Here #1290 where was a proposal to use all images as the bounding box, but it doesn't work at all, producing false results.
The questions are followed:
- What is the connection with a bounding box and shape_predictor quality, why small changes in the box give false results?
- What is the normal time to hog detector on iOS? We've got 3-8 seconds for 800x1200 images with 1 face. It seems slow but we have no benchmark. We've tried to use -O3 and enabling NEON but with no effect on speed
- Version: 19.19
- Where did you get dlib: github repo
- Platform: iOS /
- Compiler: XCode
Just my 2 cents: I think you need to retrain a new shape predictor. You just need to run the iOS face detector on the dataset that was used to train the shape predictor for face landmarks, replace the current bounding boxes with the iOS ones and retrain.
I guess that you can also find some consistency between dlib bounding boxes and iOS, so maybe you can come up an affine transformation that converts iOS bounding boxes to dlib bounding boxes.
Yes, what @arrufat says is the best solution. In particular, I would just retrain with the boxes you want to use.
@arrufat @davisking Thanks for your input. We've started from retraining with bigger jitter, and got a pretty good result on many photos. But strangely on some photos results are highly sensitive to bounding box data:
dlib.rectangle(719, 560, 1666, 1480) gives
dlib.rectangle(719, 560, 1666, 1481)
Is it normal or should we use other settings for retraining? Which settings can lead to such results?
You fundamentally need to train using the same kind of data you will use when you really run the model. So whatever process you use to generate bounding boxes needs to be the same for training data generation and for model use after training. If it's not the same then it won't work and this is what happens.
Warning: this issue has been inactive for 35 days and will be automatically closed on 2020-08-12 if there is no further activity.
If you are waiting for a response but haven't received one it's possible your question is somehow inappropriate. E.g. it is off topic, you didn't follow the issue submission instructions, or your question is easily answerable by reading the FAQ, dlib's official compilation instructions, dlib's API documentation, or a Google search.
Warning: this issue has been inactive for 42 days and will be automatically closed on 2020-08-12 if there is no further activity.
If you are waiting for a response but haven't received one it's possible your question is somehow inappropriate. E.g. it is off topic, you didn't follow the issue submission instructions, or your question is easily answerable by reading the FAQ, dlib's official compilation instructions, dlib's API documentation, or a Google search.
Notice: this issue has been closed because it has been inactive for 45 days. You may reopen this issue if it has been closed in error.