Small faces and keypoint_threshold, face_threshold and replace_tight_bbox

[IbrahimSobh]

Dears

Regarding keypoint_threshold and face_threshold, could you please explain in some detial how they work and affect the anonymization.

anonymizer = deep_privacy_anonymizer.DeepPrivacyAnonymizer(generator,
                                                           batch_size=32,
                                                           use_static_z=True,
                                                           keypoint_threshold=.1,
                                                           face_threshold=.6)

For example, I noticed that making face_threshold too small, encourage the network to predict anything as face. and making it too large, leads to missing faces. How this will work with the keypoint_threshold?

Additionally, what is replace_tight_bbox used for?

Finally, kindly share any tips to enable detection and anonymization of small faces.

Thank you and best regards

[hukkelas]

To explain the parameters:

• face_threshold: This is the confidence threshold for the face detection model we use. If you set it very low, it's going to accept a lot of objects as a face. We found 0.6 works fine.
• keypoint_threshold is the confidence threshold for the face keypoint detector (this is running in parallel with face detection). We decided to set this to a low value since our keypoint detector is struggling in cases where we have a low resolution face.
• replace_tight_bbox: This is used for when combining our anonymized face with the original image. Setting this to true will only replace the areas inside the bounding box of the face. If you set it to false, it will replace the area inside the bounding box of the face AND additional area around (If you look at this picture, replace tight bbox to True will only replace areas within the red bounding box.)

Finally, kindly share any tips to enable detection and anonymization of small faces.

It depends on how small resolution you are talking about. Most likely, the problem is the keypoint detector.
A fix for this in cases of very small faces (lets say smaller than 16x16 resolution), is to either not use pose information, or just set the face keypoints to a fixed location in the image.
Also, I recommend you to use the following hyperparameters for small faces: face_threshold=0.3, and keypoint_threshold=0.05

[IbrahimSobh]

Thank you very much