skrashevich/double-take

[FEAT] New snapshot request

rhamblen opened this issue · 5 comments

rhamblen commented

Is your feature request related to a problem? Please describe.
Sometimes we dont get matches to the snapshot because people are not looking at the camera, a few frames forward and the picture might be much better.

Describe the solution you'd like
WIthin Compreface we can use the pose, to determine if someone is looking at the camera or not.
Could we use the pose values, to send a request back to frigate, to ask for a new snapshop, when the match is outside of a specific threshold.

Additional context
recognise this may actually be a frigate issue, and we need to delay the frame for recognition from frigate.

https://exadel.com/news/compreface-new-release/
Here is how you can optimize the system:

  1. Use a face detection service instead of the face recognition service on the frames. Enable the Pose Plugin by setting the face_plugins=pose parameter.
  2. Find the best frame by defining:
  • If the person is near the camera using the square of the bounding box, by analyzing the box response field
  • If the person looks at the camera, by analyzing the pose response field
  1. Use the box response field to crop the best frame face
  2. Send it to the recognition endpoint with the detect_faces=false parameter
skrashevich commented

Great idea.

Can you advise the calculation formula, whether a person is looking at the camera or not, using data from pose plugin (pitch, roll, yaw)?

github-actions commented

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

rhamblen commented 
"pose": {
    "pitch": 12.694842967926093,
    "roll": -5.424399730251082,
    "yaw": -19.913121913104177
  },
  
  When looking at the "pose": information.
  It appears that 
  pitch is the angle of the head nose up or down
  roll is the angle of the head tilting (think sholders up/down so head rotates clockwise/anti clockwise)
  yaw is the head rotating towards or away from the front, think facing forward or behind.
  
  From what I can tell yaw would be the most significant value, and maybe it is about being within +/- 30
  Pitch can be positive, but large negative is looking at the ground. 
  Roll makes no difference as its just someone tilting their head.
github-actions commented

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions commented

This issue was closed because it has been stalled for 5 days with no activity.