ambianic/fall-detection

Optimize CPU usage - rotate image only if there is a standing up pose in the frame buffer

ivelin opened this issue · 2 comments

ivelin commented

Looking at the ambianic edge logs in a real world usage, there is a constant stream of attempts to detect a pose in the original image and +/-90' rotations. This happens because most of the time there is no person in the camera view at all.

This is a suboptimal 3x use of CPU. Normally a single posenet pass on Raspberry Pi takes about 300ms (3fps). However after 2 rotations the total inference time goes up to 1-1.2sec (0.8-1fps). See log excerpt below:

ambianic-edge    | 2021-02-04 01:17:45 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1015.96 ms, 0.98 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:46 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1094.21 ms, 0.91 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:47 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1069.70 ms, 0.93 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:48 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1163.64 ms, 0.86 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:49 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1024.56 ms, 0.97 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:50 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1090.12 ms, 0.92 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:51 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1089.76 ms, 0.92 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:52 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1223.01 ms, 0.82 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:54 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1116.52 ms, 0.89 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:55 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1173.54 ms, 0.85 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:56 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1366.06 ms, 0.73 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:57 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1145.92 ms, 0.87 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:58 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1139.79 ms, 0.87 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:17:59 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1051.62 ms, 0.95 fps in pipeline area_watch

ambianic-edge    | 2021-02-04 01:18:01 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1107.74 ms, 0.90 fps in pipeline area_watch
ivelin commented

Thinking through this a bit, it seems like it only makes sense to try rotations if there is a buffered image frame with a standing pose.

The rotations are essentially there to look for people on the ground. It's a workaround to overcome the posenet+mobilnetv1 weakness in detecting non-vertical human poses.

However detecting a horizontal pose is only helpful if there is a previously saved vertical pose to compare it to.

Therefore I suggest implementing the following optimization that should bring the CPU usage down significantly.

# use to determine if there is a standing pose in a given frame
def standing_pose(sping_vector):
   return true if abs(angle_betwen(vertical_axis, spine_vector)) <= 90-fall_threshold_angle

# in the find_keypoints function replace the rotation pre-condition
# https://github.com/ambianic/ambianic-edge/blob/b55b4474ea718945970efb5e5da48587cc1f12d4/src/ambianic/pipeline/ai/fall_detect.py#L153

if pose_score < min_score:
  standing_pose_in_buffer = filter (lambda prev_frame: prev_frame.is_standing_pose, prev_frames)
  if standing_pose_in_buffer: 
    while pose_score < min_score and rotations:
  ...

This would drop CPU usage significantly (almost 60%), because rotations will be only attempted if there is a person detected in front of the camera and shortly after it is not detected which means that they are either not standing up or they are not visible.

Thoughts?

ivelin commented

@bhavikapanara please take a look and share your comments on this optimization idea.