[BUG] pose_engine.py runtime exception
Opened this issue · 5 comments
Describe the bug
Pose engine throws intermittent runtime exceptions.
To Reproduce
Run pose engine through a variety of fall images.
Expected behavior
There should be no unexpected runtime exceptions caused by image processing during fall detections.
Host environment (please complete the following information):
- Hardware: Raspberry Pi
- OS: Raspberry OS
- Container: docker
- Version: ambianic edge 1.13.2
Additional context
2021-02-07 13:09:02 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1077.19 ms, 0.93 fps in pipeline area_watch
2021-02-07 13:09:02 ERROR /opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py.process_sample(101): Error "only size-1 arrays can be converted to Python scalars" while processing sample. Dropping sample: {'image': <PIL.JpegImagePlugin.JpegImageFile image mode=RGB size=1280x720 at 0xADB66570>}
Traceback (most recent call last):
File "/opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py", line 85, in process_sample
inference_result, thumbnail = self.fall_detect(image=image)
File "/opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py", line 344, in fall_detect
pose, thumbnail, pose_score, pose_dix = self.find_keypoints(image)
File "/opt/ambianic-edge/src/ambianic/pipeline/ai/fall_detect.py", line 146, in find_keypoints
poses, thumbnail = self._pose_engine.DetectPosesInImage(image)
File "/opt/ambianic-edge/src/ambianic/pipeline/ai/pose_engine.py", line 146, in DetectPosesInImage
kps = self.parse_output(template_heatmaps, template_offsets, 0.3)
File "/opt/ambianic-edge/src/ambianic/pipeline/ai/pose_engine.py", line 89, in parse_output
pose_kps[i, 0] = int(remap_pos[0] + offset_data[max_val_pos[0], max_val_pos[1], i])
TypeError: only size-1 arrays can be converted to Python scalars
2021-02-07 13:09:03 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1070.10 ms, 0.70 fps in pipeline area_watch
2021-02-07 13:09:04 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1003.87 ms, 0.99 fps in pipeline area_watch
2021-02-07 13:09:05 INFO /opt/ambianic-edge/src/ambianic/pipeline/ai/tf_detect.py.log_stats(178): FallDetector inference time 1042.34 ms, 0.96 fps in pipeline area_watch
Can you please provide me with a sample test fall image so I can reproduce the error and fix it?
No, because the error is intermittent and since it is not handled in a way that provides much context, I cannot correlate the message to an image. Please think how we can better encapsulate this error with a contextual information in the log that allows us to track it down.
ok...will check it.
I have tried to trace this issue but unable to locate the bug.
This function use output data(heatmap_data, offset_data) that are generated from TFLite posenet model.
pose_kps[i, 0] = int(remap_pos[0] + offset_data[max_val_pos[0], max_val_pos[1], i])
pose_kps[i, 0] = int(remap_pos[0] + offset_data[max_val_pos[0], max_val_pos[1], i]) TypeError: only size-1 arrays can be converted to Python scalars
And this error is raised when the function expects a single value but you pass an array instead.
I am trying to check at what circumstance model output is different. But not able to find it.
Can you please help me with your thoughts to deal with this bug?
thanks
It's not obvious to me what the issue is. Is it possible that
remap_pos[0] returns anything but a scalar?
Same question for offset_data[max_val_pos[0], max_val_pos[1], i].
One of these two returns a list or array, which confuses int().
I suggest surrounding these lines with an exception handling block that provides as much context as needed. That way the next time this happens, we would know exactly what caused it.