[BUG] Don't understand the `frigate.attempts.mqtt` setting
Closed this issue · 15 comments
Describe the bug
This is more of a config question than a bug. I was wondering if frgiate publishes the actual image to MQTT and DT picks it up. If true, I was trying to set frigate.attempts.mqtt to true and setting the snapshot and latest settings to 0 would mean that DT will take the image published to MQTT and not try to download it from frigate. I am not even sure I understand the functionality but was just playing around with settings. I think DT is reporting that it can't even find a person whereas Frigate using AI Server found the person and published the event. I even removed the min_area setting without luck. However, if I set attempts.latest and attempts.snapshot to default or anything other than 0, it works. So not sure what is going on here. IIRC, I remember seeing only MQTT labels in the UI thumbnail list in the previous versions but now none of them are MQTT labels.
If this is not how the integration is supposed to work, then we can ignore this and move on.
Here is my config:
---
auth: true
mqtt:
host: m
username: USERNAME
password: PASSWORD
frigate:
url: http://f:5000
update_sub_labels: true
min_area: 4096
labels:
- person
image:
height: 720
attempts:
latest: 0
snapshot: 0
mqtt: true
delay: 2
logs:
level: silly
detectors:
deepstack:
url: http://ds:5000
timeout: 15
opencv_face_required: true
aiserver:
url: http://cpai:5000
timeout: 60
opencv_face_required: true
Version of Double Take
1.13.11.8
Expected behavior
maybe pick up the image from MQTT and process it?
Screenshots
Here is my log output:
home_dt.1.m308n5oqkwkk@potter | verbose: Incoming event from frigate: {"before": {"id": "1700787361.207806-4z62ld", "camera": "garage", "frame_time": 1700787361.207806, "snapshot": null, "label": "person", "sub_label": null, "top_score": 0.0, "false_positive": true, "start_time": 1700787361.207806, "end_time": null, "score": 0.92138671875, "box": [869, 412, 1086, 950], "area": 116746, "ratio": 0.4033457249070632, "region": [681, 407, 1261, 987], "stationary": false, "motionless_count": 0, "position_changes": 0, "current_zones": [], "entered_zones": [], "has_clip": false, "has_snapshot": false, "attributes": {}, "current_attributes": []}, "after": {"id": "1700787361.207806-4z62ld", "camera": "garage", "frame_time": 1700787361.331878, "snapshot": {"frame_time": 1700787361.331878, "box": [880, 397, 1093, 941], "area": 115872, "region": [683, 390, 1267, 974], "score": 0.92578125, "attributes": []}, "label": "person", "sub_label": null, "top_score": 0.923583984375, "false_positive": false, "start_time": 1700787361.207806, "end_time": null, "score": 0.92578125, "box": [880, 397, 1093, 941], "area": 115872, "ratio": 0.3915441176470588, "region": [683, 390, 1267, 974], "stationary": false, "motionless_count": 0, "position_changes": 1, "current_zones": [], "entered_zones": [], "has_clip": true, "has_snapshot": false, "attributes": {}, "current_attributes": []}, "type": "new"}
home_dt.1.m308n5oqkwkk@potter | info: processing garage: 1700787361.207806-4z62ld
home_dt.1.m308n5oqkwkk@potter | info: done processing garage: 1700787361.207806-4z62ld in 0 sec
home_dt.1.m308n5oqkwkk@potter | info: {
home_dt.1.m308n5oqkwkk@potter | id: '1700787361.207806-4z62ld',
home_dt.1.m308n5oqkwkk@potter | duration: 0,
home_dt.1.m308n5oqkwkk@potter | timestamp: '2023-11-24T00:56:01.720Z',
home_dt.1.m308n5oqkwkk@potter | attempts: 0,
home_dt.1.m308n5oqkwkk@potter | camera: 'garage',
home_dt.1.m308n5oqkwkk@potter | zones: [],
home_dt.1.m308n5oqkwkk@potter | counts: { person: 0, match: 0, miss: 0, unknown: 0 },
home_dt.1.m308n5oqkwkk@potter | matches: [],
home_dt.1.m308n5oqkwkk@potter | misses: [],
home_dt.1.m308n5oqkwkk@potter | unknowns: [],
home_dt.1.m308n5oqkwkk@potter | token: '********'
home_dt.1.m308n5oqkwkk@potter | }
home_dt.1.m308n5oqkwkk@potter | verbose: Event type: frigate
home_dt.1.m308n5oqkwkk@potter | verbose: FRIGATE.URL: http://f:5000; FRIGATE.UPDATE_SUB_LABELS: true; best.length: 0
home_dt.1.m308n5oqkwkk@potter | verbose: Incoming event from frigate: {"before": {"id": "1700787361.207806-4z62ld", "camera": "garage", "frame_time": 1700787361.331878, "snapshot": {"frame_time": 1700787361.331878, "box": [880, 397, 1093, 941], "area": 115872, "region": [683, 390, 1267, 974], "score": 0.92578125, "attributes": []}, "label": "person", "sub_label": null, "top_score": 0.923583984375, "false_positive": false, "start_time": 1700787361.207806, "end_time": null, "score": 0.92578125, "box": [880, 397, 1093, 941], "area": 115872, "ratio": 0.3915441176470588, "region": [683, 390, 1267, 974], "stationary": false, "motionless_count": 0, "position_changes": 1, "current_zones": [], "entered_zones": [], "has_clip": true, "has_snapshot": false, "attributes": {}, "current_attributes": []}, "after": {"id": "1700787361.207806-4z62ld", "camera": "garage", "frame_time": 1700787361.53252, "snapshot": {"frame_time": 1700787361.331878, "box": [880, 397, 1093, 941], "area": 115872, "region": [683, 390, 1267, 974], "score": 0.92578125, "attributes": []}, "label": "person", "sub_label": null, "top_score": 0.923583984375, "false_positive": false, "start_time": 1700787361.207806, "end_time": null, "score": 0.86376953125, "box": [923, 424, 1126, 900], "area": 96628, "ratio": 0.4264705882352941, "region": [704, 370, 1300, 966], "stationary": false, "motionless_count": 3, "position_changes": 1, "current_zones": ["driveway"], "entered_zones": ["driveway"], "has_clip": true, "has_snapshot": true, "attributes": {}, "current_attributes": []}, "type": "update"}
home_dt.1.m308n5oqkwkk@potter | info: processing garage: 1700787361.207806-4z62ld
home_dt.1.m308n5oqkwkk@potter | info: done processing garage: 1700787361.207806-4z62ld in 0 sec
home_dt.1.m308n5oqkwkk@potter | info: {
home_dt.1.m308n5oqkwkk@potter | id: '1700787361.207806-4z62ld',
home_dt.1.m308n5oqkwkk@potter | duration: 0,
home_dt.1.m308n5oqkwkk@potter | timestamp: '2023-11-24T00:56:01.991Z',
home_dt.1.m308n5oqkwkk@potter | attempts: 0,
home_dt.1.m308n5oqkwkk@potter | camera: 'garage',
home_dt.1.m308n5oqkwkk@potter | zones: [ 'driveway' ],
home_dt.1.m308n5oqkwkk@potter | counts: { person: 0, match: 0, miss: 0, unknown: 0 },
home_dt.1.m308n5oqkwkk@potter | matches: [],
home_dt.1.m308n5oqkwkk@potter | misses: [],
home_dt.1.m308n5oqkwkk@potter | unknowns: [],
home_dt.1.m308n5oqkwkk@potter | token: '********'
home_dt.1.m308n5oqkwkk@potter | }
home_dt.1.m308n5oqkwkk@potter | verbose: Event type: frigate
home_dt.1.m308n5oqkwkk@potter | verbose: FRIGATE.URL: http://f:5000; FRIGATE.UPDATE_SUB_LABELS: true; best.length: 0
home_dt.1.m308n5oqkwkk@potter | verbose: Incoming event from frigate: {"before": {"id": "1700787361.207806-4z62ld", "camera": "garage", "frame_time": 1700787361.53252, "snapshot": {"frame_time": 1700787361.331878, "box": [880, 397, 1093, 941], "area": 115872, "region": [683, 390, 1267, 974], "score": 0.92578125, "attributes": []}, "label": "person", "sub_label": null, "top_score": 0.923583984375, "false_positive": false, "start_time": 1700787361.207806, "end_time": null, "score": 0.86376953125, "box": [923, 424, 1126, 900], "area": 96628, "ratio": 0.4264705882352941, "region": [704, 370, 1300, 966], "stationary": false, "motionless_count": 3, "position_changes": 1, "current_zones": ["driveway"], "entered_zones": ["driveway"], "has_clip": true, "has_snapshot": true, "attributes": {}, "current_attributes": []}, "after": {"id": "1700787361.207806-4z62ld", "camera": "garage", "frame_time": 1700787365.281423, "snapshot": {"frame_time": 1700787361.331878, "box": [880, 397, 1093, 941], "area": 115872, "region": [683, 390, 1267, 974], "score": 0.92578125, "attributes": []}, "label": "person", "sub_label": null, "top_score": 0.923583984375, "false_positive": false, "start_time": 1700787361.207806, "end_time": 1700787369.686066, "score": 0.748046875, "box": [1108, 297, 1186, 549], "area": 19656, "ratio": 0.30952380952380953, "region": [985, 267, 1305, 587], "stationary": false, "motionless_count": 2, "position_changes": 1, "current_zones": ["driveway"], "entered_zones": ["driveway"], "has_clip": true, "has_snapshot": true, "attributes": {}, "current_attributes": []}, "type": "end"}
Hardware
- Architecture or platform: amd64
- OS: Ubuntu Server
- Browser (if applicable): Firefox
- Docker image (if applicable): skrashevich/double-take:latest
Additional context
show your frigate config
re-read documentation, section "mqtt" in "cameras" block
# yaml-language-server: $schema=http://localhost:5000/api/config/schema.json
---
mqtt:
host: m
user: BLAHBLAH
password: BLAHBLAH
detectors:
deepstack:
type: deepstack
api_url: http://cpai:32168/v1/vision/detection
api_timeout: 5
go2rtc:
streams:
front_door:
- rtsp://BLAHBLAH:554/user=BLAHBLAH&password=BLAHBLAH&channel=1&stream=0.sdp?
garage_main:
- rtsp://BLAHBLAH:BLAHBLAH@BLAHBLAH:88/videoMain
cameras:
front_door:
ffmpeg:
hwaccel_args: preset-nvidia-h265
inputs:
- path: rtsp://127.0.0.1:8554/front_door
input_args: preset-rtsp-restream
roles:
- record
- detect
mqtt:
timestamp: false
bounding_box: false
crop: true
height: 640
required_zones:
- front_path
- front_street
snapshots:
required_zones:
- front_path
- front_street
ui:
dashboard: true
zones:
front_path:
coordinates: 1411,2160,959,1017,312,1139,611,2160
objects:
- person
front_street:
coordinates: 260,1106,2001,837,1952,599,241,855
objects:
- car
- bus
- motorcycle
- bicycle
- amazon
- ups
- fedex
motion:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
objects:
filters:
amazon:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
fedex:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
ups:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
bus:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
bicycle:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
motorcycle:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
car:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
person:
mask:
- 0,528,2701,577,3214,1164,1130,1937,1204,2160,3006,2160,3840,2160,3840,0,0,0
garage:
ffmpeg:
hwaccel_args: preset-nvidia-h264
inputs:
- path: rtsp://127.0.0.1:8554/garage_main
input_args: preset-rtsp-restream
roles:
- record
- detect
mqtt:
timestamp: false
bounding_box: false
crop: true
height: 640
required_zones:
- driveway
- garage_street
snapshots:
required_zones:
- driveway
- garage_street
ui:
dashboard: true
motion:
mask:
- 267,472,799,383,744,212,1407,286,1584,512,1920,622,1920,0,0,0,0,463
objects:
filters:
bus:
mask:
- 267,472,799,383,744,212,1407,286,1584,512,1920,622,1920,0,0,0,0,463
bicycle:
mask:
- 267,472,799,383,744,212,1407,286,1584,512,1920,622,1920,0,0,0,0,463
motorcycle:
mask:
- 267,472,799,383,744,212,1407,286,1584,512,1920,622,1920,0,0,0,0,463
car:
mask:
- 267,472,799,383,744,212,1407,286,1584,512,1920,622,1920,0,0,0,0,463
zones:
driveway:
coordinates: 871,350,1592,454,1292,1080,452,1080,281,881,137,530
labels:
- person
- car
garage_street:
coordinates: 1382,446,1529,299,891,220,732,327
labels:
- car
- bus
- motorcycle
- bicycle
- amazon
- ups
- fedex
rtmp:
enabled: false
snapshots:
enabled: true
clean_copy: true
timestamp: false
bounding_box: true
crop: true
height: 640
retain:
default: 10
objects:
person: 15
record:
enabled: true
retain:
days: 5
mode: active_objects
events:
pre_capture: 5
post_capture: 5
retain:
default: 10
mode: active_objects
objects:
person: 15
birdseye:
enabled: true
quality: 31
mode: continuous
detect:
enabled: true
height: 2160
fps: 5
objects:
track:
- person
- bicycle
- motorcycle
- car
- bus
- bird
- cat
- dog
- amazon
- ups
- fedex
filters:
dog:
threshold: 0.8
person:
min_area: 10000 <---- SET THIS A FEW MINUTES AGO TO ELIMINATE FALSE POSITIVES CAUSED BY MELTING SNOW
threshold: 0.8
bicycle:
threshold: 0.8
motorcycle:
threshold: 0.8
car:
threshold: 0.8
bus:
threshold: 0.8
bird:
threshold: 0.8
cat:
threshold: 0.8
motion:
contour_area: 30
improve_contrast: false
Try to set 'frigate.min_area' to 0 in double-take config, or 'crop: false' in each camera.mqtt on frigate configuration
And opencv_face_required to false
I am still don't see it picking up images from MQTT.
I think I may have found something. The log for latest version has this at the end:
23-11-28 02:57:52 verbose: api listening on :3000
23-11-28 02:57:52 info: MQTT: connected
23-11-28 02:57:52 info: MQTT: subscribed to frigate/events
Whereas in v1.13.11.3, I saw another topic subscribed in the logs, frigate/+/snapshot or something. Maybe that is the issue ...
Aite, I think this was the issue. So when I added the following mqtt.topics.frigate to my config file, it started picking up the MQTT events:
mqtt:
host: m
username: !secret mqtt_username
password: !secret mqtt_password
topics:
frigate: frigate/+/person/snapshot
I am not sure how but looks like the config default topic(s) for frigate MQTT events has changed. I can close this now, unless you need something else.
Same here
However, where to put frigate/events topic together?
And I noticed it stopped updating the sublabels
Bummer! Does a comma-separated list work?
BTW - what do you use the sub-labels for?
Bummer! Does a comma-separated list work?
BTW - what do you use the sub-labels for?
Quickly tested it. Comma separated is accepted by DoubleTake, however no image is being processed - like no mqtt topic is detected.
23-11-28 14:15:00 info: Double Take v1.13.11.8
23-11-28 14:15:02 info: MQTT: connected
23-11-28 14:15:02 info: MQTT: subscribed to frigate/events; frigate/+/person/snapshot
It looks like it's treating it like one topic ignoring comma.
I use sublabels in Frigate to sort folders and for some automations - mobile notification, locks, etc.
Sorry, I quickly skimmed the mqtt-util.js and looks like it might need to be an array/list.
I just tried to set it as a list and got an error while saving the config that it needs to be a string.
So I don't know. I will wait for the expert to comment.
I have suddenly started to receive mqtt and snaphots together. Here's my config:
mqtt:
host: 192.168.1.10
username: ****
password: ****
client_id: double-take
topics:
# mqtt topic for frigate message subscription
frigate: frigate/events
# mqtt topic for home assistant discovery subscription
homeassistant: homeassistant
# mqtt topic where matches are published by name
matches: double-take/matches
# mqtt topic where matches are published by camera name
cameras: double-take/cameras
frigate:
url: http://192.168.1.10:5000
update_sub_labels: true
stop_on_match: true
min_area: 0
labels:
- person
zones:
- camera: doorcam
zone: entrance
attempts:
# number of times double take will request a frigate latest.jpg for facial recognition
latest: 10
# number of times double take will request a frigate snapshot.jpg for facial recognition
snapshot: 10
# process frigate images from frigate/+/person/snapshot topics
mqtt: true
# add a delay expressed in seconds between each detection loop
delay: 0
image:
height: 1080
cameras:
- doorcam
cameras:
doorcam:
mqtt:
timestamp: False
bounding_box: False
crop: false
quality: 80
height: 1080
detectors:
compreface:
url: http://192.168.1.10:8000
# recognition api key
key: *****
# number of seconds before the request times out and is aborted
timeout: 15
# minimum required confidence that a recognized face is actually a face
# value is between 0.0 and 1.0
det_prob_threshold: 0.8
# require opencv to find a face before processing with detector
opencv_face_required: false
# comma-separated slugs of face plugins
# https://github.com/exadel-inc/CompreFace/blob/master/docs/Face-services-and-plugins.md)
# face_plugins: mask,gender,age
# only process images from specific cameras, if omitted then all cameras will be processed
# cameras:
# - front-door
# - garage
detect:
match:
# save match images
save: true
# include base64 encoded string in api results and mqtt messages
# options: true, false, box
base64: false
# minimum confidence needed to consider a result a match
confidence: 97
# hours to keep match images until they are deleted
purge: 168
# minimum area in pixels to consider a result a match
min_area: 3600
unknown:
# save unknown images
save: true
# include base64 encoded string in api results and mqtt messages
# options: true, false, box
base64: false
# minimum confidence needed before classifying a name as unknown
confidence: 97
# hours to keep unknown images until they are deleted
purge: 128
# minimum area in pixels to keep an unknown result
min_area: 3600
In the log appears now:
23-11-29 21:46:18 info: Double Take v1.13.11.8
23-11-29 21:46:19 info: MQTT: connected
23-11-29 21:46:19 info: MQTT: subscribed to frigate/events, frigate/doorcam/person/snapshot
However (might be frigate settings???) the match was done first on "latest". MQTT seems to be a bit later (mqtt was fired from frigate around 7 seconds after the match on "latest", and it's the same pic as registered as a snapshot in frigate.
I'll do some tests only with mqtt and see if this fires a few pics from frigate or just one (snapshot pic).
Sublabels also were updated.
So, as I have not changed the DoubleTake version, it seems like config file was wrong.
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.
This issue was closed because it has been stalled for 5 days with no activity.