[FEAT] Recognize "objects" too

Is your feature request related to a problem? Please describe.
Now i see that double take subscribes only at frigate/+/person events in mqtt but i would like to recognize my different cats with custom models

Describe the solution you'd like
Id like to subscribe to different topics and use different model to recognize different objects than just faces.

Additional context
i have 4 cats, every one of a different color. Id love to have an album to check they are fine when outside from home more than one day

in configuration you can add labels, which will be passed to the recognition system:

frigate:
# object labels that are allowed for facial recognition
  labels:
    - person # <-- the default one
    - cat
    - elephant # <-- you can add here labels of your home pets

list of available labels: https://raw.githubusercontent.com/blakeblackshear/frigate/dev/labelmap.txt

P.S. I really doubt that any of the face recognition detectors will be able to distinguish cats from each other. If you find a solution to this problem, please write here and I'll add this information to README

I did add cat, but i don’t see any subriscription on the frigate/+/cat on mqtt and no images of cat in the main page

I did add cat, but i don’t see any subriscription on the frigate/+/cat on mqtt and no images of cat in the main page

in frigate config you also need to make changes:

objects:
  track:
    - person
    - car
    - cat
    - elephant # <-- you can add here labels of your home pets

If you use zones in the frigate configuration, don't forget to add a new object type and to them

If it doesn't work after that, write here the frigate and double-take versions and configurations used. This will help me reproduce your configuration and understand what's going wrong

I concur with @Melkor82 findings @skrashevich - without burdening you with sharing the whole setup, I have zones defined (but nothing tracked specifically for zones) from frigate. So frigate has a generic:

objects:
  track:
    - person
    - cat
    - car

Accordingly in doubletake:

frigate:
  labels:
    - person
    - car
    - cat

Sifting through subscriptions on MQTT reveals double-take subscribed to:

 - frigate/events
 - frigate/{camera1_name}/person/snapshot
 - frigate/{camera2_name}/person/snapshot

I would assume it would also need to subscribe to frigate/camera/car/snapshot right? (Also interested in 'cat' but with CodeProject trying to do ALPR :)

Gentle nudge :)

+1 intersted in this topic as well. From what I see - double-take have default subscriptions to frigate/events, frigate/+/person/snapshot even if I add more object to track it will still remain the same.

But afaik frigate/events have all tracked events by itself, so there is no need to have additional subscription to separated topic in this case.

But models that I use from double take side (mostly deepstack) has very poor recognition of "cat" object. At least with my setup. So from like 100+ snapshots of my cats sent to double-take it can recognize only 1 as cat or even nothing.

If I start the latest version (1.13.11.3) with the above even double-take itself in the container logs indicates: info: MQTT: subscribed to frigate/events, frigate/cam1/person/snapshot, frigate/cam2/person/snapshot, frigate/cam3/person/snapshot so no subscriptions to 'other objects' :(

@skrashevich looks to me like the code is hardcoded to subscribe to frigate/+/person/snapshot or frigate/<camera>/person/snapshot topics?

double-take/api/src/util/mqtt.util.js

Line 141 in 5b4651d

? FRIGATE.CAMERAS.map((camera) => `${prefix}/${camera}/person/snapshot`)

But I would note my understanding that the frigate/+/<label>/snapshot is not the only way DT detects objects. I believe the /person/snapshot topic is only used for mqtt images, and isn't required for the snapshot and latest which are requested from Frigate by DT based on receipt of an event to frigate/events.

If you set logging to verbose, and receive events from Frigate on frigate/events, DT will log out why it chose to ignore a given event. Some of the key checks are in here:

double-take/api/src/util/frigate.util.js

Line 72 in 5b4651d

return `${id} - ${label} label not in (${FRIGATE.LABELS.join(', ')})`;

As an example, I see log entries like this:

23-10-03 15:31:15 verbose: Incoming event from frigate: {"before": {"id": "1696347069.537423-nrb1jx", "camera": "hikcam4", ...snip...}

23-10-03 15:31:15 verbose: 1696347069.537423-nrb1jx - hikcam4 zone not on approved list

So if someone tries this with logging enabled you might see where Frigate is deciding to ignore it.

Maybe I need to bump it to silly instead of verbose but it does note the (note the spelling error?) incoming event, but not sure if it actually proceeds doing anything with it

double-take  | verbose: Incominng event from frigate: {"before": {"id": "1696361557.704825", "camera": "somecam", "frame_time": 1696361558.104154, "snapshot": {"frame_time": 1696361558.104154, "box": [...], "area": ..., "region": [...], "score": 0.7265625, "attributes": []}, "label": "cat", ...

1.13.11.4 fixes the typo (and has various dependency upgrades)

So there's nothing in your logs immediately after that incomint event message? At least it's proved out your Frigate configuration is definitely sending the events to DT.

I did try this on my setup and saw Frigate images of birds show up in the DT UI so it's definitely possible and probably a configuration issue with your cameras, zones or labels. If you post a redacted full configuration happy to compare to mine.

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

This issue was closed because it has been stalled for 5 days with no activity.