mmcc-xx/WhosAtMyFeeder

How to tell if whosatmyfeeder is working?

Opened this issue · 21 comments

hermy65 commented

New to all of these software packages. I installed both Frigate and WhosAtMyFeeder on my unraid box. Frigate appears to be detecting what is needed, only had one detection of a bird and it was actually a squirrel. The issue im potentially running into is, how do i know if this is actually working. When Frigate classified the squirrel as a bird, it never showed up in WhosAtMyFeeder even though it was incorrect, i also did not see anything in the logs for WhosAtMyFeeder to suggest it decided against the squirrel.

So, how do i know this is working?

Below is my frigate config

mqtt:
  host: 192.168.0.134
  port: 1883
  topic_prefix: frigate
  user: user
  password: pass
  stats_interval: 60
detectors:
  tensorrt:
    type: tensorrt
    device: 0
model:
  path: /trt-models/yolov7-tiny-416.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 416
  height: 416
ffmpeg:
  global_args: -hide_banner -loglevel warning
  hwaccel_args: preset-nvidia-h264
  input_args: preset-rtsp-generic
  output_args:
    # Optional: output args for detect streams (default: shown below)
    detect: -threads 2 -f rawvideo -pix_fmt yuv420p
    # Optional: output args for record streams (default: shown below)
    record: preset-record-generic
detect:
  width: 1920
  height: 1080
  enabled: true
objects:
  track:
    - bird
    - dog
    - person
  filters:
    bird:
      threshold: 0.6
      min_area: 5000
snapshots:
  enabled: true
cameras:
  birdcam:
    record:
        enabled: True
        events:
          pre_capture: 5
          post_capture: 5
          objects:
            - bird
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://192.168.0.199:8554/cam
          roles:
            - detect
            - record
    live:
      stream_name: birdcam
    mqtt:
      enabled: True
      bounding_box: False #this will get rid of the box around the bird. We already know it is a bird. Sheesh.
      timestamp: False #this will get rid of the time stamp in the image. 
      quality: 95 #default quality is 70, which will get you lots of compression artifacts
stevenbrown1862 commented

Not really an expert but one way which may work would be to temporarily lower the confidence needed in frigate and this app and then hopefully you should see some random garden objects getting named as random birds.
An other way if you have any video of birds, then you could feed this video as a camera stream using go2rtc.
I've been using this second method to fine tune my pond heron detection.

hermy65 commented

Not really an expert but one way which may work would be to temporarily lower the confidence needed in frigate and this app and then hopefully you should see some random garden objects getting named as random birds. An other way if you have any video of birds, then you could feed this video as a camera stream using go2rtc. I've been using this second method to fine tune my pond heron detection.

@stevenbrown1862 Still grasping Frigate as a whole, how would i go about feeding a video into it using go2rtc?

stevenbrown1862 commented

I don't know how you have frigate set up currently. But adding something like the following to your streams section in the frigate config. You'll have to make sure that you've mounted the whole config folder, I think sometimes people only mount the config file.

streams:
birdvideo: ffmpeg:/config/my video.mp4

You will then have this video looping at rtsp://frigate ip:8554/birdvideo and so you can point a frigate camera to it.

Sorry for any typos but tricky to type on the phone.

hermy65 commented

I don't know how you have frigate set up currently. But adding something like the following to your streams section in the frigate config. You'll have to make sure that you've mounted the whole config folder, I think sometimes people only mount the config file.

streams: birdvideo: ffmpeg:/config/my video.mp4

You will then have this video looping at rtsp://frigate ip:8554/birdvideo and so you can point a frigate camera to it.

Sorry for any typos but tricky to type on the phone.

definitely dont understand frigate well enough to make this work unfortunately. When i add in what you listed above with (i think)correct spacing frigate doesnt work. Looked at the documentation and didnt find anything similar to what you had up there either

stevenbrown1862 commented

If its not working then I think the most likely cause will be that you haven't mapped the config folder into your container. Sorry I've no idea about unraid so I've no idea how that would work. This link below is the documentation for streaming an mp4 file.

https://github.com/AlexxIT/go2rtc#source-ffmpeg

hermy65 commented

@stevenbrown1862 ok i finally got a video of birds streaming over rtsp and into a new camera in frigate. Now though, its not detecting them as birds at all, more so detecting them as people or not at all?? In the config, birdtest is the youtube video im streaming to frigate

image
image

Below is my frigate config, maybe i have something set incorrectly there? Appreciate the insight!!!

mqtt:
  host: 192.168.0.134
  port: 1883
  topic_prefix: frigate
  user: user
  password: pass
  stats_interval: 60
detectors:
  tensorrt:
    type: tensorrt
    device: 0
model:
  path: /trt-models/yolov7-tiny-416.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 416
  height: 416
ffmpeg:
  global_args: -hide_banner -loglevel warning
  hwaccel_args: preset-nvidia-h264
  input_args: preset-rtsp-generic
  output_args:
    # Optional: output args for detect streams (default: shown below)
    detect: -threads 2 -f rawvideo -pix_fmt yuv420p
    # Optional: output args for record streams (default: shown below)
    record: preset-record-generic
detect:
  width: 1920
  height: 1080
  enabled: true
objects:
  track:
    - bird
    - dog
    - person
  filters:
    bird:
      threshold: 0.4
      min_area: 5000
snapshots:
  enabled: true
cameras:
  birdcam:
    record:
        enabled: True
        events:
          pre_capture: 5
          post_capture: 5
          objects:
            - bird
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://192.168.0.199:8554/cam
          roles:
            - detect
            - record
  birdtest:
    record:
        enabled: True
        events:
          pre_capture: 5
          post_capture: 5
          objects:
            - bird
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://192.168.0.115:8554/live
          roles:
            - detect
            - record
    live:
      stream_name: birdcam
    mqtt:
      enabled: True
      bounding_box: False #this will get rid of the box around the bird. We already know it is a bird. Sheesh.
      timestamp: False #this will get rid of the time stamp in the image. 
      quality: 95 #default quality is 70, which will get you lots of compression artifacts
stevenbrown1862 commented

Not sure on this one. I know frigate can be a bit random because of the coco model but not naming those pictures as birds is surprising. I'd suggest posting this on the frigate issues page to see if someone has any ideas.

hermy65 commented

@stevenbrown1862 posted over there, they said that the model im using isnt good at detecting birds but didnt offer a suggestion as to which model to use. Do you by chance know of what coco or yolo (or whatever its called) model is better at detecting birds?

Also, it did finally detect a couple birds but WhosAtMyFeeder didnt do anything with them still.

image
image

stevenbrown1862 commented

I've no idea about detectors sorry. I'm using yolov8 which is working well for me on my igpu. My use case is very different from you in that I'm trying to catch a heron stealing my fish. The easiest thing to see if your current detector is sub optimal would be to try running the default cpu detector to see if that detects better. When you can consistently get bird events then you could look in the who's at my feeder container logs as it will show the detection process assessing the picture from frigate there if its working.

daernsinstantfortress commented

Also, it did finally detect a couple birds but WhosAtMyFeeder didnt do anything with them still.

Are you seeing anything in the WhosAtMyFeeder console logs?

e.g.

Category(index=68, score=0.05859375, display_name='Cardinalis cardinalis', category_name='/m/01tgcy')
Getting image for event: 1688232892.504319-kg5mzz
Here's the URL: http://192.168.1.94:5000/api/events/1688232892.504319-kg5mzz/snapshot.jpg

The score is a percentage (threshold configurable in the app config through classification.threshold) but it may give an indication of whether things are working or just need a bit of tuning. I've got a discussion thread open here pondering just this question...

hermy65 commented

perhaps @mmcc-xx may have some input on the model portion or why whosatmyfeeder lists nothing even though it detected 2 birds?

hermy65 commented

@daernsinstantfortress nothing, this is all i see in the logs, unless there is a different location for logging i should look at.

192.168.0.115 - - [30/Jun/2023 08:44:33] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:33] "GET /static/css/styles.css HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:38] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:38] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 08:44:38] "GET /static/images/flutter.svg HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:39] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 08:44:39] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 08:44:39] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 10:49:59] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 17:04:43] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 18:16:28] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 18:16:29] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [30/Jun/2023 18:16:30] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [30/Jun/2023 18:16:30] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:09] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:11] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [01/Jul/2023 10:26:11] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:11] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:13] "GET / HTTP/1.1" 200 -
192.168.0.115 - - [01/Jul/2023 10:26:13] "GET /static/css/styles.css HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 10:26:13] "GET /static/images/flutter.svg HTTP/1.1" 304 -
192.168.0.115 - - [01/Jul/2023 11:23:34] "GET /static/css/styles.css HTTP/1.1" 304 -
Calling Main
Time: 2023-06-30 08:43:48.205
Python version
3.8.17 (default, Jun 13 2023, 16:09:51) 
[GCC 10.2.1 20210110]
Version info.
sys.version_info(major=3, minor=8, micro=17, releaselevel='final', serial=0)
Starting threads for Flask and MQTT
Starting flask app
 * Serving Flask app 'webui'
 * Debug mode: off
Starting MQTT client. Connecting to: 192.168.0.134:1883
stevenbrown1862 commented

Hmm is there anything after that, does it actually connect to the mqtt broker? I've pasted my logs below and you'll see there is confirmation of connection to mqtt.

Calling Main
Time: 2023-06-22 11:13:58.076
Python version
3.8.17 (default, Jun 13 2023, 16:09:51)
[GCC 10.2.1 20210110]
Version info.
sys.version_info(major=3, minor=8, micro=17, releaselevel='final', serial=0)
Starting threads for Flask and MQTT
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Starting flask app

hermy65 commented

@stevenbrown1862 interesting, so maybe its not actually connecting to mqtt then. in your config is it just ip:port or does it have the mqtt:// prefix? Mine is just ip:port and ive tried both true and false for auth and thats as far as it goes.

stevenbrown1862 commented

@hermy65
This is the relevant part of my config. You'll need to fix the indentations, I'm a bit rubbish at this github business.

frigate:
frigate_url: http://192.168.1.191:5000
mqtt_server: 192.168.1.191
mqtt_auth: true
mqtt_username: mqttuser
mqtt_password: mqttpass
main_topic: frigate

hermy65 commented

@stevenbrown1862 ok now we are making progress, changed the config to just mqtt ip NOT ip:port and it connected and started logging stuff like yours did. However, it is now identifying squirrels as morning doves :P

image

stevenbrown1862 commented

I guess this is why we don't need to worry about computers taking over the world and enslaving humans for a few years yet. I guess your way forward here, without having a better model, is to adjust the confidence percentages until your false positives are acceptable.

hermy65 commented

@stevenbrown1862 hate to keep hammering you on this but i just got my first bird and it looks like it id'd it in the logs but nothing is showing up on the webpage :/ Any idea what im doing wrong here?

image

image

stevenbrown1862 commented

I think the 'score' your seeing is the confidence so in other words its only 21-28% confident in the ID, therefore, it won't give you an ID in the UI and Frigate unless you lower your accepted confidence considerably (I wouldn't recommend these sort of levels though). I don't know whether the issue is the quality of the camera, the position of the camera, or just plain bad luck with how the bird was posing for you but to be honest I would probably ID that as a cat. Feel free to carry on hammering...at some point I may even have a useful answer.

mmcc-xx commented

Hi all - sorry for being tuned out for a while there. What @stevenbrown1862 said is correct - the confidence is below the threshold for a species identification to be considered real. That value is in the config file in the config directory.

Now, as to why Frigate isn't doing a great job with identifying birds in the first place...

I'd suggest trying a CPU based detector and the default built in model to see if you get any better results. You can back off on the fps in the camera section so it isn't hammering your CPU so hard. I also specify the resolution of my camera in the camera/detect section but I'm not sure how important that is.

Here's my camera definition in case it is helpful at all:

  birdcam:
    ffmpeg:
      input_args: preset-rtmp-generic
      inputs:
        - path: rtmp://192.168.1.114/bcs/channel0_main.bcs?channel=0&stream=0&user=admin&password=passwd
          roles:
            - detect
            - record
      output_args:
        record: preset-record-generic-audio-copy
      hwaccel_args: preset-vaapi
    live:
      stream_name: birdcam
    detect:
      enabled: True
      width: 2560
      height: 1920
      fps: 6
    record:
      enabled: true
    snapshots:
      enabled: true
      timestamp: False
      bounding_box: False
    mqtt:
      bounding_box: False
      timestamp: False
      quality: 95
    objects:
      track:
        - bird
        - animal
      filters:
        animal:
          threshold: 0.75
          min_area: 5000
        bird:
          threshold: 0.6
          min_area: 5000
NickM-27 commented

I also specify the resolution of my camera in the camera/detect section but I'm not sure how important that is.

if you don't it will resize the stream to 1280x720

  • 👍1