geerlingguy/pi-nvr

Optimize Frigate Performance / Utility

Opened this issue · 11 comments

geerlingguy commented

Frigate is looking like the head-and-shoulders winner for running on a Pi (at least Pi 5), especially in tandem with Coral. Would love to see other 'neural' TPUs supported too (Hailo... maybe the one built into some Rockchip SoCs?).

But I'm seeing that it won't scale too drastically with H.265, H.264, 4K, multiple cameras... it starts eating CPU just transcoding stuff and translating it for internal processing (ffmpeg is eating up 150% CPU constantly while just processing one 4K stream at H.265).

I want to make sure the Pi 5's built-in hardware decoding support is used where possible, and also make sure the TPU and any other little performance optimization is used to its fullest extent. And also streamline the deployment a bit, especially since I'd like one configuration to run both my home setup and my office (with different types and numbers of cameras).

Also would like to make it so I could easily swap over to a standard Linux PC at some point or maybe even try running it with Docker on my Mac (is that doable?), all with the same Ansible playbook.

For some inspiration, see Raspberry Pi 3/4 with ffmpeg.hwaccel_args: preset-rpi-64-h264

Things to investigate:

  • Hardware acceleration for H.265 decode?
  • Using camera substream for detect (I was using the main 4K stream, but that is not optimal)
  • Storing media on an alternate drive (NVMe) or on a network share—it's currently on the microSD (yikes!) at /mnt/frigate
  • Check on power consumption using USB power meter? Or wall power measurement with smart plug.
  • Compare overall performance to an older mini PC (maybe my M710q) and see how things are in Pi 5-land (see some discussion here)
  • Compare power consumption to an older mini PC

Follow-up to #4.

geerlingguy commented

Using the substream (640x480, H265 — the camera also allows for MJPEG if desired) for detect, I'm able to view the lower-res in browser, while clips and recordings are still done at full 4K resolution:

cameras:
  basement:
    ffmpeg:
      output_args:
        record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -tag:v hvc1 -bsf:v hevc_mp4toannexb -c:a aac
        rtmp: -c:v copy -c:a aac -f flv
      inputs:
        - path: "rtsp://jgeerling:{FRIGATE_RTSP_PASSWORD}@cam02.geerli.net:554/H264/ch1/sub/av_stream"
          roles:
            - detect
        - path: "rtsp://jgeerling:{FRIGATE_RTSP_PASSWORD}@cam02.geerli.net:554/H264/ch1/main/av_stream"
          roles:
            - record

The CPU usage is down to like 40% overall (much less than one full core), and FPS for ffmpeg/Capture are capped at 5. The recordings and clips are all at 15 fps.

So... don't use the 4K stream for detect, only for record :)

geerlingguy commented

I also brought home a Pineberry Pi HatDrive! Bottom and am 3D Printing this case, so I'm able to run everything off a 1TB NVMe drive over PCIe. Nice and fast.

geerlingguy commented

Power consumption for my Pi 5 with an attached Coral USB accelerator and NVMe via Pineberry HatDrive! Bottom (booting off NVMe, no microSD card inserted), with two Annke 4K cameras, running event detection and 24x7 recording on both, averages 5.2W (over the course of 24 hours).

Want to also test with CM4 and with the Mini PC. Maybe also some 1080p recording testing (should be nearly identical on each system vs 4K though, since I'm processing the 640x360 substream, and recording the HD/4K stream directly to disk).

platypus11 commented

Hi Jeff,
Very many thanks for this.
I've recently switched to frigate from surveillance station as the person detection is super reliable with frigate and not susceptible to false alerts caused by wind, rain etc.
I'm running 5 cameras on a rpi5 with a coral usb stick and had no cpu% issues at all.
I'm just curious- have you made any progress with the hardware acceleration for H.265 decode? I currently don't have it set up.
I've had a poke around on the web and haven't been able to find anything definitive

geerlingguy commented

What I wound up doing was using a substream for preview, which means the web UI can stream the video (and Pi can do motion detect) using very little CPU without worrying about decode. Then for playback and export, it just handles the H.265 files straight to the browser.

The slowest part is exporting footage—ffmpeg eats up a lot of CPU doing that with my 4K streams.

platypus11 commented
w-marco commented

I just stumbled across the corresponding video and found this repo.
Went onto the journey to setup frigate myself a few weeks ago. Have you tried with go2rtc and restream to compare how CPU usage does?

It also seems to be the preffered method by the frigate developers rather than directly using the RTSP streams inside ffmpeg.

A sample would look like this (the RTSP URLs are Reolink but can be adapted to match other camera makers):


go2rtc:
  streams:
    basement:
      - rtsp://USER:PASS@CAM_IP:554/h264Preview_01_main
      - "ffmpeg:basement#audio=opus"
    basement_sub:
      - rtsp://USER:PASS@CAM_IP:554/h264Preview_01_sub
  webrtc:
    candidates:
      - FRIGATE_IP:8555
      - stun:8555

cameras:
  basement:
    enabled: true
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/basement
          roles:
            - record
        - path: rtsp://127.0.0.1:8554/basement_sub
          roles:
            - detect

One could then also use the ffmpeg input_args preset named 'preset-rtsp-restream'.

danluckner commented

@w-marco

I have a Reolink Doorbell PoE and have been trying to get my RPi5 working well with your configuration idea, but I'm getting ffmpeg errors. Tried both h264 and h265 streams from the Doorbell, but cannot get it to work:

2024-06-24 16:44:16.669821843 [2024-06-24 16:44:14] frigate.video ERROR : reolink: ffmpeg process is not running. exiting capture thread...
2024-06-24 16:44:44.182194872 [2024-06-24 16:44:44] ffmpeg.reolink.record ERROR : [tcp @ 0xaaaad9da9e60] Connection to tcp://FRIGATE_IP:8554?timeout=5000000 failed: Connection refused
2024-06-24 16:44:44.182200075 [2024-06-24 16:44:44] ffmpeg.reolink.record ERROR : rtsp://FRIGATE_IP/reolink: Connection refused

Note: I tried this in a Proxmox homelab machine with x64 processor and it works fine, so I'm sure my configuration is correct. I assume it is the inability of the RPi5 dealing with H265 decoding.

geerlingguy commented

@danluckner - This issue isn't probably the best location for debugging Reolink issues, but the Pi 5 should be able to handle H.265 okay. It's more likely a configuration issue; I would consider posting your full config maybe to the frigate issue queue or just double checking every value, especially to make sure it's all spelled correctly.

See for example blakeblackshear/frigate#699 or blakeblackshear/frigate#8505

  • ❤️1
danluckner commented

@geerlingguy thanks! I will check the references and post it in the right forum, if needed.
I will come back if I have a solution.

danluckner commented

Hey guys, I think I was able to fix it. I let it run overnight to make sure it's stable and it seems so.
The main changes were:

  • on the go2rtc config, use Frigate's webiste guide for Reolink cameras (https://docs.frigate.video/configuration/camera_specific/)
  • on the webrtc candidates, I had the 127.0.0.1 IP, but it makes it unstable. So I changed to the actual IP where Frigate in instanced and it made it much more stable.