CasparCG/server

Bug: ffmpeg consumer constant failure - cannot allocate memory

paveltonev opened this issue · 4 comments

paveltonev commented

Observed Behavior

Use CasparCG server v2.4.0-rc1 with ffmpeg 6.1.1 and start ffmpeg consumer started via:

<consumers>
    <ffmpeg>
        <path>udp://127.0.0.1:9094?pkt_size=1316&bitrate=8000000&reuse=1&overrun_nonfatal=1&max_interleave_delta=1000000&fifo_size=xxxxxxx&buffer_size=yyyyyyyyy</path>
        <args>-codec:v h264_nvenc -preset:v p7 -profile:v high -tune:v ll -level:v 4.2 -g:v 50 -rc:v cbr -b_ref_mode:v middle -temporal-aq:v 1 -rc-lookahead:v 20 -i_qfactor:v 0.75 -b_qfactor:v 1.1 -bufsize:v 0.7M -max_delay 0.7M -pcr_period 30 -minrate:v 7M -maxrate:v 7M -b:v 7M -bf:v 2 -qmin:v 0 -ac:a 2 -channel_layout:a stereo -codec:a aac -ar:a 48000 -b:a 256K -bufsize:a 128K -minrate:a 256K -maxrate:a 256K -format mpegts -muxrate 8M -mpegts_flags +system_b -fflags +genpts+nofillin -flush_packets 0 -mpegts_pmt_start_pid 0x1500 -mpegts_start_pid 0x150 -mpegts_service_type advanced_codec_digital_hdtv</args>
    </ffmpeg>
</consumers>

with CasparCG client v2.3.0-rc1 executing a 48 hours playlist. Everything started as expected and on the 12th minute the ffmpeg cosumer terminates with Cannot allocate memory issue described here. It appeared that this error has already been reported as issue on ffmpeg trac system here.

I increased 10 times the default fifo_size and buffer_size values, as a result the ffmpeg consumer failed in around 120 minutes.

I increased 20 times the default fifo_size and buffer_size values, as a result the ffmpeg consumer failed in around 240 minutes.

Expected behaviour

I expect ffmpeg consumer not to throw such an error.

Steps to reproduce

  1. Install CasparCG server v2.4.0-rc1 under Ubuntu 22.04
  2. Create consumer with the above configuration without fifo_size and buffer_size settings.
  3. Use CasparCG client v2.3.0-rc1 under Ubuntu 22.04 and create a more than 12 hours playlist
  4. Wait for no more than 20 minutes, the consumer will fail with the issue described above.
  5. Increase fifo_size and buffer_size proportionally to their default values, the failure is proportional to the change.

Environment

Julusian commented

I'm struggling to reproduce this, the command provided above does not work with ffmpeg 4.4, which is what ubuntu 22.04 uses by default.

paveltonev commented

I use ffmpeg 6.1.1 built from source with NVIDIA GPU. The workflow is following:

  1. Install NVIDIA driver
  2. Install CUDA from https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#ubuntu
  3. Build from source FFMPEG with all required modules and CasparCG. Use this file
    install_direct_6.1.1.sh.zip
    as a reference and change INSTALL_PATH to point your project location.
paveltonev commented

I did a test also with v2.4.0-stable from yesterday and same error appeared.
Please find attached the log, where I started 5 channels with no fifo_size and buffer_size and 4 of them terminated with "Cannot allocate memory" error.
nohup.out.zip

jpc0 commented

I just ran the same output with a random input video directly in ffmpeg and got the same issue.

From the final comment in the trac thread you posted

[comment:26](https://trac.ffmpeg.org/ticket/6463#comment:26) by Marton Balint, [5 years ago](https://trac.ffmpeg.org/timeline?from=2019-03-17T00%3A10%3A58%2B02%3A00&precision=second)

It is most probably not a memory issue, simply there is not enough space in the output fifo. Error message is a bit misleading. Try increasing the fifo size on the output.

Does not seem to be an issue with casparcg? Could you try the same test.