livekit/python-sdks

rtc.VideoSource().capture_frame consume high CPU when publishing 25fps video stream

Closed this issue · 5 comments

quoctuanck99 commented

I'm working from this example here, and here is my modified version for 25fps video streaming:

WIDTH = 540
HEIGHT = 960

async def entrypoint(job: JobContext):
    await job.connect()

    room = job.room
    source = rtc.VideoSource(WIDTH, HEIGHT)
    track = rtc.LocalVideoTrack.create_video_track("single-color", source)
    options = rtc.TrackPublishOptions(source=rtc.TrackSource.SOURCE_UNKNOWN)
    publication = await room.local_participant.publish_track(track, options)
    logging.info("published track", extra={"track_sid": publication.sid})

    async def _draw_color():
        argb_frame = bytearray(WIDTH * HEIGHT * 4)
        while True:
            await asyncio.sleep(1/25)  # 25fps

            # Create a new random color
            argb_frame = bytearray(WIDTH * HEIGHT * 4)
            r, g, b = [random.randint(0, 255) for _ in range(3)]
            color = bytes([r, g, b, 255])

            argb_frame[:] = color * WIDTH * HEIGHT
            frame = rtc.VideoFrame(WIDTH, HEIGHT, rtc.VideoBufferType.RGB24, argb_frame) -
            source.capture_frame(frame)

    await _draw_color()


if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

Notably, CPU usage for each call is high at around 30%. After debugging, I found that most of the CPU load comes from this line: source.capture_frame
Does anyone have suggestions to reduce CPU usage for this?

Here are the docker stats:

docker stats >

CONTAINER ID   NAME           CPU %     MEM USAGE / LIMIT     MEM %     NET I/O   BLOCK I/O        PIDS
8116cd040c0a   livekit_perf   28.42%    1.221GiB / 15.44GiB   7.91%     0B / 0B   7.75MB / 721kB   496
davidzhao commented

how much resources did you give to the container? this is likely expected as each frame needs to be encoded for transmission

  • 👍1
quoctuanck99 commented

@davidzhao I didn't limit resources for this container.

this is likely expected as each frame needs to be encoded for transmission

We are limited by hardware, and this is really a challenge for us. Please provide us with suggestions.

theomonnom commented

@quoctuanck99 Which codec are you using?

quoctuanck99 commented

What do you mean exactly? Are you referring to the video codec? In the example above, I'm not using any video.

davidzhao commented

I'm pretty sure you are trying to publish frames as a video track.

In any case, if you are constrained on compute the only way to reduce consumption are to use a lower resolution and/or reduce FPS

  • 👍1