Prototype sharer
harrynull opened this issue · 3 comments
Prototype a screen sharing client that captures the screen and encode it to a video file ('sharer').
Goal:
- Find the best API for screen capture in Windows (in terms of performance and capacities)
- Experiment and compare different codec that enables real-time (60FPS) encoding
Non-goal:
- Sound capturing
- Cross-platform
- Network
- Code style
For screen capturing, there are usually two options (same as what OBS screen capture supports):
- Windows.Graphics.Capture
- Supports only Win 10 1903 +
- Newer and usually recommended
- Works cross-GPU (e.g. DXGI can't capture games in dedicated graphics card if it's run with the integrated graphics card)
- A border around the capturing target i.e. the screen, UNTIL Windows 11
- DXGI
- better compatibility
- https://github.com/quadrupleslap/scrap/
H.264 seems to be the best codec (although it is patented); VP8 is a possible royalty-free alternative.
H.265, VP9, and AV1 all have better compression/image quality but their encoding performance is much worse.
texture time 0
pre encoding time 3
post convert time 38
post encoding time 165
post write time 165
encoded frame IDR ts 0 layers 2, size 77456
layer ftype 1/0 size 2 spid 0, tid 0 sequence id 0, quality id 0
layer ftype 1/1 size 1 spid 0, tid 0 sequence id 0, quality id 0
finish time 165
Results of encoding the first frame with OpenH264. It only supports YUV input so 35ms is spent to convert from BGR0 to YUV and then 127ms spent on encoding. A total of 165 ms is used to encode one key frame.
texture time 0
pre encoding time 1
post convert time 36
post encoding time 40
post write time 40
encoded frame Skip ts 5000 layers 0, size 8403
finish time 40
The encoding for a still image is faster, with 35ms convertion time and 4ms for encoding, adding up to 40ms.
x264 (GPL licensed) has support for BGR input and is much faster
pre encoding time 2
post encoding time 17
finish time 17
17ms for encoding one frame