[Bug]: can't support h.264 interlace encoder
DaveHu-TVU opened this issue · 14 comments
Which component impacted?
Encode
Is it regression? Good in old configuration?
No, this issue exist a long time
What happened?
OneVpl 23.1.5
Ubuntu 22.04
cmd:/opt/intel/media/share/vpl/samples/_bin/sample_encode h264 -i ./cnn.yuv -o ./output.h264 -w 1920 -h 1080 -b 5120 -f 30 -lowpower:on -u speed -tff
error log:
CONFIGURE LOADER: required implementation: hw
CONFIGURE LOADER: required implementation mfxAccelerationMode: MFX_ACCEL_MODE_VIA_VAAPI
libva info: VA-API version 1.18.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /opt/intel/media/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_18
libva info: va_openDriver() returns 0
libva info: VA-API version 1.18.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /opt/intel/media/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_18
libva info: va_openDriver() returns 0
libva info: VA-API version 1.18.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /opt/intel/media/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_18
libva info: va_openDriver() returns 0
[ERROR], sts=MFX_ERR_UNSUPPORTED(-3), AllocFrames, Query (for encoder) failed at /opt/src/vpl-dispatcher_src/tools/legacy/sample_encode/src/pipeline_encode.cpp:1003
[ERROR], sts=MFX_ERR_UNSUPPORTED(-3), ResetMFXComponents, AllocFrames failed at /opt/src/vpl-dispatcher_src/tools/legacy/sample_encode/src/pipeline_encode.cpp:2086
[ERROR], sts=MFX_ERR_UNSUPPORTED(-3), Init, ResetMFXComponents failed at /opt/src/vpl-dispatcher_src/tools/legacy/sample_encode/src/pipeline_encode.cpp:1866
[ERROR], sts=MFX_ERR_UNSUPPORTED(-3), main, pPipeline->Init failed at /opt/src/vpl-dispatcher_src/tools/legacy/sample_encode/src/sample_encode.cpp:1941
Frame number: 0
Encoding fps: -nan
mfxlib_Pid85807_Tid140115397953344.log
What's the usage scenario when you are seeing the problem?
Transcode for media delivery
What impacted?
No response
Debug Information
Do you want to contribute a patch to fix the issue?
Yes, I'm glad to submit a patch to fix it
Hi @yefeizhou any update? Thanks
Auto Created VSMGWL-66141 for further analysis.
Hi @daleksan, Can you help to take a look?
Hi @DaveHu-TVU @OttoYang apologize for delay in response, @DenWolf Could you take a look on the issue above?
Hi @DaveHu-TVU ,
H264/AVC LowPower Encoder (same as VDEnc - "lowpower:on") does not support Interlace encoding, it support only progressive - so, actual behavior is correct.
I see 2 options here:
- If you need AVC Interlace encode - you can use VME-based Encoder - set "lowpower:off" in your command-line or remove this "lowpower" option at all -> in both cases it will switch to VME Encoder.
- If you need LowPower Encoder - you should use Progressive mode -> remove "-tff" option.
Which Intel platforms are you using?
Based on LibVA traces shared by you I suppose it's a TGL (as I see MPEG2e and AV1d) - but please help to confirm.
If it's possible - please help to share more information about your use-case and "final" goal - maybe we can suggest another solution for your purpose.
P.S.: you can find more information about HW Encoding support on our Media Driver page - https://github.com/intel/media-driver#decodingencoding-features
Feel free to ask any questions.
Best regards,
Denis
Hi @DenWolf In our case we need 8+channels encoder(HEVC or H264), if we can't enable low power, the GPU does not support such a multi-path encoding performance, so we must enable lowpower.
our CPU is [12th Gen Intel(R) Core(TM) i7-12700], I think it's a [Tiger Lake].
another quesion: why HEVC support interlace encode when enable low power?
Hi @DaveHu-TVU ,
Thank you for a quick response! I will reply to your questions one-by-one via a separate comments, let's start from the last one:
>> another question: why HEVC support interlace encode when enable low power?
It's a pretty interesting case as there is a significant difference between AVC and HEVC - originally Interlace support was not enabled into HEVC Standard/Specification - HEVC focused on Progressive encoding, primary for HD+ (typical resolution for Interlace is SD - 480i).
Later some kind of analog of Interlace support has been added into HEVC via SEI (+ VPS / SPS / VUI).
Roughly say, HEVC Interlace is some kind of "SW hack" based on Progressive Encoding algorithms.
Thant's why Intel HEVC Encoder supports Interlace for VME and LowPower cases.
On the other hand, there are special algorithms for AVC Encoder (based on Spec) specially for Interlace encoding - and it's required special HW methods support.
As results, this support has been added into AVC VME Encoder, but was not added into AVC LowPower - as there were no such requirements/requests (+ Interlace encoding is a pretty specific and rarely cases now).
Some good quote from https://vicuesoft.com/blog/titles/h265_revision2/ :
The first version of the H.265 standard (also known as HEVC) was approved in 2013. Initially, interlace encoding support was not assumed in H.265 standard and was added merely as SEI messages later. So some pretty efficient encoding tools, like special algorithms for reference lists building, didn't shift from the previous H.264 standard. In terms of H.265, interlace encoding is a simple progressive encoding with special SEI indicating that the encoded access unit is a field of fields pair but not a progressive frame.
I also put some more links with useful information about HEVC Interlace:
https://www.ramugedia.com/interlace-support-in-hevc
https://en.wikipedia.org/wiki/High_Efficiency_Video_Coding#Video_coding_layer
MediaSDK / VPL HEVC Interlace documentation:
https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk_hevc_interlace_whitepaper.md
MediaSDK / VPL structure for Interlace:
https://github.com/Intel-Media-SDK/MediaSDK/blob/master/doc/mediasdk-man.md#picstruct
https://github.com/Intel-Media-SDK/MediaSDK/blob/7a72de33a15d6e7cdb842b12b901a003f7154f0a/api/include/mfxstructures.h#L133
@DaveHu-TVU - now about your first/original question:
For Interlace Encoding on Intel Gen12 you can use HEVC LowPower encoder (and VME if it's needed) and AVC VME encoder only. I also verified on my Tiger Lake that all these 3 Encoders are working.
12th Gen Intel(R) Core(TM) i7-12700 - it's an Alder Lake (ADL) - current HW is based on TGL, Media functionality (AVC/HEVC) is a similar as for TGL.
https://ark.intel.com/content/www/us/en/ark/products/134591/intel-core-i712700-processor-25m-cache-up-to-4-90-ghz.html
Also, ADL is a pretty powerful machine, performance for AVC VME Encoder is also good even for Multi-encoding.
So, I suppose you could also try to use AVC VME for your scenarios, maybe actual performance will be applicable for your cases (especially as there are no other tasks/threads for EUs on GPU).
If you want to use AVC LowPower Encoder - it could work only in Progressive mode, no Interlace support.
Best regards,
Denis
Hi @DenWolf Thanks your help. I know the resason why h.264 interlace encoder can't support when enable low power now.
Our another cpu is [ 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz], we need encoder 9 channels interlace h264.
4 channels is 1080i60 10M, 4 channels is 1080i60 15M and 1 channel is 720 1M/s, After our tests, using AVC VME can't support too much Multi-encoder, also our other programs running will hero the system read and write, thus affecting the encoding performance, do you have good suggestions for this usage scenario? Thanks!
Hi @DenWolf Thanks your help. I know the resason why h.264 interlace encoder can't support when enable low power now.
Our another cpu is [ 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz], we need encoder 9 channels interlace h264. 4 channels is 1080i60 10M, 4 channels is 1080i60 15M and 1 channel is 720 1M/s, After our tests, using AVC VME can't support too much Multi-encoder, also our other programs running will hero the system read and write, thus affecting the encoding performance, do you have good suggestions for this usage scenario? Thanks!
Hi @DaveHu-TVU ,
11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz is a TGL
Your pipeline - 8 FullHD + 1 HD simultaneous Interlace encoding in BRC mode - is a pretty heavy case for AVC VME using only 1 TGL machine (+ some additional SW occupied HW).
You can additionally check GPU utilization, but I suppose it should be already closed to 100%.
Plus you are already using TargetUsage=7 ("-u speed") which is fully correct this case.
Below I try to provide some tips which could somehow increase the performance in your pipeline without significant Quality degradation or scenarios breaking - but it's needed to be checked via practical experiments, and most likely it's not introduced a significant changes due to GPU utilization as I described upper:
- try to set difference AsyncDepth (asynchronous mode) - started from 1 up to 9 or even 18 (default is 3)-> maybe it will somehow improve the pipes synchronization
- try to set UseRawRef (using for prediction original/Raw frame instead of Reconstruct) - but here could be 2 problems: (1) I'm not sure that is' workable for Interlace; (2) Quality degradation (if any) is not predictable
- if B-frames are not required - you can use I-/P-frames only by setting GopRefDist=1
- you can use CQP or ICQ modes instead of BRC - but here is a pretty difficult to keep the required BitRate (+ no HRD) - so, most likely it's also not applicable for you...
@DaveHu-TVU - one last idea from my side - you can extend your HW capabilities by enabling addition discrete GPU - DG1 . It has a similar HW Media as TGL, so TGL+DG1 will be both used for Encoding.
Hi @DaveHu-TVU ,
Do you have some other questions of help-needed topics related to Interlace on AVC or HEVC ?
Or we can close current ticket?
Original request has been clarified + there was no new questions within the last month - closed an actual ticket.
Feel free to re-open this issue or create a new one in case of new questions / requests / problems.