Request - Integration of an Advanced Deinterlacing Filter Utilizing OpenCL
quamt opened this issue · 18 comments
Introduction
I propose an unconventional yet potentially impactful enhancement for qsvenc: integrating an advanced deinterlacing filter utilizing OpenCL. This approach aims to address combing artefacts found in efficiently telecined progressive streams and interlaced streams, intending to significantly improve visual quality in video processing and encoding.
Technical Rationale
This advanced deinterlacing filter brings several benefits:
Adaptive Deinterlacing:
It selectively deinterlaces only the affected parts of a frame, preserving detail in static image areas more effectively than standard methods.
Modeless Recovery:
The filter effectively manages any telecine or capture patterns without relying on predetermined patterns or phasing, making it adaptable to various standards such as 3:2 NTSC and PAL.
GPU Acceleration:
By leveraging OpenCL, this implementation can utilize GPU resources, which are typically more efficient than CPU at parallel processing tasks, potentially increasing processing speed and reducing the load on the CPU.
Feasibility and Comparative Advantage
Given the detailed source code and well-documented approaches available here https://www.rationalqm.us/decomb/decombnew.html, how feasible would it be to implement this advanced deinterlacing filter into qsvenc via OpenCL, especially considering the potential for significant speed increases and efficiency improvements over the current CPU-based implementation in Avisynth?
Evidence of Effectiveness
Comparative tests indicate that this filter outperforms existing deinterlacing tools by better preserving original detail and effectively reducing visible combing artefacts. Its adaptability to various motion and detail levels makes it highly versatile for multiple content types, from static to dynamic scenes.
Conclusion
Integrating this adaptive deinterlacing filter into qsvenc would enhance the tool's capabilities and align it with cutting-edge video processing technologies, meeting the demands for high-quality video production. This proposal aims to improve the software's functionality and expand its efficiency and applicability in professional and consumer markets.
I recognize that I frequently propose new features and request numerous enhancements. My eagerness stems from my deep appreciation of @rigaya's exceptional work and the proven effectiveness of these tools. By suggesting these improvements, I hope to contribute to the project's continued excellence and advancement.
I'm sorry but I am not very positive about this, so I must say that the priority is low, I'll think about adding it after I've finished some works I want to do in other repositories.
This is because we already have 4 deinterlace filters in QSVEnc (--vpp-deinterlace, --vpp-afs, --vpp-yadif, --vpp-nnedi), and personally I am happy enough with --vpp-deinterlace for 60i to 30p/60p and --vpp-afs for 60i to 24p. Furthermore, thinking about high quality adaptive methods, we can consider using QTGMC from Avisynth (which is way too far compilcated for me to implement in QSVEnc).
I have checked the adaptive deinterlace method you introduced and it seems to have several filters, FieldDeinterlace, Decimate and Telecide. Of these, I'll think about adding FieldDeinterlace based filter as --vpp-decomb, and I agree that speedup by OpenCL implementation might be useful.
Checking the source code, it seems that it is a combination of several processes and is not as simple as denoising. Also, there are other repository tasks that we would like to work on first. Therefore, I think it will take more time than I spent on vpp-nlmeans or vpp-denoise-dct.
Thank you for taking a look at my suggestion.
I understand that other tasks have higher priority and that implementing new features is only sometimes feasible in QSVEnc.
It's good to know that you're considering adding FieldDeinterlace as --vpp-decomb when you get the chance, but there's no rush; take all the time you need.
With four existing deinterlace filters already in QSVEnc, I see how adding another might not be high on the list, especially when there are other complex tasks.
I'm grateful for the available options and appreciate your honesty about the time and effort required for new features.
Thank you for your hard work.
@quamt What about opening a 3ad in their forum about this ?
Note that they already have a specific CUDA Filters section...
@rigaya Keep an eye on Donald Graft's Tools and Utilities page: he developed interesting stuff that could inspire.
Last but not least, you certainly know that you can also "grab" functions implemented into the @intel's libXCam library...
Hi @forart,
Thank you for your suggestion. However, the forum you mentioned mostly deals with filters, which are not directly related to OpenCL integration within QSVEnc, NVEnc, and VCEEnc. I appreciate your input, but my focus is to ensure that any improvements align specifically with QSVEnc's capabilities. Let's continue brainstorming and discussing ideas that will best enhance QSVEnc.
Best,
quamt
my focus is to ensure that any improvements align specifically with QSVEnc's capabilities.
OpenCL-based filters are - of course - more "cross-GPU" (then less efficient ?) but, if you look for more @intel -specific optimizations, you have to ask them to implement in OneVPL (which QSVenc - and not only - relies on)...
Anyway involving other open source devs could be a good approach for mutual evolution of codes, IMHO.
Hope thet inspires.
Hi @forart,
Thank you for your participation. I value your eagerness to leverage open-source collaborations for mutual benefits.
I would like to respectfully bring to your attention that this particular thread is intended to explore the possibility of integrating via OpenCL within QSVEnc. While the subject of broader GPU compatibility and collaborations is certainly valuable, it may be more appropriate to have a separate discussion to ensure that our current thread remains focused and productive.
I appreciate your understanding and encourage you to initiate other threads where broader optimization strategies and collaborations can be thoroughly explored. This will help maintain clarity and relevance in each discussion, ensuring that all topics receive the attention they deserve.
Thank you for your cooperation, and I am looking forward to seeing your insights in other discussions!
Best,
quamt
QSVEnc 7.65 adds --vpp-decomb, which is added as a similar filter to FieldDeinterlace.
It seems to have nice performance and fair quality, but seems to be rather weak for scenes including diagonal lines.
Thank you very much for adding this filter.
Does this work like the other deinterlacer where you can only use one at a time?
I'll test-run it later.
Does this work like the other deinterlacer where you can only use one at a time?
Yes, it shall work like other deinterlacer, only one can be activated.
Thank you very much.
Should this topic be closed since the filter has been implemented?
Or is it left open in case you are implementing anything else from here?
@rigaya
Decomb works well.
When encoding interlaced sources, I noticed that it shows the "real" fps in the cmd.
Example:
Input Info avqsv: MPEG2, 720x480, 24000/1001 fps
avcodec: ac3
VPP cspconv(nv12 -> yv12)
decomb: full on, threshold 20, dthreshold 7, blend off
denoise(nlmeans): sigma 0.005, h 0.050, patch 5, search 11, fp16 blockdiff
resize(lanczos4): 720x480 -> 1280x720
cspconv(yv12 -> nv12)
Trim 0-2000 [offset: 0]
AVSync auto
Output H.264/AVC(yuv420) High @ Level 3.1
1280x720p 1:1 23.976fps (24000/1001fps)
avwriter: h264, ac3 => matroska
What I mean by "real" is that the video shows 29.970 FPS with media info the source is mostly FILM, which would be 23.976fps as shown in the cmd output.
How can I adjust the audio in this case to match the video's new frame rate?
Thank you.
Is the audio out of sync in this case?
It depends on the input file but it might be coded as RFF.
Would you please rerun adding "--log-framelist framelist.csv" and upload framelist.csv? I might be able to see a hint of the behavior.
Is the audio out of sync in this case?
It depends on the input file but it might be coded as RFF.
Would you please rerun adding "--log-framelist framelist.csv" and upload framelist.csv? I might be able to see a hint of the behavior.
Yes, the audio gets out of sync.
I have attached the CSV
framelist.csv
Looking at the CSV it seems to indicate the presence of Repeat Field Flags (RFF).
It seems like it is mixture of RFF and ordinary interlaced frames, it might be difficult to solve. As result of RFF usage, the actual frame rate is near to 23.976 and not 29.97.
Adding --avsync vfr might resolve the audio out of sync error. If not, --avsync vfr --vpp-rff might work.
I ran it with both, and the audio remains out of sync.
--avsync vfr and --avsync vfr --vpp-rff
I did check the video with AGK, and it recommended Telecine for these DVDs.
When I use --vpp-decomb on PAL DVDs, it works like a charm.
I saw that FFmpeg has a detelecine filter, and I plan to try it to see if it resolves the issue of telecine artefacts in this type of video. Not all telecine artefacts are removed when using other QSVEnc filters with this specific source.
Decomb does a great job, but the audio still gets out of sync.
I’ll update you on the results once I’ve tested the detelecine filter.
FFMpeg shows the same behaviour.
Since the async issue is not directly the fault of decomb, should this topic be closed now that decomb has been implemented?
Thanks, I'll close the issue as the requested feature has been added.