alelievr/HDRP-Custom-Passes

render object and camera motion vectors

sniklaus opened this issue · 8 comments

sniklaus commented

Huge thanks for sharing your stuff, this is great! I was just wondering how one would go about modifying RenderMotionVectors.cs to not only get object motion vectors but also camera motion vectors, do you have any pointers by chance? Thanks!

Chuudy commented

Huge thanks for sharing your stuff, this is great! I was just wondering how one would go about modifying RenderMotionVectors.cs to not only get object motion vectors but also camera motion vectors, do you have any pointers by chance? Thanks!

I would be also interested in this one.

alelievr commented

Hello,

With the current API, you can only use motion vectors in shaders (with the _CameraMotionVectorsTexture name). Modifying them is much harder though, that's why we're adding a new API in HDRP 12.0 that will allow to access the motion vectors buffer from C#.

I made a small example here: https://github.com/alelievr/HDRP-Custom-Passes/blob/2021.2/Assets/CustomPasses/CopyPass/CopyPass.cs#L79

Chuudy commented

Hi,

Thanks for your quick response. Unfortunately, I cannot load the project in Unity, as I receive an error on invalid dependencies.
I believe it's due to some absolute paths in the project, as it states the path: 'C:\Users\AntoineLelievre\Github...'

With the current API, you can only use motion vectors in shaders (with the _CameraMotionVectorsTexture name).

Any guidelines on how to do it with shaders? I had an implementation that worked in the standard rendering pipeline in Unity 2019.4, but cannot make it work with HDRP in 2021.1.

alelievr commented

Yeah, there are a lot of examples in the code of HDRP like the TAA implementation here:

https://github.com/Unity-Technologies/Graphics/blob/608fe2362c4f3be978f86398b7db4859eb33c6e4/com.unity.render-pipelines.high-definition/Runtime/PostProcessing/Shaders/TemporalAntiAliasing.shader#L161

But basically, you need to use the DecodeMotionVector with the result of the sample of _CameraMotionVectorsTexture like so:

    float2 motionVector;
    DecodeMotionVector(LOAD_TEXTURE2D_X(_CameraMotionVectorsTexture, closest), motionVector);

DecodeMotionVector is in the include file BuiltinData.hlsl so make sure to add this include in your code:
#include Packages/com.unity.render-pipelines.high-definition/Runtime/Material/Builtin/BuiltinData.hlsl

Chuudy commented

Thanks a lot, Antoine! I managed to render motion vectors to the render texture thanks to your help.
I wonder is there any way to get camera motion vectors in the custom pass at the 'BeforeRendering' or 'AfterOpaqueDepthAndNormal' injection point?
Perhaps it could be achieved when combined together with DrawRenderersCustomPass?
I would be very grateful if you could tell me whether it's possible at all.
My issue is that I need the motion vectors for the current frame during the rendering (including motion cause by animated objects and the camera itself). Before I was using a separate camera to render the motion vectors beforehand, but it required to go through the rendering process of the whole scene twice, and obviously wasn't very performant.

alelievr commented

I don't think that's possible without modifying HDRP. If you take a look at the pass graph of HDRP here, you'll see that the camera motion vector pass happens after the "After Opaque And Normals" pass:

May I ask for which kind of effect do you need them at this point in the pipeline?

Chuudy commented

You're right, it happens after the second injection point. That's why I hoped that maybe coupling it with some DrawRenderersCustomPass will force the engine to render out the motion vectors earlier.

May I ask for which kind of effect do you need them at this point in the pipeline?

Sure, I am not planning to do any post-process effect, I need the motion vectors for a current frame to steer the custom algorithm for NVidia VRS.