Rendering Additional Images with View Modes

[zlou]

Hi, Thanks for the great work.

I am new to unreal so my questions may not be making sense and thanks for the patience in advance.

1. My project may need rendering unlit images and wonder if there are anyway to render unlit or other view modes from Unreal with EasySynth?
2. This may be related to Q1, I'm wondering if there are any references about how EasySynth communicates with Unreal to pull the RGB, depth data, etc?

[NikolaJov96]

HI @zlou, thanks!

Your questions are pretty related. Getting different types of outputs from the UE is done in the following way. The scene is captured by a CameraCompoment object. By default, its outputs are the same as what you can see in the editor, but you can also create a custom post-process material (shader) and assign it to the camera component. The post-process materials used by EasySynth are trivial and mostly only access different preexisting UE buffers. You can examine those materials by opening them inside the UE editor (see the image below on where to find them).

Modifying the plugin to render the unlit scene (base colors of scene components) can be done by modifying the M_PPColorImage post-process material responsible for color image output. The SceneTexture input node should be changed from PostProcessInput0 to BaseColor in the same way it is used inside the M_PPSemanticImage for semantic outputs.

(Screenshots are made using the dev branch and there may be some differences compared to what is currently in main)

[zlou]

Thanks for the timely response! I'll try to do it and let you know if there are any differences need to be noticed for the main branch if others want to do similar things!