Greyscale Functionality Question

[WHS-Phoenix]

I get how r_greyscale works, but I would like to know if it's possible to do this to just the main render window while keeping everything else color, such as HUD, 3D icons, and the console. This would need to be toggled by a tr.refdef flag as it's part of a powerup-based special effect.

I have a method that works so long as r_ext_multisample is "0". If r_ext_multisample is anything other than "0" it repeatedly grabs the screen and blends it into a white image. I checked what's going on and if I draw to 1/4 of the screen I can see it repeating the view off to infinity. If I set r_ext_multisample "0" it draws exactly once. This problem only happens with the multisample buffers enabled. I tried running a modified version of the existing greyscale shader program at the end of postprocess, but that greyscales everything since it's being done in rb_swapbuffer so the 2D data has already been written.

Right now I have multisample disabled to get around this since it works fine with supersampling, but I would like it to work with multisampling. My OpenGL knowledge is very lacking and the framebuffer logic is confusing so any help is appreciated.

[ec-]

It heavily depends on a renderer: in OpenGL you need to set color mask before each drawcall, in Vulkan - use dedicated greyscale pipeline (-> 2x pipelines in general), as for me - not worth an effort over added renderer(s) complications

[WHS-Phoenix]

I'm using OpenGL. I haven't messed with the Vulkan renderer yet.

The answer you provided doesn't really explain anything to me that I can understand. Again, I know next to nothing about how OpenGL works, and I find it difficult to grasp. I did try to add the qglColorMask statement before and after the draw functions without any good effect. If set to GL_FALSE it just locks the current frame and won't ever release it. If set to GL_TRUE it does nothing different.

Here is the current greyscale function as I have it written. Perhaps you could take a look and see what's wrong? I have tried running this after R_RenderView as well as calling it from a rendercommand_t. The same thing happens either way. It works without the multitexture buffers, but repeatedly draws if they are enabled.

void FBO_GreyScaleScreen(int x, int y, int width, int height)
{
image_t *image;

// Phoenix - this works, just need to get it to not draw multiple times.
// r_ext_multisample 0 stops it but not sure how to get around it with multisample enabled.

// Must sync render thread so that the GL commands will work.
R_IssuePendingRenderCommands();
qglFinish();

// Use a scratch image from the array to write our framebuffer into

if (!tr.scratchImage[15]) {
	tr.scratchImage[15] = R_CreateImage(va("*scratch%i", 15), NULL, NULL, width, height, IMGFLAG_CLAMPTOEDGE | IMGFLAG_RGB | IMGFLAG_NOSCALE);
}

image = tr.scratchImage[15];
GL_Bind(image);

FBO_Bind(GL_READ_FRAMEBUFFER, frameBuffers[0].fbo);


// Copy down the framebuffer onto the scratch image, converting to greyscale by using
// GL_LUMINANCE as the channel instead of GL_RGBA
// Make sure the color is not set funky
qglColor3f(1, 1, 1);
qglCopyTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, x, y, width, height, 0);


// Set up for 2D rendering

RB_SetGL2D();

// Draw the actual screen overlay.  This will cover the full render window
// regardless of FOV or screen width.
qglBegin(GL_QUADS);
qglTexCoord2f(0, 1);
qglVertex2f(x, y);
qglTexCoord2f(1, 1);
qglVertex2f(x + width, y);
qglTexCoord2f(1, 0);
qglVertex2f(x + width, y + height);
qglTexCoord2f(0, 0);
qglVertex2f(x, y + height);
qglEnd();

}

[ec-]

I was not correct about OpenGL:

1. create separate dedicated post-processing ARB/GLSL shader
2. do render to the separate render target, save as texture for p.3
3. apply post-processing fullscreen quad in correct place (right before UI 2d rendering starts)

This all is OpenGL-specific + slow and not predictable since there is no reliable way to detect transition to UI rendering because some mods may have full 3D UI - thus, not worth an effort