A crazy idea about TAA... USE THE MOUSE
mhtvsSFrpHdE opened this issue · 7 comments
Some Razer device now support mouse movement in their Razer synapse driver,
you might also can get automatically mouse movement macro use AutoHotKey software.
Since the TAA is based on a high speed shaking 3D camera,
and we can't ask game enging to shake because this is post-processing...
So what about use mouse macro to shake?
Could this bring any means to get a true TAA implementation?
Imagine you use a mouse to shake a PC video game and play with an XBOX controller.
UPDATE at 1/10/2022:
These gets done, a video demo to show results.
https://www.youtube.com/watch?v=vIg0JeLIjTc
This is a raw screenshot image without any anti aliasing,
I'm on a high building so things on the ground all became small details.
When I use the mouse macro to shake the camera in very high speed, I got this...
IT WAS ACTUALLY DOING ANTI-ALIASING!
What I do is first the mouse stay at x 0 y 0,
then after 16ms the mouse will move to x 1 y 1 the right-bottom direction,
again, after 16ms the mouse will move to x 1 y 0 the top direction,
finally, after 16ms the mouse move back to x 0 y 0.
Every 48ms the camera will shake around a triangle.
But the thing is, although this anti aliasing do better than FXAA(of course, multi frame vs single frame),
I still didn't get temporal stabilization. When I WASD move, those flicker edges didn't get erased by TAA.
Some of them are less noticeable, but right now, it's just become a stronger
post-processing anti aliasing method, without bring temporal benefits.
Could you provide any tweak suggestion?
The Issue with this is that it can't move at the sub-pixel level that is needed for TAA to work well. I made an external app and tried what you said. But, it limited me too full pixel steps.
Good Idea but windows limitations keep this from working.
Also, full pixel steps are jarring. But, if you get it working good.
The Issue with this is that it can't move at the sub-pixel level that is needed for TAA to work well. I made an external app and tried what you said. But, it limited me too full pixel steps.
Good Idea but windows limitations keep this from working.
Also, full pixel steps are jarring. But, if you get it working good.
I spend two days on this topic and "created a pipeline" that combine several plugins to work together.
The Pipeline
GShade
A reshade branch that able to isolate UI element from shading
SMAA will ruin the UI texts with default setting, so any solution can isolate UI are good.
Razer Synapse 3
Their mouse macor driver with Razer Death Adder 1800 dpi mouse
I use Razer software managed to move the mouse cursor between two position in very high speed.
Here are the mouse macro loop pseudocode:
start
0msx0 y0 // initial mouse position
16ms x1 y1 17msx0 y0 17msx1 y1
16msx0 y0 17ms x1 y1 17ms x0 y0
goto start
1000 ms / 60 frames is about 16.66666666667 ms,
(1000 ms / 60 frames) * 3 = 50 ms,
16 ms + 17 ms + 17 ms = 49 ms
so these value is the most stable value I can find to align the 60 fps.
CeeJayDK's SMAA fx porting
Start the game, enable SMAA before TAA, use default settings.
If enable SMAA after TAA, some aliasing edge will not be processed, reason is unknown.
Your Temporal AA fx porting
Enable TAA after SMAA, back to game, enable Razer mouse macro,
game engine did start to shake with mouse and now TAA start to blend frames.
Quality and Why SMAA
This pipeline now are successfully brings me a VERY ACCEPTABLE AA quality.
I think it can beat the MSAA 2x and very nears to MSAA 4x but just
requires about half of MSAA 2x performance or even less.
This is a success, by means "pure post-processing".
Let's assume a true TAA is "pan moving" the 3D camera, so all 3D objects will get captured twice:
And I might call this pipeline "Manual shaking temporal AA":
It was not actually "pan moving" the 3D camera, instead,
it based on keeps relocating camera focus from time to time.
You do can get multi angle aliasing information for far 3D objects
and double the signal-to-noise ratio by this way, but not the near objects.
Because 1px is way too small to change the angle, so near objects are static instead of shaking.
Here I ask SMAA to use tradition single frame AA to process near objects.
The near objects are usually rendering at relatively high resolution so SMAA works good.
The Bad
Temporal
Can't completely erase temporal flickering while WASD moving,
but it did reduce a lot of flickering edges once the mouse starts to shake.
I mean, a shaking mouse reduced more flicker than when not shake the mouse.
Mouse acceleration
The game must follow mouse movement in real-time.
Some game enabled "mouse acceleration", this will cause very minor move vector to be ignored.
I owe you my thanks, all the shader plugin developers guys.
The Final Fantasy XIV's Yoshida dev teams didn't include any anti aliasing method other than FXAA,
and it doesn't seem going to happen in the future.
With your great help, I finally get my dreamt of anti-alising method what I wish,
which could running on a poor graphics card with high quality.
I suppose any other game has mouse controlled camera could also benefit from this pipeline.
For example The Elder Scrolls V Skyrim.
Although its SE version already got full support of TAA.
There are things I interested in, I use SMAA and your TAA, but with default options.
I see there are value sliders and dropdown menus in TAA settings,
but can't understand what are they supposed to effect.
Later I'll record videos and upload to YouTube, so you can see how the pipeline to actually work.
But anyway, "How the end user(me) use your program" LOL
You:
If it works. All good.
In games for TAA we Jitter camera projection matrix. So it's independent of mouse movement. I guess if we had a cam mod. We can solve this issue. So we can get a lot of different angles.
What you are doing should work. It's just not Idea.
A video demo to show results.