Screen upscaling shaders improvements
fp64 opened this issue · 13 comments
What should happen
Actually, 2 issues rolled into one.
First, the current implementation of Spline36 seems to shift the display by 1/2 of (source-space) pixel toward upper-left. At least it does on my (admittedly convoluted) setup: win32 PPSSPP (tested on ppsspp-v1.12.3-1215-g3bf0c0088-windows-x86) under wine, using OpenGL (independent confirmation would be nice).
The fix seems to be replacing
vec2 inputPos = outputPos / u_texelDelta;
by
vec2 inputPos = outputPos / u_texelDelta - HALF_PIXEL;
in assets/shaders/upscale_spline36.fsh.
Second, it might be nice to have more screen upscaling shaders out-of-the-box. Currently this is either (not shader-based) bilinear, or Spline36 (not counting something like 5xBR).
Some form of bicubic (e.g. Catmull-Rom) screen upscaling might be nice, as Spline36 can be a bit slow on a sufficiently ancient machine (it is on mine).
In case anyone is interested, here are my implementations of Catmull-Rom and Mitchell-Netravali upscalers (.fsh only, since .vsh is the same as Spline36):
Catmull-Rom:
https://pastebin.com/2BKMG11Q
Mitchell-Netravali:
https://pastebin.com/vtdXgyVw
WARNING: not tested much, basically gutted Spline36 code.
I personally prefer Catmull-Rom visually (for upscaling, anyway).
I don't expect them to be better than Spline36 quality-wise.
Who would this benefit
Probably most people using upscaling shaders (albeit slightly).
People with older hardware.
Platform (if relevant)
No response
Games this would be useful in
Any.
Other emulators or software with a similar feature
No response
Checklist
- Check the latest git build in case it's already implemented.
- Search for other requests of the same feature.
Happy to accept pull requests with all of the mentioned changes!
You're right that not much time has been spent validating that spline36 actually does the right thing... However I think the main intended use of that one was actually quality downscaling for getting AA by running crazy high rendering resolutions, downscaling to your monitor. Maybe the half pixel offset didn't matter in that case, not sure (because it seems like it would...)
It's named "upscale_spline36", and the discussion in #4292 suggests the indended usage is 1x internal. Also doesn't PPSSPP now force 1x internal for upscaling shaders according to this? Empirically, it seems to still do that.
Just to be sure: pull request would amount to modifying assets/shaders/upscale_spline36.fsh and assets/shaders/defaultshaders.ini as well as adding .vsh and .fsh for both new shaders (Catmull-Rom and Mitchell-Netravali) and that's it, right?
Hm, I guess I'm misremembering, oops :/. There was some shader like that intended for downscaling, oh well.
Right, the upscaling shader idea is to take a PSP-original-size image and use 2D tricks to upscale it, similarly to SNES game upscalers and such. These are generally useless for 3D games, where you rather want to use texture upscaling and rendering at a higher resolution.
And indeed, a PR would just modify some shader files like that.
Let's see if I got this right.
Obviously, none of the 3 shaders (Spline36, Catmull-Rom, Mitchell-Netravali) are pixelart-aware (unlike 5xBR). They are just general-purpose scalers (and do not require exact integer scale ratio), and mostly a (usually) nicer-looking alternative to bilinear.
I think enabling (up/down-)scaling from internal resolutions other than 1x would make sense (if that's not too much trouble). It would allow people to do what you were talking about: get some AA out of downscaling from high internal resolution to monitor. Quality increase would probably be slight, but if people like it, why not?
For one thing, presently the GUI tells you wrong internal resolution with upscaling shaders (it shows whatever you set, not the 1x, that is actually happening).
Note: for downscaling, Mitchell-Netravali may actually look better than Catmull-Rom, at least according to some people. See e.g. this which discusses algorithm choice in stb_image_resize.h.
Gui showing resolution set by the user is correct since upscaling shaders only takes 1x render res as an input, they still output whatever you're using.
If you want to "enable" scaling from internal resolutions different than x1 just don't add "upscaling=true" to your shader. This flag exists because upscaling algorithms like xBR were not doing their job in any resolution outside x1 and enforcing it was the easiest workaround, however that's actually because they were written that way. My custom shaders include xBR/xBRZ version which not only scales at any resolution, it allows manual tweaks(useful for games ported from older platforms as their "pixel size" doesn't match PSP original res). I wrote it pretty much specially for street fighter game, since it can't work with texture scaling and at x1 render res it's too blurry to scale via xBR family scalers.
Edit:
Should probably add that to algorithms like xBR under typical circumstances forcing x1 res is better than allowing users to use it at any res since it's heavy algorithm that 99.9% of the time doesn't benefit at all from using higher input resolution and it just wastes power, typical PPSSPP user, especially mobile userbase isn't too tech sawwy and just abuse settings without much thinking, then complain on the issues they create.
That makes sense, but why does Spline36 also have Upscaling=True in defaultshaders.ini? And now shaders in my PR do too, since I just copied Spline36 portion of the .ini.
As for the GUI for internal resolution thing, I'm still not sure I have a correct mental model, even after your explanation. I now feel even stronger, that someone knowlegeable should just draw a diagram of PPSSPP's rendering pipeline and post it somewhere visible.
As for my PR. I opted for longer, more descriptive names, so now "Bicubic (Mitchell-Netravali) Upscaler" is the longest name on the list and just barely fits into the advanced options GUI (for chaining postprocess shaders, not sure what the proper name is).
It still might not be completly obvious from the names how they compare to Spline36 (they are faster, but lower qualilty).
I used "bicubic" rather than "cubic", even though some people feel like "cubic" is the proper terminology. PPSSPP already uses the term "bicubic" for texture upscaling, but it also uses just "linear" (not "bilinear") for (not shader-based) screen upscaling.
For super-cheap scaling there's also this, which is... peculiar. Can look surprisingly decent.
Spline36 also has upscaling=true, because with high res it was pretty much doing nothing on 2D graphics and it's name having "upscaler" it suggests it's an effect for use with 2D games. If it looses upscaling flag, it should probably be renamed as well.
For 3D games there's really no need for any alternative of "downscaling shader" as doesn't matter if you use the cheap FXAA version PPSSPP has or any other blurry effect, it'll provide similar anti-aliasting with increased res. Default SSAA on modern high res screens is already overkill and doesn't leave any jaggies, mobiles doesn't even need that due to pixel density on pretty much anything that's not an grandma phone.
In general having tons of effects providing similar results is just cluttering interface and makes user experience worse by adding choices which actually doesn't matter, which is why I never proposed my shader pack to be merged into PPSSPP, but also why my shader pack includes customizable options and tweaks within every shader instead of providing X versions of each similar algorithm.
I disagree somewhat that downscaling shaders don't make sense, if you run say 6xAA in a 2xAA sized window, regular interpolation will lose information that should contribute to the result. However it's generally not a big deal given the ever increasing screen resolutions and so on.
Well, I do think it's useful that they automatically set the resolution the way you probably want. Not that arbitrary scaling is wrong, but most users enabling xBRZ or honestly even Spline36 are probably looking for an upscale from 1x and might even be confused if we changed that now.
Additional shaders can be set without the upscale flag on (which btw also forces nearest sampling for that shader pass), so it is supported just not with the preloaded shaders.
Should we close this now that the Spline36 fix is merged and other shaders are added? Not trying to stop the conversation but I also don't want this to just sputter out and sit open with no new replies for a year until someone notices and finally closes it.
-[Unknown]
Thanks!
Yeah, even Spline36 "only" touches 36 input pixels for each output pixel, so downscaling by more than 6x (in both directions) means some input pixels do not contribute at all.
As for whether it's better to have "Upscaling=True" in the .ini (and "Upscaler" in the names), I leave the decision to you.
Can be closed, as far as I'm concerned.
I'm thinking that we should probably have some kind of visual split of regular post processing shaders and 1x-to-higher upscaling shaders (though still have the ability to choose either at the same place). But yeah, closing this.