Beam settings
andersstorhaug opened this issue · 16 comments
First off, great work with this port. I realize that this may be a sore subject at this point.
So, scanlines. When I'm able to achieve VSync, interlacing + scanline thickness works pretty well.
When I can't achieve VSync due to slowness or lack of VSync support, I can resort to scanline blend. However this produces comb artifacts with movement, much like traditional video deinterlacing would do. For example Model 2 Emulator only renders at max 58 FPS, and I effectively have to use a scanline blend of 1.0 to remove stuttering, which naturally produces an image with heavy combing artifacts. I understand that is pretty much unavoidable.
With Royale in RetroArch I think you could disable interlacing and adjust beam min/max sigma to produce scanlines in cases like this. But as I understand, beam settings have been removed in this port to support the interlacing-based scanline effect.
Referencing the EMU-GEN wiki, CRT-Royale:
Min and max sigma settings affect the size of each scan line relative to the brightness of the image. A large range of beam sigmas causes the scanlines to vary in width depending on brightness, while small ranges result in a less variable width of scanlines. Higher sigmas increase the size of the scan lines, resulting in smaller gaps and more vertical blurring.
I tried to check out an older version that had beam settings (677c006) to play with, though, it seems that at least at that point adjusting the beam settings doesn't give the expected result. But with Royale in RetroArch, setting the beam sigmas to lower values does produce more pronounced scanlines (with interlacing disabled).
With all of that said.. any thoughts on supporting beam settings?
I guess one thing that I didn't realize is that, supposedly, emulators typically handle interlaced output by showing both even & odd fields twice. In that case the scanline blending impl also makes a lot of sense and combing artifacts would be minimal.
I am most frustrated by the hard binding to FPS. There are games that are fundamentally incapable of working at 60 fps, but use 25 or 30. Thus, nothing can be done. Is there really nothing you can do about it?
For the beam settings to work, it seems that we'd need to know the native (/unscaled) image height of the source/emulator?
With ReShade of course all that we know is BUFFER_WIDTH and BUFFER_HEIGHT. And then with this shader there is also content_size.
If there were additional parameters to supply the source height/lines (corresponding to RetroArch's SourceSize.y) would that allow for the beam rendering to be ported over? Say if it were called SOURCE_HEIGHT semantically I think it would mean, that the image in the content area was scaled from a source image with the given height.
I realize that this can be dynamic and even switch at runtime, depending on the emulator and/or content, so we may not be able to ever 100% mimic RA's SourceSize. But, it would seem that there could be a parameter-based (or macro-based) solution that would work most of the time..
My hope is that the author shows up later with a new update, or someone takes from here and continues development, its easy to replicate most of the presets from retroarch, but lines and blur needs work.
Plus, this shader has tons of potential, even more than retroarch when it comes even to old 3d games on wrappers like nglide, dgvoodoo, dxgi or Dosbox builds like ECE and dosbox-X.
Also i think reshade can replicate SourceSize, all it takes is Crosire explaining how at the forums.
I think this is actually the next piece I'm going to try to rebuild. I ended up nuking a lot of the gaussian logic previously because I couldn't get them to behave properly. Now that I've done some refactoring, I think I'll be able to give this a shot again.
Also, @Screamerofstars, it won't be possible to replicate SourceSize in reshade. If I'm remembering its meaning correctly, it refers to the original unaltered size of the console's video output, e.g. 480 pixels high. In libretro, the rescaling and upsampling is baked into the shaders; so it's possible to build shaders that know both the console's original dimensions and also the target dimensions. Unfortunately, reshade sits on top of the program you use it on; so it has no visibility into any emulator's rescaling logic. By the time reshade sees the image, it's already 1080p or 720p or 4k or whatever size the rendered window is at the moment. That original 480 value is irretrievable unless the user provides it manually.
Thankfully the rules for working around this are actually fairly simple. First, set your emulator to use integer scaling with your desired aspect ratio. Then choose your scaling factor, such as x1, x2, x3, x4, etc. Then set your scanline thickness to 1, 2, 3, 4, etc. The combination of integer scaling and matching the scanline thickness to the scaling factor ensures that crt-royale's interlacing should line up with the game's.
Also, I'm wanting to figure out a way to deal with the combing artifacts. They were a characteristic of the original crt-royale, so any methodology I find will be optional; but I do want to find something for that. My latest update included a refactor that'll let me add that option as one of several deinterlacing algorithms, but I'm still trying to figure out one that doesn't have combing.
Alright, check out the branch feature/gaussian-scanlines to see my first pass on adding that back in. I can't do a comparison against the libretro version right now, but I think it's pretty neat.
Toggle between the two behaviors with the "Beam Shape Mode" setting. Digital is the previous behavior where scanlines were discrete and had constant power and shape. Gaussian is the "new" behavior where the brightness is calculated based on vertical distance from the nearest in-field scanline. I also added back in the horizontal interpolation effect of the electron beam.
Feel free to provide feedback on this feature. This is venturing off into details that I don't know as much about.
That's a pretty huge improvement.
It looks like some of the scanlines kind of bleed into another though? I'm not sure how to describe what I'm seeing so here's an exagerrated example:
There are some harsh edges where it looks like some blur intersects with the scanline instead of fading maybe? Most obvious on the headlights.
You're right. I've got a bug somewhere.
Technically I have two, because I already found one; but that hasn't fixed it yet.
Found it! I multiplied a thing that didn't need multiplying as it turns out.
Next steps in no particular order:
- Get the Min Shape and Max Shape parameters reintegrated.
- Make it harder to induce clipping between scanlines. This will prooooobably require calculating components from the two nearest in-field scanlines rather than just the nearest.
- Figure out what's going on with the blur effect in the out-of-field scanlines. It doesn't pass the smell test.
Yeah the clipping between lines is a bit of a regression from the previous version.
224 native vertical image upscaled to 1792:
No interlacing, scanlines set to 8:
Static interlacing, scanlines set to 4:
With no interlacing you can see the white of a lot of the text bleeding into the line below. With static interlacing, the center character select borders start bleeding into the surrounding lines.
I think a part of that was the beam shape logic not accounting for the enable_interlacing setting. I just made a tweak that corrects the vertical distances when you toggle interlacing. In theory, that means toggling interlacing shouldn't require drastic changes to the beam settings?
I can replicate a similar effect on my end, but I don't think it's the effect you're seeing. I can induce it on my end by choosing a scanline width that's badly misaligned with the viewport's scaling factor. But that's an expected behavior, and your scanline width is aligned correctly. This quirk will probably have to wait until next week when I can test on a bigger monitor.
As a side note, the blur effect I mentioned is, surprise, an artifact of the blur used for bloom. I don't know why it clips at scanline boundaries, though.
I added a new gaussian beam shape that looks at neighboring scanlines too, so you can make the beams wider without hitting an edge as easily. It probably won't make a difference in most cases, but I'm sure there's someone out there who will appreciate it.
I also fixed the weird diffusion/halation behavior. The original crt-royale filled color into the out-of-field scanlines to avoid artifacts in its blurring stages, but that in turn was introducing artifacts in the diffusion/halation stage. As a bonus, the correction yielded a hefty performance boost for gargantuan scanline widths.
This looks excellent now. Really can't overstate what a difference this makes.
@MilkManEX I got PCSX2 set up and started investigating the interlacing bleed. I think it might be something specific to PCSX2? It's absolutely bizarre, but I'm seeing artifacts in the crop-pass output that are dependent on the beam shape parameter. It's as if there's a feedback loop from the beam pass back to the cropping, except that shouldn't be possible in DX11.
I'll keep digging into it. There's definitely something weird going on.
I haven't tried using it with PCSX2 yet, those images were from the Steam release of KOF '98UM. I'll try out the new build tonight or tomorrow and see if the old issues still crop up.
Some interesting results, hugely improved from previous builds. These are on a 4K display running with integer scaling at 3184x1792 for 1:1 vertical pixel scaling of a 320x244 source.
Static interlacing, thickness 4, backfield first unchecked
Static interlacing, thickness 4, backfield first checked
No interlacing, thickness 8, backfield first unchecked
The first example is perfect, zero problems (besides being over-bright, which I just hadn't gotten to yet). The 2nd image should look the same as the first if I understand it correctly, but it's still showing some of that scanline ghosting above the top row of portraits and the text at the top. Turning interlacing off and doubling the thickness yields a result similar to the 2nd image.