Depth fade is not compatible with outline effects

[pink-arcana]

As of v0.2.0, depth fade works with normal outlines, but is not compatible with most outline effects. An exception is Subpixel AA, which will work with "width fade" but not "alpha fade".

Why it doesn't work: During the JFA passes, when depth fade is enabled, we remove outline seeds that are obscured by the outlines of seeds closer to the camera than them. Doing this allows the outlines at different depths to intersect appropriately, but disrupts the integrity of the distance field itself. The final width cutoff distances are still accurate, but the intermediate distance values (which we rely on for effects) aren't.

Within the JFA passes, I haven't found a way to retain the integrity of the outline effects without worsening the appearance of the depth fade. It may be possible to fix the distance field with additional shader passes. Once we know which seeds to keep, we should be able to re-create the distance field using only those seeds. But I haven't tested that.

Fixing this may not be that useful. Normal outlines without effects are the primary use case of depth fade (and this project in general, probably), and normal outlines seem to work well with the current algorithm.

In my thinking, the big use case for outline effects is to highlight individual objects, likely using an SDF. You may still want depth fade for that. But, in that case, the object's depth could be used to determine the outline width, and a screen-space depth fade like we have here would not be needed.

For this reason, I'm adding a "not planned" label. But I'll keep the issue open so people can find it and comment.

A few diagrams explaining the dilemma.