Fresnel model
niklasharrysson opened this issue · 27 comments
At the Physically Based Shading course at Siggraph this year Naty Hoffman presented issues with established Fresnel models. Both how it's incorrect to use the physical Fresnel equations directly in an RGB renderer, and also how Gulbrandsens method for making the parameterization artist friendly in fact fails at this on several counts.
slides
In addition Laurent Belcour presented a new Fresnel model, that is more accurate than Schlick's approximation and still works well for both offline and real-time rendering.
slides & code
Since Standard Surface is using Gulbrandsens methods, and definitely needs to be usable and accurate for non-spectral renderers, this is something we need to look into. Should we move away from Gulbrandsens method? And if so, what should we use instead?
Feedback from the rendering community is much appreciated!
Many thanks for starting the discussion Niklas!
There are many questions that needs answers from the community to be able to nail the specifications of a good 'standard' Fresnel model. Here are a couple of points I found interesting:
- Are we (meaning the rendering community) interested in having grazing angle control? That could mean either producing goniochromatism (without interference effects) for cheap, or reproducing measured IORs materials, or other reason. I like the ability to tune it, but I don't know what artists think of this.
- Are we interested in having a 'standard' Fresnel model simpler (in the sense of Schlick, parametrized by R0 only) but closer to specific reflectances (for example matching specific dieletrics for low R0 but matching specific conductors for high R0).
For grazing angle control, from our side I would say yes absolutely. When it comes to Standard Surface we have had the edge tint control for a while now and I think artist will expect this to remain controllable in some form. But since we use Gulbrandsen now, it is not really artist friendly, for the reasons Naty points out. And in addition with the reparameterization into the full physical equations we end up using that in our non-spectral/RGB renderers, so it's a problem with accuracy as well (as pointed out by Naty).
I think ideally, on our side, we would like to keep the F0/F90 parameterization (with F90 I here mean "some" type of gracing control). But having F90 more intuitive and linear, and solve the inaccuracy caused by using single eta/kappa values in an RGB renderer.
Naty's conclusion was that going back to Schlick is the best choice in most cases. Just using generalized Schlick (F0/F90) would then be the simplest choice. But it gives the artist more freedom (maybe to much?) to produce non-physical results? For full control we would also need to expose the additional 'exponent' parameter. What are peoples experience with generalized Schlick? I know it's used in several other shading models, commercial and in-house.
+1 to needing the grazing angle control. I would even argue that having a third parameter to control the transition between the two colors is also important in many cases.
For a bit of context from the Imageworks side, we started with the Gulbrandsen parameterization because we happened to be revamping our bsdf model around the same time the paper came out. After the model was pushed into production, one of our more vocal artists lobbied strongly for us adding a "generalized Schlick" artistic mode: F0+(F90-F0)*(1-cos(theta))^p. The arguments were similar to the ones Naty made in his talk. The most convincing one for me being that the fresnel curve is actually a place we can capture other aspects of the physics than the raw fresnel equations allow. As Naty alluded to, real materials used on set in VFX production are rarely (never?) pure metals but instead layers of metallic paint on top of lighter materials. These interactions would be super complex to model physically. However since we had already rolled out the other model, we had to leave it as our default - and we haven't revisited this since (the curse of backwards compatibility).
I've asked to see if we can crawl our in house material libraries to see how often the artists deviate from defaults (this is a bit involved to do well given the flexible nature of material inheritance and overrides, etc ...). I'll try to report back if we can come up with anything quantitative.
Just to summarize the models under consideration for the thread:
- Vanilla Schlick (R0 only)
- Generalized Schlick (with power control above) (R0, R90, p)
- Naty's model from his MAM talk (R0, R82)
- Gulbandsen model (R0, g)
- Laurent's model (which can fit several of the ones above from what I understand, including being a way to reparameterize the Gulbrandsen model)
Any others we should be thinking about? Many years ago, the Maxwell renderer added a specific control to their bsdf model for very similar reasons [1]. From my recollection of lurking on their forums in those days, I believe this was also an artist driven request that came from a desire to match painted props.
[1] https://nextlimitsupport.atlassian.net/wiki/spaces/maxwell/pages/22684122/Custom+Fresnel+curve+-+r2
I wanted to start by flagged that I haven't had the chance to dig deep into Laurent Belcour proposal but from what I've seen of the slides, it looks very interesting and it could present a great solution for the various opinions, including mine that i've posted underneath.
To me the fundamental question is what are we trying to achieve artistically. In VFX, we're trying to get as close as possible to the real world but with enough flexibility to create the impossible. For that reason, I would expect any reference shader (and therefore its fresnel parametrization) to use a method that guarantee the plausibility of the material over its absolute realism.
In that context, I believe that moving away from the full fresnel equation would be a mistake and here are some point why :
- Schlick Fresnel is never beneficial in a dielectric context. It uses the same artistic friendly parametrization (R0) as a full dielectric fresnel but without the accuracy. Considering that the vast majority of materials are dielectric materials, it seems a bit silly to me to reduce the dielectric materials accuracy for a few conductor materials.
- Generalized Schlick edge controls (R90) and any similar approach (R82) very often results in non plausible material due to many uninformed artists creating materials with lesser reflectance towards the grazing angle than towards the incident angle.
- Making a decision on Fresnel based on a small set of pure material scan is not beneficial against the vast majority of completely random materials created by artists. It also important to acknowledge that 'reference' scanned material are only as good as the scan itself. This is also why there is a substantial amount of variation in the scan data depending on the source of the data.
- An approximation will always be an approximation. We have the full equations, I would vouch for more research on how to improve its artistic parametrization instead of reverting to an approximation.
For those reasons, I would defend the Guldbrandsen parametrization (or any improvement over it, if it exists) because
- the Gulbrandsen parametrization associated with a full conductor fresnel guarantees a plausible material (even if it doesn't mean that you've created a real material).
- the guldbrandsen parametrization is artist friendly enough and linear enough : an edgeTint of [0,0,0] will result in a fallback to a colored dielectric fresnel reflection and any value above that will progressively increase the reflectance towards grazing angle. It will also tint the color towards grazing angle as long as you don't contradict the color provided by R0 (which is a good behavior).
- the full fresnel conductor parametrized by eta/kappa (and therefore by a mapping to guldbrandsen) can be accurate to a scan, as long as you apply the correct conversion to the spectral data (as demonstrated by Naty).
Ultimately I think we're discussing a non issue. Regardless of the method used, I've never had a problem with Fresnel as an artist over the last 15 years in VFX. I've used Schlick, Generalized Schlick and Guldbrandsen for years and in the hands of a good artist the difference is negligible but in the hands of an uninformed artist, I've always encounter better, more realistic results from the "constraint" that Guldbrandsen pushes on people.
I think it is great that Naty started the conversation, but I don't buy his conclusions. I find @belcour is going in a more interesting direction, especially with the Gulbrandsen re-parameterization.
My experience totally matches @noj86's post.
Speed, parameterization and accuracy are important but they go hand in hand with physical plausibility constraints.
I often joked about revoking someone's look-dev license after having debugged their implausible materials. These errors cost a lot of time and money in VFX and moving away from physical plausibility would clearly be a step backward.
Does anybody see how to use any of Schlick's, Gulbrandsen's, Hoffman's, or @belcour's model for a material inside some dielectric medium?
I understand that it could be solved by additional artistic control, but there are models (especially, @belcour's thin film that is also used in Standard Surface) that need access to underlying real/complex IORs.
Maybe I should exclude Gulbrandsen's model from my question, because his article contains transformations from (R0, g) to (n, k), so we can use these IORs (they will be "averaged IORs" in some color space).
Thanks for the excellent feedback so far!
@fpsunflower some quantitative results of what your artists prefer using would be very interesting.
@noj86 @pleprince, I fully agree we need constraints to stay physically plausible, and I think everyone agree we should not move away from this. Parameters added for artistic control should have good defaults and preferably be protected by chains and locks in the UI.
I think the unification of a Fresnel parameterization for real-time vs offline is the most interesting problem to solve. As long as the parameterization is the same everyone can choose what to do internally. With Gulbrandsen we can transform back and forth to physical IORs. @nickkuk made a good point that we still need this in the case of thin-film interference. @belcour how does this hold up with the Gulbrandsen re-parameterization proposed in your work? Admittedly I have not had a chance to deep dive into it yet either.
@nickkuk You should be careful when using Gulbransen's parametrization when your ambient medium does not have an IOR of 1, since this is a basic assumption made in Gulbransen's paper.
If for instance you assume that your metallic material is coated with a dielectric layer (to make things simpler, let's assume it's not a thin film), then not only will the reflectivity r should differ due to Fresnel transmittances when going from air to the coating medium; but the effect of the edge tint will also be reduced for two reasons. First, at grazing angles, most of the radiance will be reflected by the first interface, hence reducing the usefulness of the edge tint parameter. Second, Snell's law of refraction will make incoming directions less grazing, so that rays will be less likely to reach grazing angles on the second interface...
As far as I know, there has not been much work done on that specific question (I mean in academic research), even though coated metallic materials seem to be useful to model.
@pbarla, thanks! I see the complexity of the problem.
What can you say about "larger medium", e.g., metallic material under water? On a such scale we haven't any light transfer problems in a path tracer - only local Fresnel reflections are in question.
@nickkuk In a larger medium such as water, there is still another issue I forgot to mention: Gulbransen's approach relies on a formulation of Fresnel equations that is exact only for an ambient medium with an IOR of 1. Hence even if you re-derive the approach for a different ambient medium, you will not get a physically exact solution...
I specifically talk about adapting Schlick for external media in my SIGGRAPH 2020 talk. It is possible with a very simple adjustment which gives good accuracy for water and is less accurate for higher-IOR external media (e.g. diamond). I also present an alternative which is more accurate for high-IOR external media but has some other drawbacks. See slides 106-122 here: https://blog.selfshadow.com/publications/s2020-shading-course/hoffman/s2020_pbs_hoffman_slides.pdf
Also regarding having constraints to stay physically plausible, yes, that's the whole point of going with physically motivated models to begin with! This is something I've been advocating since 2004. But in my opinion Gulbrandsen is actually much worse for this than plain Schlick. With Schlick you can set F0 to whatever you like and you will always get a plausible material. Even values between 0.2-0.4 correspond to semiconductors. But with Gulbrandsen it is difficult to set edgeTint so that the result is plausible. Setting edgeTint [0,0,0] will not result in "a fallback to a colored dielectric fresnel reflection", it will (as I clearly show in slides 137-142) result in a bizarre darkening and color-shift at the edge. I suspect that the reason that people say Gulbrandsen is fine is that a lot of these edge effects are barely noticeable due to foreshortening, but if you don't care about edge behavior, why not just stay with Schlick?
And regarding the benefits of Schlick Fresnel for dielectrics, the big benefit here is being able to handle the entire continuum of materials from dielectrics to semiconductors to conductors within a single specular node.
Regarding F90, yes it is non-physical because it doesn't have the "post-dip rise" to white. But the same foreshortening that squeezes the dip itself into a handful of pixels tends to squeeze that "post-dip rise" to nothing. Here's a close-up of a shaderball edge with Gulbrandsen g set to red. The shift to red is very clear, but I can't see any shift from red to white at the edge. There are probably configurations where that is more visible (maybe a large plane seen edge-on?), and I would love to see renders where the "post-dip rise" is visible if anyone has them.
I just noticed that the slide PDF link I attached earlier redirects to the course page, so it's probably best to go to the course page (https://blog.selfshadow.com/publications/s2020-shading-course) and click on the "slides" link under "Some Thoughts on the Fresnel Term". The presentation video will also be available on that site in the near future, for anyone without a SIGGRAPH registration.
And it's important to note that there are non-Fresnel surfaces (e.g. fine loose dust) for which F90 enables a more physically accurate rendition than the Fresnel equations (e.g. Gulbrandsen).
The one big advantage F90 has over other edge-tint controls like Gulbrandsen "g" or my F82 is that it is trivially "opt out". Leave it at [1,1,1] and forget about it. Maybe bury that texture channel option in an "Advanced" tab or something so less experienced artists are less likely to use it.
@niklasharrysson regarding the re-parametrization: we still maintain the conversion to (eta, kappa) using the reparametrized Gulbrandsen. Each configuration (r,beta) maps to a valid (r,g). Hence we can get back the (eta, kappa). It only requires to stores this mapping. We had no need for it in our demo, so we didn't include it. But it is possible. Any reparametrization of Gulbrandsen will allow to get back to an (eta,kappa) pair, as long as it is contrained to (r,g) in [0,1]^2.
A small note regarding the analysis presented so far.
Since we only consider the case of "natural" (unpolarized) light, Fresnel reflectance is computed as the average of the parallel and perpendicular polarization components. The resulting formula is rather complex. Formulas for a particular polarization are comparatively simple. Additionally, the behavior of the two components is rather different as they approach the grazing angle. Specifically, at Brewster's angle, the parallel (reflection) component reaches its minimum value (which is 0 for dielectrics) while the perpendicular component does not. So it may be worthwhile to analyze both components separately (which does not necessarily mean that we need individual parameters for each).
Another point made earlier that I wanted to address: "An approximation will always be an approximation. We have the full equations, I would vouch for more research on how to improve its artistic parametrization instead of reverting to an approximation.". This is also addressed in my SIGGRAPH talk. I showed quite clearly (in slides 35-48) that the Gulbrandsen model is as much an approximation as Schlick is, since Gulbrandsen is based on evaluating the Fresnel equations on RGB data which is incorrect. In fact, Gulbrandsen is a less well-founded approximation than Schlick. Schlick is based on the formulation that was recommended in the Cook-Torrance paper for applying Fresnel in RGB renderers (as I show in slides 28-36 and 50-51).
Generalized Schlick was (still is?) what has been in use at ILM, for as long as I can remember (1993 to 2010). Obviously, by default, the edge (F90) color was set to white and the exponent to 5.
It is also the fresnel model we ended up putting in the shaders at Pixar, including for RIS on Finding Dory (also later in PxrSurface, where optionally a physical fresnel could be activated in its place).
This led several times to interesting discussions as to what to set specularEdgeColor (F90) when a material layer was represented as a complex blend between two specular lobes (remember that we offered specular and roughSpecular, both considered to represent a single layer located under clearcoat and above all diffuse components - see Pixar's Foundation for Materials Sig 2017).
I believe some shows decided to put the two EdgeColors (specularEdgeColor and roughSpecularEdgeColor) at (0.5,0.5,0.5), so that the resulting combined complex lobe would sum to (1,1,1) on edge, while others opted for white on both. Separately, energy conservation to the diffuse substrates was then enforced via an explicit tuning knob (specularEnergyCompensation).
Btw, I believe the physical fresnel option was never employed within Pixar's productions (at least till I left in 2019), technical directors favoring largely the direct controls of the Schlick artistic fresnel.
@natyh I believe that the full conductor fresnel equation with a colored eta (which can be converted from the colored R0 in the Guldbrandsen parametrization) and a kappa value of [0,0,0] (which is what a Gulbrandsen's edgeTint at [0,0,0] would produce) does match the reflection of a full dielectric fresnel with the same colored eta. For me, the comparisons in your slides are slightly misleading because you're comparing a badly parametrized conductor to the conductor ground truth but not to the equivalent cg material which uses a colored dielectric full fresnel. (which is what I stated would match in my previous post)
The full conductor fresnel equation expects a kappa value to produce a "valid" metal surface so it's logical that without this value it won't behave properly. I acknowledge that if we're using only R0 in a Schlick fresnel the results will be closer to the ground truth for most pure metal surface but once again we're overlooking the loss of accuracy for all the non-metallic material.
For me there is two questions, the one of the model at the core and the one of the parametrization. Here is a quick summary of how I perceive the various cases :
Schlick
- Artistic Parameterization : R0 is intuitive and easy to use. It is the expected default value for this method.
- Physical Parameterization : IOR which can easily be turn into a matching R0
- Core Result: Dielectric: Good approximation to the ground truth
- Core Result: Conductor: Good approximation to the ground truth
Full Fresnel Dielectric
- Artistic Parameterization : R0 is intuitive and easy to use ; this value is converted to the matching IOR.
- Physical Parameterization : IOR. It is the default expected value for this method.
- Core Resulted: Match the ground truth
Full Fresnel Conductor
- Physical Parameterization : eta + kappa, difficult to use and not intuitive. Ideally it should use spectral value.
- Artistic Parameterization : Guldbrandsen's R0 + EdgeTint produces an eta/kappa values. Intuitive and easy to create a metallic looking surface but not easy to create a match to the ground truth and as it was pointed out by Naty, it's also simple to produce less matching metals to the ground truth than the equivalent R0 used with Schlick.
- Core Result: Plausible material since the equation is the ground truth but when it's badly parametrize it can quickly results in further away look to the ground truth than the equivalent material using schlick and R0 only. If well parametrize you can get a match to the ground truth but this isn't simple.
Overall, to me, on the question of the core model, it seems more valuable to have an accurate dielectric model and an accurate conductor model even if some improvements on the artistic parameterization are still required, compared to a model that will stay an approximation (even if it's a very good one) forever. As I stated earlier, if you have the "correct" base math, you systematically produce a plausible material even if it isn't one that perfectly matches a real scan.
On the question of the parameterization, there is no doubt that physical and artistic parameterization will continue to co-exist. I think R0 is a great artistic solution and doesn't need to be changed. That being said on a personal level, for dielectric surface, I think of material in terms of Index of Refraction first.
For edge controls in metals, R90, R82, EdgeTint all have their sets of complexion. I've seen a lot of problematic asset in the past over-using (badly) R90 and fresnel power controls which is what lead me to favor any parameterization that prevents non-plausible behavior. This is why, so far, I prefer EdgeTint over the other two others. I think @belcour approach is improving on that parameterization too.
When it comes to the preference between a physical or an artistic approach it really depends on your background.
I'm making some generalities but most experienced lookdev/shading artists at MPC, Framestore, Dneg, Cinesite will favor a physical set of values because this is what they're historically used to. At ILM, as @cherycode pointed out, a lot of artist will prefer the artistic method - but not all of them.
I'm not a rendering engineer so I hope that I'm not missing some obvious points in all your posts but ultimately I think that the difference between the various options are very small once in the hands of good artists.
The course presentation videos are now up: https://blog.selfshadow.com/publications/s2020-shading-course/
@noj86 - it's true that from a mathematical perspective using Gulbrandsen with edgeTint = [0,0,0] produces the same result as using the dielectric equations with the same reflectance value. But that's not very relevant for the question of how artist-friendly edgeTint is and how useful edgeTint = [0,0,0] is as a default value. From a look dev perspective, a metal with edgeTint = [0,0,0] doesn't look anything like a bright colored dielectric, it looks in fact the opposite - like a kind of "super-metal" with a bigger dip than real metals.
Regarding the "Gulbransen/accurate" vs. "Schlick /approximate" division, the point I'm trying to make is that it's a false division. Both are approximate, and in some important senses Schlick is more "correct" and in practice it leads to more accurate results (yes, it's possible to get more accurate results with Gulbrandsen if you do a numerical fitting to measured data, but in practice edgeTint will be painted "by eyeball" in which case it will usually be less accurate).
I do agree with you that a good artist can produce good results with any of these models. But I think that some of these models will take more work to produce such results than others. And then there are other considerations like preserving the look under blending and color conversion operations, performance, real-time friendliness (more important than ever in the age of real-time in-camera effects using systems like ILM StageCraft), etc.
While it doesn't tick all of the artist-friendly checkboxes, I would note that it is possible to color correct, do spectral projections, and exchange materials between different 2-parameter Fresnel models by using F0 and the spherical/bond albedo of the material as the two color parameters for the Fresnel equation. This should work better than [F0,F82] because the dip isn't always strongest at 82 degrees, and better than [F0,g] for the reasons Naty has mentioned. The spherical albedo is simply the integral of the reflectance over the hemisphere (i.e. the average pixel value of a spherical particle in a unit white IBL).
Hi there !
@fpsunflower told me that he couldn't finish the data collection on Fresnel usage he wanted to share here (see #12 (comment)) before leaving Imageworks. Is anyone with such data at hand (or close to) would be willing to take over Christopher's idea?