Monte Carlo path tracing
Closed this issue · 1 comments
First of all, awesome job :)
Do you plan to integrate Monte Carlo path tracing? I am working on a JS library for medical library and I am starting to investigate "cinematic" rendering. It aims to provide very realistic visualizations of datasets. It seems pretty advanced and I hope to be able to leverage some existing libraries such as yours.
The illumination model relies on Monte Carlo path tracing:
https://arxiv.org/pdf/1605.02029
Regular ray marching volume rendering:
https://fnndsc.github.io/ami/#vr_singlepass
With a more complex illumination setup, we could achieve very realistic rendering (performance is not an issue).
Please let me know if this question is out of scope or if you already solved a similar problem or if it sounds interesting challenge to you!
Hi @NicolasRannou
Thank you for the compliment and the question! Actually path tracers, including mine, all rely on Monte Carlo estimation. I don't know how much you already know about rendering so forgive me if I mention things you are already aware of. Path tracing is an evolution of Ray tracing in that we start with each pixel on the screen and follow the light ray that illuminates that pixel all the way backwards into the scene until we find a light source that it may have originated from. Ray tracing typically only deals with perfectly specular surfaces, either reflective (mirrors, etc.), or refractive (glass, etc.). Therefore ray tracing usually doesn't involve any estimation as the ray directions and their contributions are mathematically predictable and follow the laws of optics.
The key difference is when trying to calculate color for diffuse surfaces (painted walls of a room, bones, etc.). The traditional ray tracer fails in this regard because it only calculates one bounce direction for diffuse surfaces - it does not take into account other rays' contributions that went into that spot as well, that give us the nice warm blended color associated with matte/diffuse surfaces. In order to correctly render a small point on that type of surface, we need to integrate all possible ray contributions coming into its hemisphere (oriented along the normal pointing outward from the surface). This is computationally impossible because there are an infinite number of rays we can choose from, and each surface where those rays come from in turn presents the same problem again and again, making it hopelessly multidimensional.
So path tracers employ Monte Carlo integration to try and estimate the summed infinite rays' contributions to the diffuse surface spot in question. When the tracer hits a diffuse surface, it randomly chooses a path to follow through the rest of the scene. The first estimate (as in other Monte Carlo examples of estimating PI by throwing darts at a circular target inside a square border and getting the ratio of hit inside the circle vs. miss outside the circle), may be wildly inaccurate. But the power of Monte Carlo method is in the repetition of the random sampling and averaging the results by the number of tries. Each random sample gets you closer to the real answer - towards PI, or towards a photographic and physically correct rendering in the case of path tracing. This is why all path tracers initially display noise on diffuse surfaces - not enough samples have been taken yet to give a close estimate of the true color of the surface in question. De-noising is an active area of research by universities, NVidia, OTOY, etc.
The other active area of research is in speeding up the tracing process which, by itself, is super slow. Which brings me to my answer regarding your project of rendering medical data. My path tracer is good at small sets of data, say up to 10 objects. I have not yet completed a working BVH acceleration structure to handle large sets of data (triangles of a model, or point clouds, etc.). The aim of my project is to have real time path tracing (although noisy) for interactive web applications. I haven't really developed it with ground truth quality rendering (while you wait) in mind. I'm assuming the medical data sets that you'd be working with have many more than 10 or so elements (triangles, points) to display. Although I really enjoyed reading through the paper you linked to (those path traced renderings of the skeletal and muscular system are awesome! - what a great idea), I doubt my little tracer could even load a data set of that magnitude without crashing the browser at this juncture.
If real-time performance is not a concern, you might enjoy rendering the data with traditional renderers like Cycles inside of Blender, or Octane inside Cinema4D. These path tracers can usually handle any size data you throw at them, assuming your medical data is triangle-models (like an .obj file), or a point set, or volume set. Those renderers also use ray marching so you can render volumes like artificially-lit blood in the human system, brain tissue, etc.
I hope I have clarified some of the goals of my project and what you could possibly look into for your medical rendering project. If anything is still unclear, please let me know.
Thanks and good luck with your project!
-Erich