Add a visualization of the aabb cube
Closed this issue · 7 comments
if the aabb volume is set to 1, is that a cube of 1x1x1? why is the default 4? It would be good to have a visual representation of the aabb box to understand what it is.
The aabb bounding box is a metric that was introduced in Instant NGP to bound the volume region in which the NeRF rays are scattered. I am however unsure on how they exactly implement it, and would particularly have to pay attention to the conversion from Blender units to NGP units (they might be the same).
As far as I understand, a larger aabb does not actually penalise the scene quality by much, but it does increase rendering time (according to my own limited experience of playing with the NGP code). The reason for having it as a default of 4 is therefore for a good trade off between scene size and rendering speed.
Yeah, it's a little confusing what the aabb box is, but it appears to be the volume within which the scene is calculated, so if your scene is larger than 1x1x1 in iNGP units (no clue how they relate to real-world units) then you need to increase the aabb multiplier. In my very limited testing so far it seems like aabb 1 = Blender 1m x 1m x 1m. It might be safe to do a visualization cube that scales up and down with your aabb multiplier?
Yes you might be correct. However I would rather avoid introducing potentially erroneous code in the add-on, in order to avoid any future confusions. Additionally, adding an aabb cube might be a simple task for the COS method, but the SOF and TTC methods rely on cameras with free range of motion. Where would one then have the centre of the cube?
But if the NGP aabb bounding box indeed shares the same metric and units as in Blender, then I believe it is safe to assume that everyone is capable of creating a cube of size aabb * aabb * aabb m3 within Blender. For optimal quality one should then pay attention to have the scene fully fit within this cube, and adapt the BlenderNeRF aabb parameter to do so as tightly as possible.
The aabb cube is at the origin of your scene since the camera positions in the JSON file are relative to 0,0,0, so it's safe to use a cube at the origin and I THINK it is safe to assume 1 Blender Unit = aabb 1... Requires more testing. I just know that a NeRF whose cameras are scaled for a scene that fits inside a 1x1x1 cube is clipped in iNGP when aabb is set to the default of 4, but looks as expected when aabb=1.
Ah yes of course, the origin for the aabb bounding box will be the same as the one for the cameras :)
And could you elaborate on what you mean with the clipped scene in NGP? I have done experiments with a scene fully fitting within a 1 m3 cube, while having set aabb = 4. The rendered NGP results looked correct. In fact, the donut scene on the README fits into a 1 m3 Blender cube, but is rendered in NGP with aabb = 4.
I'll render an example of what I experienced if I can find the bad earlier versions. AFAIK if your scene fits inside 1x1x1 in Blender and aabb = 1 everything works great. Increasing the aabb just decreases the resolution inside the 1x1x1 by looking increasingly further outside that 1u cube, so an aabb=128 will be 1/128th the detail of the same NeRF at an aabb=1. It doesn't scale the scene of the NeRF, it scales the volume within which rays are traced. My earliest failures were because the Blender scene was much larger than 1u, so the only way to get it to fully process was increasing the aabb value, it was "clipped" in the training if I left it at 4.
I am not full sure, this depends on whether the 3D grid in which the NeRF samples points from grows with the aabb bounding box. If the grid resolution stays fix, then you are indeed right: the render quality will decrease with larger aabb.
However from my previous experiments, I remember that the rendering time increased with higher aabb, which suggests that the grid resolution increases with aabb. This correlation might not be linear though, since this might impact rendering time (and therefore training speed) too much for large bounding boxes. Perhaps the NGP authors were fine with sacrificing a bit of the local rendering quality when a larger scene is being rendered. To be verified in their code.
No matter what, it is clear that keeping the bounding volume as tight as possible to the scene scale will result in the best rendered NGP results. For that, any BlenderNeRF user can just create a cube of width aabb and fit the scene within this cube (assuming that NGP uses the same metric and units as Blender).