Update VTKOverlayWindow to have 5 layers instead of 3
Closed this issue · 3 comments
Currently, we have 3 rendering layers:
- level 0 = background video, as RGB
- level 1 = foreground VTK models
- level 2 = additional text annotations that must always be on top.
So, AR can be achieved by adding VTK actors to level 1, and setting opacity to <1. But this rather naive method always makes overlaid surfaces appear in front of the video rather than providing x-ray vision to see inside. So, for a tumour inside the liver, the liver and tumour BOTH appear IN FRONT of the video.
So, this Issue is to investigate 5 layers.
- level 0 = optional background video
- level 1 = 'internal' anatomy, like tumours and vessels inside a liver
- level 2 = video, which is rendered as RGBA, so when alpha channel < 1, appears transparent.
- level 3 = 'external' anatomy, like the liver surface, or any contours that need rendering on top of the video
- level 4 = additional text annotations that must always be on top.
So, after the updates above:
The default 3 layer renderer still works, showing a gradient image as the "video", overlaid with a transparent liver and transparent tumour. These pictures reuse VTK models already in other unit tests.
To illustrate the idea of having video in front of the models, and an optional mask, here the video is in layer 2, a mask is defined to make a transparent hole, and then the liver surface and liver tumour appear behind the video.
But for medical augmented reality, here we have:
- level 1 = internal anatomy, in this case a tumour
- level 2 = video, with a mask to create a hole in it - at the moment, it's a hard edge, but you could also blur it for a gradual edge.
- level 3 = external anatomy like the liver surface, additionally rendered as an outline
