Constructing the DRR problem

[linquanxu]

Hi, thank you very much for your open-source code. I am very interested in your work. Recently, I encountered a few confusions while running the code:

1 For example, when constructing the initial translation, why is by set to 850? translations = torch.tensor([[0.0, 850.0, 0.0]], device=device)
2 Is the coordinate system of the volume data LSP?
3 If I crop a section of L4 vertebra with dimensions (302, 264, 56) from a CT scan with dimensions (512, 512, 245) as the volume data, how should I set the initial posture? Is this posture based on the coordinate system of the volume data?
4 In this situation, do I need to set the principal point offset?

[eigenvivek]

Hi @linquanxu ,

1. For the SE(3) parameterization used in the example ("euler_angles" with ordering "ZXY"), the translation y=850.0 corresponds to the source to detector distance. That is, how far the is X-ray source from the isocenter of the volume. Here, y=850.0 means the X-ray source is 850 mm from the center of the chest CT.
2. The coordinate system is defined by the affine matrix of the volume.
3. Depends on what view you want, but yes, the coordinate system is based on the volume's affine.
4. You only need a principal point offset if your C-arm images were acquired with an offset. Some detector planes can move independent of the source. If your images were acquired with that setting, you need the offset. Otherwise, you can leave x0 = y0 = 0.

[linquanxu]

Hi @eigenvivek ，
Thank you for your prompt response. I encountered a case of DRR registration failure, and I am seeking further assistance.

1. I used TotalSegmentator to segment the spine (L1-L5 and T12) from the 3D CT DICOM data. The size of the CT data is (512, 512, 245).
   

2. I cropped out a section of the spine, such as L4. The size after cropping is (302, 264, 56).
   

3. I collected the corresponding 2D X-ray images of the patient and segmented L4.
   

4 I used the 2D X-ray as the ground truth for DRR registration. The initial configuration and some parameters are as follows.
The registration results are as follows:

SDD = 1085.6
HEIGHT = 300
DELX = 0.373

true_params = {
    "sdr": SDD,
    "alpha": 0.0,
    "beta": 0.0,
    "gamma": 0.0,
    "bx": 0.0,
    "by": 595.0,
    "bz": 0.0,
}

lr_rotations=5e-1, 
lr_translations=1e-1,

Why are the images generated by DRR so blurry?
Based on your experience, could you please advise if there are any aspects of my experimental steps or parameter settings that could be further optimized?

[eigenvivek]

A few things stand out:

1. Are you sure the intrinsic parameters (sdd, height, delx`) are correct? Those need to match the X-ray exactly (and can be read from a DICOM header). They don't necessarily look right to me.
2. You're trying to register a lateral view, but your initial view for the DRR is a frontal view. That's a large gap for most registration algorithms to bridge. You could start with alpha = -torch.pi / 2.
3. Your lr_rotations is too high. As a rule of thumb, I usually set it to 2-3 orders of magnitude less than the lr_translations.
4. The degree of blurriness of a DRR is tied to the resolution of the CT. If the CT is low res, the DRR is low res, no matter how big you make the DRR detector size. One thing you can try is setting renderer="trilinear" - I've noticed it's usually less aliased than renderer="siddon" for low-resolution CTs.

[linquanxu]

@eigenvivek
Thank you for your patient explanation. I have a few more questions.

1 Since I am using cropped volume data to render DRR, there is an offset between the center of the cropped data and the original volume data. Will this affect the generation of DRR? How can I ensure that the crop volume data is within the scan range of the light source?
2 Through continuous iteration between the DRR and the real X-ray, an optimal camera pose is generated. Once we have this optimal camera pose, does the DRR image rendered with this pose need further alignment with the real X-ray image?
3 How can I find the coordinates of a point on the DRR image rendered with the optimal camera pose in the CT volume data? Are there any relevant tutorials I can refer to?

[eigenvivek]

1. As long as you keep track of the affine matrix of the cropped volume, they'll be lined up. The 3D geometry functions can help you quickly verify that your camera is located in the right place.
2. No, once you've run iterative optimization, the optimized camera pose is final.
3. If you want to project a 3D point onto the image plane of a 2D detector plane, you can use drr.perspective_projection.

[linquanxu]

Thank you very much for your help. I will close this issue for now. This is an amazing piece of work and worth further study. If I encounter any problems in the future, I will seek your guidance again.

[eigenvivek]

you're welcome!