Dynamic display of registration process

[shenao1994]

I am writing code to plot the registration process. My registration work has been completed. I also want to have the effect in GIF, but I am not familiar with pyvista. Can you share how this is achieved? In addition, marker points can be removed during the process.

[eigenvivek]

Hi @shenao1994 this is the combination of two separate GIFs.

The one on the left is generated with the plotting function in PyVista. Each frame was generated with pyvista and saved to disk. Then they were combined into a single GIF using ffmpeg.

The one on the right was generated using diffdrr.visualization.plot_drr. Again, each frame is saved to disk and then combined with ffmpeg.

Finally, the two GIFs were combined into a single one with ffmpeg. The script for doing all this is buried somewhere on my computer and I'll try to dig it up.

[eigenvivek]

Glad to hear your registration worked!

[shenao1994]

Hi @shenao1994 this is the combination of two separate GIFs.

The one on the left is generated with the plotting function in PyVista. Each frame was generated with pyvista and saved to disk. Then they were combined into a single GIF using ffmpeg.

The one on the right was generated using diffdrr.visualization.plot_drr. Again, each frame is saved to disk and then combined with ffmpeg.

Finally, the two GIFs were combined into a single one with ffmpeg. The script for doing all this is buried somewhere on my computer and I'll try to dig it up.

Thank you for your advice!

But I have encountered some problems. Every time I generate a frame image, I need to remove the mesh of the previous frame. When I add_mesh CT again, the effect will become very strange.

It is normal to add_mesh for the first time.

But the final effect of CT will become less stereoscopic.

My code is as follows:

    ct = drr_to_mesh(vert_drr, "surface_nets", threshold=300, verbose=False)
    plotter = pyvista.Plotter()
    plotter.add_mesh(ct)
    plotter.open_gif('reg.gif', fps=20)
    for idx, row in poses_data.iterrows():
        rotate = (
            torch.tensor(row[["alpha", "beta", "gamma"]].values).unsqueeze(0)
            .to(device)
        )
        la_rotations = rotate + torch.tensor([[-torch.pi/2, 0, 0]]).to(device)
        transl = (
            torch.tensor(row[["bx", "by", "bz"]].values).unsqueeze(0).to(device)
        )
        print(rotate)
        print(transl)
        pose = convert(rotate.float(), transl.float(), parameterization='euler_angles', convention="ZYX")
        camera, detector, _, principal_ray = img_to_mesh(
            vert_drr, pose
        )
        img = vert_drr(pose)
        img = img.detach().cpu().squeeze().numpy()
        img = (img - img.min()) / (img.max() - img.min())
        texture = (255.0 * img).astype(np.uint8)
        plotter.add_mesh(ct)
        add_gt_xray_mesh(plotter, la_rotations, transl, vert_drr)
        plotter.add_mesh(camera, show_edges=True, line_width=1.5)
        plotter.add_mesh(principal_ray, color="lime", line_width=3)
        plotter.add_mesh(detector, texture=texture)
        plotter.write_frame()
        plotter.clear()
    plotter.close()

[eigenvivek]

Hi @shenao1994 sorry i forgot to follow up on this. Please see #23 and let me know if you have any more questions.