Image-to-image translation tutorial does segmentation?
brudfors opened this issue · 4 comments
Hi,
I am interested in doing 3D image-to-image translation. Looking at the image-to-image translation tutorial notebook it looks like it is doing segmentation, not image-to-image translation? Or am I missing something?
Is there a tutorial for image-to-image translation in 3D? If not, do you know if it is easy to adapt one of the 3D tutorials for this task?
Thank you!
Hey Michael!
Segmentation can be framed as synthesis one type of image (a label here!) from another, which is why its classified as such! What sort of task are you interested in, something like modality synthesis? If so, the notebook you linked to is a good starting point, you would just want to replace the label channel in the input with the desired target image (assuming you have paired data for supervised training available).
To move from 2D to 3D is fairly straightforward, you could build on this 3D DDPM notebook to help. If you want larger data you would need to move to an LDM, example here.
Does that help? Happy to chat more
Thanks Marc!
Def. agree that segmentation can be conceptualized as image synthesize, just got a a bit confused of reading image-to-image, image-to-label would make it clearer (to me at least), or change folder name from from image-to-image to image-synthesize/image-translation. Anyway, no big deal :)
It is CBCT recon, the images are 256**3, and I have paired data. I will have a look at the links you shared, hopefully it won't be too hard to implement.
Yeah that is fair - will consider changing the name!
For your data size you will need an LDM - will likely to have train seperate autoencoders on the two images modalities then concatenate the encoded representations of the two to serve as your inputs.
Okay, good to know, I will give it a try!