There are a variety of manufacturers of machines that take x-rays at each hospital. The Vertebra segmentation model was trained for X-ray images from one manufacturer. If the model is evaluated with images taken by other manufacturers, the performance of the model will be considerably reduced. I have been studied domain adaptation methodology with the aim of minimizing these domain shift problems. I have applied methods based GAN, Meta-learning, and fine-tuning for domain adaptation.
CycleGAN is a proposed model to convert the source domains into target domains without pairing.
The source domain of the model is set as training data of the segmentation model, and the target domain is set as external data taken by other manufacturers.
Zhu et al. proposed the CycleGAN network, Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Left is a target image and right is a source image.
Model-transformed images
The results of the model show that the texture of the spine is not preserved and the quality of the image is much lower.
Most of the models specialized in medical images use the U-Net structure, and it is that medical images are most important to preserve their unique characteristics. Therefore, to preserve the characteristics of the original image, I set the existing CycleGAN generator structure as U-Net and developed the Unet-CycleGAN using the cycle-consistency loss for domain conversion.
Left is a source image and right is a transformed image. (same as input datasets of CycleGAN)
The fake data synthesized by Cycle-UNetGAN was evaluated by the segmentation model.
The normal domain adaptation method requires a large amount of data. The method of this model is to approach the unlabeled data of the source domain and to adapt it to the unconditional image generation.
Utkarsh et al. proposed the Few-shotGAN network, Few-shot Image Generation via Cross-domain Correspondence
I studied methods that solves domain adaptation using meta-learning. Meta-learning is a method to learn new concepts and skills with less data. For the quality preservation of medical images, U-Net was used as a base model of meta-learning technique. The support set was all three domains, and 40,40,100 images are used for each domain.
MAML method was used to optimize parameters of meta-learning. Paper: Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Example of query set (Image, Predicted mask)
The proposed method was evaluated by source domain images and yielded an average Dice score of 0.7896.
As CycleGAN is only 1:1 domain transformable, I used a StarGAN network to convert it into many domains using one neural network.
Y. Choi et al. proposed the StarGAN_v1 network, StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation
F and D Vendor are source domains, and G vendor is a target domain.
In StarGAN v1, there was a problem that each domain has similar results according to the original image. (Extensibility to various domains) StarGAN v2 complements these problems and generates various images with scalability for the domain.
Y. Choi et al. proposed the StarGAN_v2 network, StarGAN v2: Diverse Image Synthesis for Multiple Domains
The datasets are same as StarGAN_v1.
It shows good results in terms of style to convert each domain into G Vendor. However, the exact resolution of the vertebrae are not obtained enough to perform segmentation.
I tried image processing for domain adaptation to transform image histogram.
