Official implementation of Cyclic Learning: Bridging Image-level Labels and Nuclei Instance Segmentation. The original paper link is here: arXiv link, TMI link.
Nuclei instance segmentation on histopathology images is of great clinical value for disease analysis. Generally, fully-supervised algorithms for this task require pixel-wise manual annotations, which is especially time-consuming and laborious for the high nuclei density. To alleviate the annotation burden, we seek to solve the problem through image-level weakly supervised learning, which has not been achieved by any previous work. Compared with most existing methods using other weak annotations (scribble, point, etc.) for nuclei instance segmentation, our method is more labor-saving. The obstacle to using image-level annotations in nuclei instance segmentation is the lack of adequate location information, leading to severe nuclei omission or overlaps.Cyclic learning comprises a front-end classification task and a back-end semi-supervised instance segmentation task to benefit from multi-task learning (MTL). We utilize the interpretability of a CNN as the front-end to convert image-level labels to sets of high-confidence pseudo masks and establish a semi-supervised architecture as the back-end to conduct nuclei instance segmentation under the supervision of these pseudo masks. Most importantly, cyclic learning is designed to circularly share knowledge between the front-end CNN and the back-end semi-supervised part, which allows the whole system to fully extract the underlying information from image-level labels and converge to a better optimum. Experiments on Three datasets validate that our method outperforms other image-level weakly supervised methods on nuclei instance segmentation and is close to the fully-supervised one in performance.
This project provides the Mask-RCNN version of our method. We will also release the hovernet version of our cyclic-learning.
-
Our project is developed on Mask_RCNN. Please first build up this project, then put our code repository under the directory
Mask_RCNN/samples/. -
Create an environment meets the requirements as listed in
requirements.txt
-
Download the Monusac dataset (pwd：mseg) and cropped Monusac dataset (pwd : mseg), and put it in the
Mask_RCNN/datasets/directory. Original paper: MoNu -
Download the ccrcc dataset (pwd：mseg), and put it in the
Mask_RCNN/datasets/directory. Originial paper: CCRCC -
Download the consep dataset (pwd：mseg), and put it in the
Mask_RCNN/datasets/directory. Original paper: CoNSeP -
Download the positive-and-negative nucleus image classification dataset (pwd : mseg) which is obtained by cropping out tile images from TCGA WSI(whole slide image). Datasets are organized in the following way:
datasets/
MyNP/
negative/
positive/
MoNuSACGT/
MoNuSACCROP/
stage1_train/
images/
masks/
ccrcccrop/
Test/
Train/
Valid/
consepcrop/
Test/
Train/
Valid/
Before training, please download pretrain weights of big nature image datasets, for which we use COCO pretrain weights. Remember to change the path in the code.
Training Cyclic Learning on MoNusac dataset:
python nucleus_recursion_50.pyTraining Cyclic Learning on ccrcc dataset:
python nucleus_recursion_ccrcc.pyTraining Cyclic Learning on consep dataset:
python nucleus_recursion_consep.pyIf you use Cyclic Learning in your work or wish to refer to the results published in this repo, please cite our paper:
@ARTICLE{zhou2023cyclic,
author={Zhou, Yang and Wu, Yongjian and Wang, Zihua and Wei, Bingzheng and Lai, Maode and Shou, Jianzhong and Fan, Yubo and Xu, Yan},
journal={IEEE Transactions on Medical Imaging},
title={Cyclic Learning: Bridging Image-level Labels and Nuclei Instance Segmentation},
year={2023},
doi={10.1109/TMI.2023.3275609}}