NoCodeSeg: Deep segmentation made easy!

⚠️Latest: Generic multiclass support has been added to the pipeline!

This is the official repository for the manuscript "Code-free development and deployment of deep segmentation models for digital pathology", published open access in Frontiers in Medicine.

The repository contains trained deep models for epithelium segmentation of HE and CD3 immunostained WSIs, as well as source code relevant for importing/exporting annotations/predictions in QuPath, both from DeepMIB and FastPathology.

All relevant scripts for working with our pipeline can be found in the source directory.

See here for how to access the trained models.

See here for how to download the 251 annotated WSIs.

Getting started

A video tutorial of the proposed pipeline was published on YouTube. It demonstrates the steps for:

Downloading and installing the software
QuPath
- Create a project, then export annotations as patches with label files
- Export patches from unannotated images for prediction in DeepMIB
- (later) Import predictions for MIB and FastPathology as annotations
MIB
- Use the annotated patches/labels exported from QuPath
- Configuring and training deep segmentation models (i.e. U-Net/SegNet)
- Use the trained U-Net to predict unannotated patches exported from QuPath
- Export trained models into the ONNX format for use in FastPathology
FastPathology
- Importing and creating a configuration file for the DeepMIB exported ONNX model
- Create a project and load WSIs into a project
- Use the U-Net ONNX model to render predictions on top of the WSI in real time
- Export full sized WSI tiffs for import into QuPath

Note that the version of FastPathology used in the demonstration was v0.2.0 (this exact version can be downloaded from here). The software is continuously in development, and features presented in the video are therefore prone to changes in the near future. To get information regarding changes and new releases, please, visit the FastPathology repository.

Data

The 251 annotated WSIs are made openly available for anyone on DataverseNO. Alternatively, the data can be downloaded directly from Google Drive (click here to access the dataset). Information on how to cite the IBDColEpi dataset can be found on DataverseNO.

Reading annotations

The annotations are stored as tiled, pyramidal TIFFs, which makes it easy to generate patches from the data without the need for any preprocessing. Reading these files and working with them to generate training data, is already described in the tutorial video above.

TL;DR: Load TIFF as annotations in QuPath using provided groovy script and exporting these as labelled tiles.

Reading annotation in Python

However, if you wish to use Python, the annotations can be read exactly the same way as regular WSIs (for instance using OpenSlide):

import openslide

reader = ops.OpenSlide("path-to-annotation-image.tiff")
patch = reader.read_region(location=(x, y), level, size=(w, h))
reader.close()

Pixels here will be one-to-one with the original WSI. To generate patches for training, it is also possible to use pyFAST, which does the patching for you. For an example see here.

Models

Note that the trained models can only be used for academic purposes due to MIB's license. Trained model files (.mibDeep for MIB and .onnx for FastPathology) are made openly available on Google Drive. Simply download the file "trained-models.zip" and uncompress to get access the respective files.

Applications of pipeline

Røyset et al., Deep learning-based image analysis reveals significant differences in the number and distribution of mucosal CD3 and γδ T cells between Crohn's disease and ulcerative colitis, The Journal of Pathology, https://doi.org/10.1002/cjp2.301
Pettersen et al., Code-free development and deployment of deep segmentation models for digital pathology (2022), Frontiers in Medicine, https://doi.org/10.3389/fmed.2021.816281

How to cite

Please, consider citing our paper, if you find the work useful:

  @article{10.3389/fmed.2021.816281,
  author={Pettersen, Henrik Sahlin and Belevich, Ilya and Røyset, Elin Synnøve and Smistad, Erik and Simpson, Melanie Rae and Jokitalo, Eija and Reinertsen, Ingerid and Bakke, Ingunn and Pedersen, André},   
  title={Code-Free Development and Deployment of Deep Segmentation Models for Digital Pathology},      
  journal={Frontiers in Medicine},      
  volume={8},      
  year={2022},      
  url={https://www.frontiersin.org/article/10.3389/fmed.2021.816281},       
  doi={10.3389/fmed.2021.816281},      
  issn={2296-858X}}

In addition, if you used the data set in your work, please, cite the following:

  @data{TLA01U_2021,
  author = {Pettersen, Henrik Sahlin and Belevich, Ilya and Røyset, Elin Synnøve and Smistad, Erik and Jokitalo, Eija and Reinertsen, Ingerid and Bakke, Ingunn and Pedersen, André},
  publisher = {DataverseNO},
  title = {{140 HE and 111 CD3-stained colon biopsies of active and inactivate inflammatory bowel disease with epithelium annotated: the IBDColEpi dataset}},
  year = {2021},
  version = {V2},
  doi = {10.18710/TLA01U},
  url = {https://doi.org/10.18710/TLA01U}}

Acknowledgements

We wish to give our praise to Peter Bankhead and the QuPath team for their continuous support and assistance with QuPath and for assisting us in developing the scripts related to this study.

cavenel/NoCodeSeg

NoCodeSeg: Deep segmentation made easy!