Multi-modal digital pathology for colorectal cancer diagnosis by high-plex immunofluorescence imaging and traditional histology of the same tissue section

Jia-Ren Lin^1,2,*, Yu-An Chen^1,2,*, Daniel Campton^3,*, Jeremy Cooper³, Shannon Coy^1,4, Clarence Yapp^1,2, Juliann B. Tefft^1,2, Erin McCarty³, Keith L. Ligon⁴, Scott J. Rodig⁴, Steven Reese³, Tad George³, Sandro Santagata^1,2,4,±, Peter K. Sorger^1,2,±

* These authors contributed equally
± These authors contributed equally

Human Tissue Atlas Center
¹ Laboratory of Systems Pharmacology, Harvard Medical School, Boston, MA, 02115, USA.
² Ludwig Center at Harvard, Harvard Medical School, Boston, MA 02115, USA.
³ RareCyte, Inc., 2601 Fourth Ave., Seattle, WA, 98121, USA.
⁴ Department of Pathology, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA 02115, USA.

Scientific summary

Precision medicine is critically dependent on better methods for diagnosing and staging disease and predicting drug response. Histopathology using Hematoxylin and Eosin (H&E) stained tissue - not genomics – remains the primary diagnostic modality in cancer. Moreover, recently developed, highly multiplexed tissue imaging represents a means of enhancing histology workflows with single cell mechanisms. Here we describe an approach for collecting and analyzing H&E and high-plex immunofluorescence (IF) images from the same cells in a whole-slide format suitable for translational and clinical research and eventual deployment in diagnosis. Using data from 40 human colorectal cancer resections (60 million cells) we show that IF and H&E images provide human experts and machine learning algorithms with complementary information. We demonstrate the automated generation and ranking of computational models, based either on immune infiltration or tumor-intrinsic features, that are highly predictive of progression-free survival. When these models are combined, a hazard ratio of ~0.045 is achieved, demonstrating the ability of multi-modal digital pathology to generate high-performance and interpretable biomarkers.

Data availability

Full-resolution images, derived imaging data, and single-cell spatial feature tables will be released by the National Cancer Institute sponsored repository for Human Tumor Atlas Network. While the public resource is still undergoing extensive development, we provided exemplar data via Synapse for demonstration purposes. Light-weight image viewing for all the bio-specimens used in this study is made available using Minerva and Scope2Screen [1].

Exemplar data (`syn38990468`)

To download, please refer to the Synapse documentation

Files in the exemplar data

P37_S29-CRC01/
├── quantification
│   └── p37_s29_a24_c59kx_e15__at__20220106_014304_946511_cellringmask.csv
│       # single-cell feature table
├── segmentation
│   └── cellRingMask.tif
│       # labeled mask used to quantify single-cell features
└── registration
    ├── p37_s29_a24_c59kx_e15__at__20220106_014304_946511.ome.tiff
    │   # orion IF image (19-channel, 16-bit)
    └── 18459-lsp10353-us-scan-or-001 _093059-registered.ome.tif
        # post-orion H&E image registered to orion image (3-channel, 8-bit)

Image viewing using web browsers

Light-weight image viewing for all the bio-specimens used in this study is made available using Minerva and Scope2Screen [1]. Visit https://labsyspharm.github.io/orion-crc to access the images.

Codes for imaging data processing

Highplex Orion whole-slide images were processed using MCMICRO modules with customizations to cope with the large X-Y dimension in the dataset. Registration of Orion immunofluorescence images and post-Orion H&E images was done with PALOM.

Files in the `scripts-processing/` folder

To process Orion images into single tables, we ran unmicst and S3segmenter to generated labeled mask and run quantification using the segmentation mask and the 19-channel Orion image. Version and repositories of the processing modules is listed in the github_repo.md file.

conda-env yaml files: environment specs for creating conda envs to run customized mcmicro modules
github_repo.md file: version and repositories of the processing modules
.py files: scripts for batch processing multiple input images
file_list.csv file: file index for batch processing
markers.csv file: antibody target names of channels in the Orion images

Codes for single-cell spatial analysis

Single-cell spatial analysis was performed using MATLAB 2019b. To run the provided scripts on your data, please contact authors for more details.

Demo for running the analysis and plotting scripts

Please contact the authors for detailed information.

%% Optimize ImmuneScore

markers = {'CD3','CD8','CD45','CD45RO','CD68','CD163','CD4','CD20','SMA'};
regions = {'R1','R2'};
figure;

for i = 1:length(markers)
    for j = 1:length(regions)
        subplot(3,6,(i-1)*length(regions)+j);
        marker1 = strcat('norm_',markers{i},regions{j});
        list1 = sumAllsample{:,marker1};
        list2 = sumAllsample.PFSDays;
        scatter(list1,list2,30,'b','fill');
        lsline;
        title(num2str(corr(list1,list2),'%0.2f'),'FontSize',16);
        set(gca,'xtick',[]);
        xlabel(marker1,'Interpreter','none');
        set(gca,'ytick',[]);
        ylabel('PFS Days');
    end
end

Funding

This work was supported by NCI grants U54-CA225088 and U2C-CA233262 (PKS, SS), an NCI SBIR small business grant to RareCyte and PKS (R41-CA224503), and commercial investment from RareCyte; data processing software was developed with support from a Team Science Grant from the Gray Foundation and Ludwig Cancer Research (PKS, SS). SS is supported by the BWH President’s Scholars Award.

References

[1] J. Jessup and R. Krueger et al., "Scope2Screen: Focus+Context Techniques for Pathology Tumor Assessment in Multivariate Image Data," in IEEE Transactions on Visualization and Computer Graphics, vol. 28, no. 1, pp. 259-269, Jan. 2022, doi: 10.1109/TVCG.2021.3114786.

Yu-AnChen/orion-crc