Comparison of imaging based single-cell resolution spatial transcriptomics profiling platforms using formalin-fixed paraffin-embedded tumor samples

Abstract

Imaging-based spatial transcriptomics (ST) is evolving as a pivotal technology in studying tumor biology and associated microenvironments. However, the strengths of the commercially available ST platforms in studying spatial biology have not been systematically evaluated using rigorously controlled experiments. We use serial 5 μm sections of formalin-fixed, paraffin-embedded surgically resected lung adenocarcinoma and pleural mesothelioma samples in tissue microarrays to compare the performance of the ST platforms (CosMx, MERFISH, and Xenium (uni/multi-modal)) in reference to bulk RNA sequencing, multiplex immunofluorescence, GeoMx, and hematoxylin and eosin staining data. In addition to an objective assessment of automatic cell segmentation and phenotyping, we perform a manual phenotyping evaluation to assess pathologically meaningful comparisons between ST platforms. Here, we show the intricate differences between the ST platforms, reveal the importance of parameters such as probe design in determining the data quality, and suggest reliable workflows for accurate spatial profiling and molecular discovery.

Spatial cell distribution within a tissue microenvironment is a rapidly advancing field. Here, authors assess three commercially available single-cell resolution spatial transcriptomics approaches (CosMx, MERFISH, and Xenium) to inform which technology outperforms for immune profiling of solid tumors using patient samples.

Introduction

Studying the whole panorama of cellular and molecular interactions in tissues accurately and within their functional context is vital for understanding health and disease. Spatial transcriptomics (ST) assays characterize gene expression profiles and localize them on histological tissue sections, preserving the context of interactions present in the tissue. Nature Methods selected ST as its Method of the Year 2020¹, and this method has evolved rapidly since, with many technology companies including it in their assays². ST has special significance in studying the association between a tumor and its microenvironment in cancer biology³. This novel technology can detect gene expression while preserving the location of genes at the single-cell level when using imaging-based ST techniques that rely on fluorescence in situ hybridization⁴. This allows researchers to investigate tissue sections and gain an understanding of complex interactions between cell populations and their arrangements within tissues⁵.

Multiple commercial ST solutions have become available recently, such as CosMx Spatial Molecular Imaging (CosMx; NanoString, a Bruker company), MERFISH (Vizgen), and Xenium (10x Genomics), which are used to perform multiple cycles of nucleic acid hybridization of fluorescent molecular barcodes to identify RNA molecules while mapping their locations. However, they differ in their sample preparation protocols during amplification, gene selection for panel design, and cell-segmentation processes^{6, 7, 8–9}. Aside from data generation differences, these platforms differ in panel size, use of quality control probes (negative controls and blank probe selections), H&E staining, profiling area, morphology marker staining, cell segmentation algorithms and interactive viewer software (Supplementary Data 1).

Comparison of imaging-based ST assays by multiple teams using different types of tissue is ongoing^{9, 10–11}. These studies will help researchers select optimal single-cell ST methods for their assays, which are vital to producing high-quality data. Wang et al.⁹ compared CosMx, MERFISH and Xenium using tumor and normal tissue TMAs constructed with multiple tissue types, and correlated ST data with reference RNA-seq datasets from TCGA. Cook et al.¹⁰ compared CosMx and Xenium using prostate adenocarcinoma and annotated data with snRNAseq on the same tissue. Hartman et al.¹¹ investigated the differences between MERFISH, Xenium and other academically available platforms using previously generated publicly available mouse brain datasets. A recent example from Ren et al.¹² compared high coverage imaging-based (CosMx 6K, Xenium Prime 5 K panels) and other sequencing-based ST platforms (i.e: Visium-HD, Stereo-seq), with proteomic validation in adjacent serial sections using CODEX. However, to apply these methodologies in translational research, it is also important to evaluate the correlation of the overall gene expression data obtained with these platforms with other orthogonal standard methods, such as bulk RNA-seq data or with spatial RNA expression data from the same tissue cores, in addition to gaining more confidence in the spatial data obtained from the different phenotypes. Furthermore, there is a need to evaluate the spatial patterns and cell segmentations of main cell types with expected histological patterns observed with standard H&E staining or immunohistochemistry methods, as providing a critical step in understanding tissue morphology and determining the accuracy of these ST platforms in cell segmentation and cell type annotation¹³.

Here, we show a comparison of commercially available imaging-based ST platforms, CosMx, MERFISH, and Xenium, with unimodal (Xenium-UM) and multimodal (Xenium-MM) segmentation using 5μm serial sections of formalin-fixed, paraffin-embedded (FFPE) surgically resected lung adenocarcinoma and pleural mesothelioma samples obtained from 2016 to 2022 and placed in TMAs. We evaluate each platform by comparing multiple metrics, including tissue age, average transcript counts, and uniquely expressed gene counts per cell, and signal detection above background by using negative control probes as well as blank probes due to a lack of negative control probes in MERFISH and blank probes in CosMx panels. Later, we investigate the performance of manufacturers’ cell segmentation algorithms across the platforms by evaluating the presence of transcripts in cells and individual cell area sizes, as well as co-expression of disjoint genes by measuring the level of joint detections from genes that are predominantly exclusive among cell populations. We also measure concordance of the imaging-based ST data across the ST platforms with data obtained using bulk RNA sequencing (RNA-seq) and GeoMx DSP WTA of the same specimens. In addition, we compare cell type annotations among platforms based on selected genes in the panels of each ST platform and benchmark against the pathologists’ evaluation of phenotyping using mIF and H&E stained sections of the samples.

The objectives of this study were to provide a comprehensive explanation of the differences in imaging, multiplexing and tissue age capability, and data acquisition among the three ST platforms using a bioinformatics data analysis pipeline and a pathology-oriented review of the final cell type annotations produced by each platform’s probe reads.

Results

Selection of lung adenocarcinoma and pleural mesothelioma data sets

To ensure that our comparison of the performance of single-cell ST platforms can be used for translational studies of immuno-oncology research, we used archived FFPE tumor samples, which represent the current standard for sample processing and archiving in pathology. All tissue samples processed in FFPE blocks using standard operating procedure at UT MD Anderson Cancer Center. Specifically, we used four TMAs containing samples of surgically resected tumor tissue obtained in different years and with different immune profiling features: two TMAs containing samples of lung adenocarcinoma (ICON1 and ICON2; collected from 2016 to 2018), which is regarded as an immune hot tumor, and two TMAs containing samples of pleural mesothelioma (MESO1 and MESO2; collected from 2020 to 2022), which is regarded as an immune cold tumor. We submitted serial sections of each TMA to the ST companies to run the single-cell imaging–based ST assays. We subjected the ICON2 and MESO2 TMAs to all three ST assays, while ICON1 and MESO1 were unavailable in Xenium and MERFISH due to assay and tissue constraints, respectively. We used the best available panel for immuno-oncology research, which included genes relevant to lung adenocarcinoma and pleural mesothelioma (Fig. 1a).

Fig. 1 Experimental design, panel comparison, and nuclear staining with the ST platforms. [Images not available. See PDF.]

a TMAs containing tissue samples from patients with pleural mesothelioma (MESO1, MESO2; n = 22) or non-small cell lung cancer (NSCLC; ICON1, ICON2; n = 22). Tumor samples were sectioned at a thickness of 5 μm and submitted to CosMx, MERFISH, Xenium-UM, and Xenium-MM assays based on tissue availability. Data analyses were performed using Seurat pipeline and evaluation of pathologists with guidance of mIF and H&E staining. b Venn diagram displays shared genes between panels of the ST platforms. c DAPI staining images of ICON2 and MESO2 TMAs and CosMx whole images of the ICON2 and MESO2 TMAs with staining with morphology markers before Field of View (FOV) selection. Red/white squares display selected FOVs of CosMx. The CosMx FOV, MERFISH, Xenium-UM, and Xenium-MM images display nuclear staining of selected FOV regions and whole tissue cores. Only TMAs with data available from all ST platforms were selected for comparisons. Created in BioRender. Ozirmak lermi, N. (2025) https://BioRender.com/y68w242.

Comparison of gene panel coverage and imaging

The panels used in our study were the standard CosMx Human Universal Cell Characterization Panel (RNA, 1,000-plex), human MERSCOPE Immuno-Oncology Panel for MERFISH (RNA, 500-plex), and a 289-plex human lung panel plus a panel of 50 custom genes for Xenium, with 93 genes shared by all the panels. The CosMx and Xenium panels shared 154 genes, the CosMx and MERFISH panels shared 302 genes, and the Xenium and MERFISH panels shared 118 genes (Fig. 1b and Supplementary Data 2).

The CosMx pipeline required region selection (field of view [FOV]) with 545 μm  ×  545 μm, and we acquired up to 47 FOVs per slide in our experiment. Thus, we could not analyze whole tissue cores in CosMx data. The MERFISH and Xenium pipelines covered the whole tissue area mounted on each slide according to the manufacturers’ instructions (Methods) (Fig. 1c).

Variation of transcripts and unique gene detections per cell among ST platforms

Transcript counts per cell were crucial to annotating cell types in the downstream analysis. We filtered cells with fewer than 30 transcript counts and that were five times larger than the geometric mean of cell area sizes of all cells on the CosMx platform, based on the recommendations of NanoString. For analysis of MERFISH and Xenium, we removed cells with fewer than 10 transcript counts to avoid cells without any transcript counts or with low transcript counts. We followed different filtering parameters in CosMx to eliminate cells that caused poor downstream data analyses. After cell filtering, we calculated the number of transcripts per cell and unique gene counts per cell in each TMA. We normalized transcripts and gene counts per cell with panel sizes. The more recently constructed MESO TMAs had higher numbers of transcripts and uniquely expressed genes per cell with CosMx and MERFISH than Xenium. CosMx detected the highest transcript counts and uniquely expressed gene counts per cell among all TMAs (p < 2.2e−16), whereas MERFISH detected lower transcript and uniquely expressed gene counts per cell in the ICON1 and ICON2 TMAs than in the newer MESO2 TMA (p < 2.2e−16). When comparing the two Xenium segmentation modalities, Xenium-UM assay had higher transcript and gene counts per cell than did Xenium-MM assay (p < 2.2e−16) (Fig. 2a, b).

Fig. 2 Technical comparison of the ST platforms. [Images not available. See PDF.]

a, b Box plots of the transcript count and uniquely expressed genes per cell that are captured on each ST platform for CosMx (blue) ICON1 (n = 36512), CosMx ICON2 (n = 38395), CosMx MESO1 (n = 58996), CosMx MESO2 (n = 45898), MERFISH (purple) ICON1 (n = 31431), MERFISH ICON2 (n = 94539), MERFISH MESO2 (n = 98149), Xenium-UM (yellow) ICON2 (n = 133019), Xenium-UM MESO1 (n = 142721), Xenium-UM MESO2 (n = 85196), Xenium-MM (orange) ICON2 (n = 113387), Xenium-MM MESO1 (n = 128042), Xenium-MM MESO2 (n = 70472). Data were normalized by panel size. Each dot represents a cell. The center of the box plot is denoted by the median, a horizontal line dividing the box into two equal halves. The bounds of the box are defined by the lower quartile (25th percentile) and the upper quartile (75th percentile). The whiskers extend from the box and represent the data points that fall within 1.5 times the interquartile range (IQR) from the lower and upper quartiles. Any data point outside this range is considered an outlier and plotted individually. The red diamonds correspond to the average transcript counts and uniquely expressed genes in the blocks. Significance assessed using Mann–Whitney–Wilcoxon two-sided test (p < 2.2e−16, p = 0.00039). c Dot plot of the total transcript counts per probe across the ST platforms. Each dot represents a gene probe (red) or a negative control probe (green). d Bar graph of False Discovery Rates (FDR) calculated using negative control or blank probes and total read counts. The y-axis represents FDR as a percentage. The gray bars represent unavailable data. The blue and green bars represent FDR calculated using blank and negative control probes, respectively. Source data are provided as a Source Data file.

Comparison of the expression of negative control and gene probes

We plotted the expression levels of the negative control probes, and the target gene probes to determine whether the target gene probes were less expressed than the negative controls. The CosMx panel included 10 negative control probes, the MERFISH panel included 50 blank probes, and the Xenium panel included 20 negative control probes, 41 negative control code words, and 141 blank code words. CosMx displayed multiple target gene probes that expressed the same as negative control probes in all TMAs (ICON1, n = 8 [0.8%]; ICON2, n = 27 [2.7%]; MESO1, n = 196 [19.6%]; MESO2, n = 319 [31.9%]). These target gene probes in the CosMx dataset were important for cell type annotation (e.g., CD3D, CD40LG, FOXP3, MS4A1, MYH11) (Fig. 2c, Supplementary Data 3). Xenium-MM exhibited few target gene probes (MESO2, n = 2 [0.6%]), whereas Xenium-UM did not have any target gene probes that expressed similarly to negative controls. We could not perform this comparison with MERFISH owing to a lack of negative control probes. We did not exclude any markers from the panels even if target genes were within the range of expression of the negative control probes.

We then assessed the specificity of each platform by calculating false discovery rates (FDRs) using negative control and blank code words/probes individually^9,14,15. We observed that CosMx had 10% and 11% FDR in ICON TMAs and 5.8% and 6.2% FDR in MESO TMAs based on FDR calculation with negative control probes; MERFISH had 4.4% and 6% FDR in ICON TMAs and 4.8% FDR in MESO1 TMA based on FDR calculation with blank probes; Xenium-UM and Xenium-MM had less than 0.09% FDR in ICON and MESO TMAs based on FDR values with negative control and blank probes. There were no significant differences between FDR values of CosMx and Xenium-UM or Xenium-MM (p = 0.057), MERFISH and Xenium-UM or Xenium-MM (p = 0.25), and Xenium-UM and Xenium-MM (p = 1) (Fig. 2d).

Impact of cell segmentation on average cell area size

Cell area sizes were crucial for interpreting transcript counts per cell. CosMx, MERFISH, and Xenium-MM included staining for morphological markers that are effective at identifying cell boundaries, and we processed ST data using standard cell-segmentation pipelines provided by the manufacturers. We performed cell segmentation with Xenium-UM via nuclear detection with a nuclear stain and cell boundary expansion algorithm. For this dataset, we tested different DAPI pixel-intensity settings to adjust the nuclear detection (50, 75, and 100 photoelectrons) and adjusted the maximum distance for cell membrane expansion from 7 μm to 15 μm to find the optimum cell-segmentation parameters with the Xenium Ranger software tool¹⁶. We accepted a Xenium-UM setting of 75-pixel intensity from the DAPI stain for nuclear detection and an expansion distance of 7 μm as the best parameters for cell segmentation after evaluating the results with a pathologist by visual assessment of cell segmentation masks.

Next, we calculated the average cell area size per cell on each ST platform. Even with our best assessment of the Xenium-UM segmentation parameters, it generated larger cell area sizes than did the cell-segmentation algorithms with morphology marker guidance in CosMx, MERFISH, and Xenium-MM (p < 2.2e−16). Xenium-MM produced the smallest average cell area size across all TMAs (p < 2.2e−16) and the closest cell shapes in biology that we observed microscopically (e.g., elongated shapes of fibroblasts). The range in cell area size differed but not significant between CosMx, MERFISH, Xenium-UM and Xenium-MM (Fig. 3a, b, Supplementary Data 4).

Fig. 3 Comparative analysis of cell segmentation across the ST platforms. [Images not available. See PDF.]

a Zoomed images of nuclear staining (green) overlaid with cell boundaries (white and red) obtained on all platforms using a tissue core of ICON2 TMA (CosMx (n = 14), MERFISH (n = 21), Xenium-UM (n = 26), Xenium-MM (n = 22). Cells with red boundaries represent filtered cells after quality controls. b Bubble plot of the mean cell area sizes (μm²) across the tissue blocks in CosMx (blue), MERFISH (purple), Xenium-UM (yellow) and Xenium-MM (orange) assays. The bubble size corresponds to the difference between the smallest and largest cell areas within each block. c Percentages of the remaining cells after filtering. The dashed lines indicate transcript count thresholds for: Xenium (yellow and orange) (10), MERFISH (purple) (10), and CosMx (blue) (30). Source data are provided as a Source Data file.

Impact of cell detection and segmentation on transcripts per cell and phenotyping

The fractions of remaining cells after filtering were varied. The cell filtering process of CosMx, Xenium-UM and Xenium-MM assays did not remove more than 3% of detected cells, with the exception of CosMx for the ICON1 TMA. However, we lost more than 40% of detected cells in MERFISH TMA assays and approximately 75% of detected cells in MERFISH ICON1 TMA. The percentage of remaining cells after filtering step was following in each TMAs: CosMx: ICON1, 84.0%; ICON2, 98.2%; MESO1, 97.8%; MESO2, 97.8%; Xenium-UM: ICON2, 97.4%; MESO1, 98.2%; MESO2, 98.8%; and Xenium-MM: ICON2, 96.8%; MESO1, 95.7%; MESO2, 98.0%); MERFISH: ICON1, 22.6%; ICON2, 40.0%; MESO2, 64.4%. (Fig. 3a, c).

Next, we chose canonical markers for distinct cell populations in pairs (CD19 - CD3E; EPCAM - CD3E; CD8A - CD4; CD68 - CD3E), B-cell marker CD19 and T-cell marker CD3E; EPCAM, a marker for epithelial cancer cells; CD4 and CD8A, which are markers for T-cell subsets; and CD68, a myeloid/macrophage marker. We calculated the percentage of cells that co-expressed gene pairs on each TMA per platform. While Xenium-UM, Xenium-MM and MERFISH displayed similar (p = 0.75, p = 1) percentage of cells with co-expression of CD19 -CD3E and EPCAM - CD3E, CosMx displayed more cells with co-expression (p = 0.057) than the other platforms. In addition, CosMx, Xenium-UM and Xenium-MM had relatively similar percentages of cells with co-expressions of CD68 - CD3E (p = 1, p = 0.86, p = 0.5, respectively). All platforms displayed a similar percentage of cells with co-expression of CD8A - CD4 (p = 0.63, p = 0.86, p = 0.5, p = 0.25, respectively) (Supplementary Fig. 1a, Supplementary Data 5).

The percentage of cells with co-expression of these pairs increased when the differences in biological cell area size between cell types increased (e.g., T cells and tumor cells; T cells and macrophages). While CosMx had significant differences between percentage of cells based on biological cell size differences (p = 0.023), Xenium-UM and Xenium-MM also showed a trend towards a higher percentage of cells with gene pairs of bigger biological differences (p = 0.31). MERFISH showed the opposite trend in the comparison of biological cell size differences (p = 0.094) (Supplementary Fig. 1b, Supplementary Data 5).

Correlation of gene expression results with bulk RNA-seq and DSP whole-transcriptome assay

We displayed concordance of ST data with orthogonal data generated with bulk RNA-seq and DSP whole-transcriptome assay to evaluate the results of ST assays. The correlation of imaging-based ST platforms data with bulk RNA-seq was positive among all data sets. Whereas the concordance of ICON2 was lower for CosMx (R = 0.43, p < 0.0001) and MERFISH (R = 0.39, p < 0.0001) than Xenium-MM, the same correlation in the newest TMA (MESO2) was higher for both CosMx (R = 0.58, p < 0.0001) and MERFISH (R = 0.64, p < 0.0001) than ICON2 CosMx and MERFISH. On the other hand, the concordance of ICON2 for Xenium-UM (R = 0.62, p < 0.0001) and Xenium-MM (R = 0.62, p < 0.0001) was lower than the concordance in MESO2 for Xenium-UM (R = 0.66, p < 0.0001) and Xenium-MM (R = 0.65, p < 0.0001), but the difference between ICON2 and MESO2 correlation of Xenium-UM with bulk RNA-seq was not as big as what we observed for CosMx and MERFISH (Fig. 4a, b).

Fig. 4 Concordance of RNA levels between ST platforms and bulk RNA-seq data. [Images not available. See PDF.]

Scatter plots of the average expression of overlapping genes in ST platform and bulk RNA-seq data of patients of the ICON2 (a) and MESO2 (b) TMAs. The red lines represent linear regression of mean log-normalized expression per cell type ± 95% confidence interval. Pearson correlation coefficients (R) are provided in the top left corner of each plot. Outlier genes are annotated. Source data are provided as a Source Data file.

Afterward, we compared ST data with the GeoMx DSP WTA results. CosMx data (ICON2: R = 0.84, p < 0.0001; MESO2: R = 0.85, p < 0.0001) had better correlation with DSP WTA data than did MERFISH (ICON2: R = 0.67, p < 0.0001; MESO2: R = 0.77, p < 0.0001), Xenium-UM (ICON2: R = 0.70, p < 0.0001; MESO2: R = 0.80, p < 0.0001), and Xenium-MM data (ICON2: R = 0.68, p < 0.0001; MESO2: R = 0.80, p < 0.0001) (Fig. 5).

Fig. 5 Concordance of ST platforms with DSP WTA. [Images not available. See PDF.]

Scatter plots display correlation of the ST platforms and DSP WTA based on shared genes in ICON2 (a) and MESO2 (b) TMAs. The blue lines represent linear regression of mean log-normalized expression per cell type ± 95% confidence interval. The Pearson correlation coefficient (R) is provided in the top left corner of each plot. Source data are provided as a Source Data file.

We then compared the ST platforms with each other using shared genes to examine the reproducibility of ST platforms. CosMx and Xenium-UM data (ICON2: R = 0.86, p < 0.0001; MESO2: R = 0.88, p < 0.0001) correlated better than did CosMx and MERFISH (ICON2: R = 0.74, p < 0.0001; MESO2: R = 0.79, p < 0.0001) and Xenium-UM and MERFISH (ICON2: R = 0.67, p < 0.0001; MESO2: R = 0.84, p < 0.0001). Xenium-UM and Xenium-MM correlated strongly in ICON2 and MESO2 tissue blocks (R = 0.99, p < 0.0001). High correlation between Xenium-UM and Xenium-MM demonstrated the reproducibility of Xenium data even when using different cell-segmentation methods (Fig. 6).

Fig. 6 Comparison of gene count detection among the different ST platforms. [Images not available. See PDF.]

Scatter plots display correlation of different ST platforms based on shared genes in ICON2 (a) and MESO2 (b) TMAs. The green lines represent linear regression of mean log-normalized expression per cell type ± 95% confidence interval. The Pearson correlation coefficient (R) is provided in the top left corner of each plot. Source data are provided as a Source Data file.

Differences in cell type annotations among ST platforms

To compare each ST platform’s ability to phenotype cells accurately, we used two different approaches for each set of TMAs. With the ICON1 and ICON2 TMAs, we performed an unsupervised clustering algorithm and annotated clusters with a cell type manually based on highly expressed genes in each cluster, guided by a series of lineage markers that we selected based on the available genes in each panel (Methods).

Using CosMx and the ICON TMAs, we were able to identify nine major cell types: macrophages, monocytes, fibroblasts, tumor cells, endothelial cells, smooth muscle cells, plasma cells, mast cells, and T cells. We labeled a cluster as being of unknown cell type if it had expression of CD19, CD3E and EPCAM in the same cluster. T-cell cluster had low expression of B-cell markers (e.g., CD19, MS4A1). Upon further investigation, we concluded that signals from CD3E and CD19 came from every cell type in the clustering analysis, thus rendering these phenotypes undetectable. Moreover, correlation analysis of the assigned cell types of CosMx with ICON2 TMA demonstrated high levels of correlation between different cell types in multiple genes that distinguished each cluster (Fig. 7a and Supplementary Fig. 2).

Fig. 7 Cell type annotation performance of the ST platforms using the ICON2 TMA. [Images not available. See PDF.]

a UMAP and spatial locations of manually annotated cell type clusters across the platforms. Clusters are labeled with their corresponding cell types based on top-expressed genes. Each color represents a cell type. b Box plots of the F1-scores display performance of cell segmentation and cell type annotations in ST platforms. Each dot represents number of tissue cores in CosMx (blue) ICON2 (n = 14), MERFISH (purple) ICON2 (n = 21), Xenium-MM (orange) ICON2 (n = 21). The center of the box plot is denoted by the median, a horizontal line dividing the box into two equal halves. The bounds of the box are defined by the lower quartile (25th percentile) and the upper quartile (75th percentile). The whiskers extend from the box and represent the data points that fall within 1.5 times the interquartile range (IQR) from the lower and upper quartiles. Any data point outside this range is considered an outlier and plotted individually. Source data are provided as a Source Data file.

For MERFISH, we excluded ICON1 data from our downstream data analysis owing to low transcript counts and missing cell segmentation in tissue cores and thus analyzed only ICON2 data. We detected six major cell types in MERFISH ICON2 data: macrophages, fibroblasts, endothelial cells, plasma cells, T-cells, and B-cells. We also identified an unknown cell type owing to low gene expression and a lack of differentially expressed genes to annotate the cluster using the clustering-based manual annotation method. Correlation analysis of the cell types displayed a higher correlation among macrophages, fibroblasts, plasma cells, and T-cells than expected (Fig. 7a, Supplementary Fig 3).

Using Xenium-UM with the ICON2 TMA, we detected 14 cell types via manual annotation: alveolar macrophages, classical monocytes, pericytes, alveolar type 2 cells, fibroblasts, plasmacytoid dendritic cells, lymphatic endothelial cells, venous system endothelial cells, smooth muscle cells, tumor cells, mast cells, plasma cells, T cells, and B cells. Differentially expressed genes in each cluster were helpful for annotation with a specific cell type. Xenium-UM was effective at differentiating some cell subtypes, such as lymphatic and venous system endothelial cells (Fig. 7a, Supplementary Fig. 4).

Regarding Xenium-MM with ICON2, we detected dendritic cells, T-cell subsets (CD4⁺ and CD8⁺), and macrophages (interstitial and alveolar) in addition to Xenium-UM via clustering-based manual annotation, but we did not detect plasmacytoid dendritic cells (Fig. 7a). Correlation analysis of the different cell types demonstrated distinct separation between specific cell lineages. We annotated clusters of unique cell types even with the limited number of genes offered by the panel design of the ST platform (Supplementary Fig 5).

Next, we correlated assigned cell types on each platform in the ICON2 TMA using genes shared by the three ST platforms. Assigned cell types in the CosMx ICON2 data were strongly correlated with the same and other assigned cell types in MERFISH and Xenium-UM ICON2 data. The assigned cell types in MERFISH data were weakly correlated with the cell types in Xenium-UM data. Examination of the correlations between assigned cell types in the Xenium-UM and Xenium-MM data demonstrated strong correlations between the same cell types and weak correlations between distant cell lineage types (Supplementary Fig. 6).

For our second cell type annotation approach, we performed automated cell type annotation, a label transfer method from reference scRNA-seq data with MESO TMAs. Reference scRNA-seq was generated with 11 pleural mesothelioma samples and 5 adjacent normal pleural mesothelioma samples that corresponded to a subset of the cases in the MESO TMAs. We annotated some cells as unknown due to low prediction scores (<10%) in CosMx and MERFISH (Fig. 8, Supplementary Fig. 7, 8). Pleural mesothelioma tumor cells have unique biology, thus manual annotation of ST data with mesothelioma samples was challenging. The label transferring method from reference scRNA-seq data performed better than manual cell type annotation with pleural mesothelioma TMAs to identify sub-cell types due to sparsity of ST data and tumor cell marker complexity in pleural mesothelioma (Fig. 8). Xenium-UM and Xenium-MM were equally successful in annotating cell types by indicating the distinct separation among the cell types and lineages in the MESO2 TMA (Supplementary Figs. 9, 10).

Fig. 8 Cell type annotation performance of the ST platforms in the MESO2 TMA. [Images not available. See PDF.]

UMAP and spatial locations of annotated cell types across the platforms. Each color represents a cell type. The cell type labels were transferred from reference scRNA-seq data of pleural mesothelioma cohorts using Seurat in R.

Finally, we examined the correlation of the assigned cell types on each ST platform in the MESO2 TMA using shared genes as we performed for the ICON2 TMA to show the similarity of assigned cell type between ST assays. Most of the assigned cell types of CosMx and MERFISH, and Xenium-UM and MERFISH correlated weakly (R < 0.6) (Supplementary Fig. 11a, b). This is related to low gene expression in MERFISH data. The majority of assigned cell types of CosMx and Xenium_UM correlated moderately (0.6 < R < 0.8) with the same cell type and showed moderate correlation (R > 0.6) with different cell types (Supplementary Fig. 11c). This could be related to the probe sensitivity issue of CosMx. Xenium-UM and Xenium-MM displayed almost perfect correlations for the same cell types (R > 0.95) (Supplementary Fig. 11d). Automated cell type annotation using scRNA-seq of pleural mesothelioma samples produced better correlations of the same cell types on the platforms compared to manual cell type annotation of ICON2 TMA with CosMx and MERFISH (Supplementary Fig. 6).

Visual evaluation of cell phenotyping and segmentation

To assess the performance of cell phenotyping and segmentation on each ST platform, three independent pathologists experienced in immune profiling and image analysis compared the ST images with cell type annotations produced by each platform for the ICON2 TMA with the images of the serial sections of TMAs stained with H&E and mIF for Syto13 (nuclei), CK (epithelium), CD3 (T cells), and CD68 (macrophages). First, the pathologists assessed the presence and distribution of the following cell types: epithelial cells, plasma cells, T cells, macrophages, endothelial cells, and fibroblasts based on the H&E and mIF-stained slides. These cell types were recognized based on morphological cellular and histological characteristics like cell size, nuclear size and features, and general shape of cells visualized with H&E staining, and cellular and subcellular patterns of protein expression of biomarkers visualized using mIF. Then, based on this evaluation, a score was assigned (range, 1-5; lower score represents greater similarity of histological/mIF features) to each tissue core for the overall cell segmentation and the cell type annotation masks (Supplementary Data 6). We averaged the scores generated by 3 independent pathologists to deal with inter-observer variability. Next, we calculated the F1-score to measure of predictive performance of cell type annotation of ST platforms (range, 0–1), accepting the average scores of pathologists’ evaluation as the ground truth, and compared the F1 results for all three platforms (Methods, Supplementary Fig. 12). We excluded Xenium-UM from pathologists’ evaluation due to a lack of cell membrane detection for cell segmentation. Our results demonstrated that F1-scores¹⁷ for cell segmentation were similar among platforms. CosMx had higher variation in the evaluation of cell segmentation, but in general, cell segmentation among the platforms was acceptable. However, Xenium-MM was the only platform with its exact section in H&E staining in the evaluation. We observed better F1-scores with Xenium-MM (median, > 75%) than other ST platforms for all cell types; thus, Xenium-MM data were highly consistent with standard morphological evaluation and cell type annotation results. CosMx had the greatest range of F1-scores, demonstrating that the platform struggled to produce annotations that matched the morphology in certain scenarios. Additional exploration of these scenarios is needed to explain the factors that makeup this wide range of agreement. MERFISH had the lowest F1-scores for all cell types (Fig. 7b).

Discussion

In this study, we performed a systematic comparison of three commercially available single-cell imaging–based ST platforms using archived FFPE lung adenocarcinoma and pleural mesothelioma samples included in TMAs constructed from tissue blocks collected in different years and with annotated information of bulk RNA-seq, which were performed using fresh tissue samples of patients that were in the TMAs, and digital spatial profiling which was performed with serial sections of TMAs. The information on the tissue samples provided by these cutting-edge techniques highlights differences in panel size and gene coverage at the time of the evaluation, imaging workflows, and assay performance by including sensitivity and specificity (background) of probes, cell segmentation, and phenotyping information. Furthermore, we integrated histopathological evaluation of the patterns obtained from different cell phenotypes with the expected cellular patterns. All of the features that we evaluated in this study are key considerations for experimental design of single-cell imaging–based ST assays for immuno-oncology research.

The ST platforms had multiple differences in performance. The first difference was in the ability to produce data on aged tissue samples. Whereas CosMx and MERFISH performed better with newly processed TMAs, Xenium performed uniformly across different tissue ages. The panel sizes and gene selections that we used differed substantially (CosMx, 1000-plex; MERFISH, 500-plex; Xenium, 289-plex lung panel + 50 custom genes). Only 93 genes were shared by all three platforms. Researchers should investigate the panel design before running an assay to understand whether the panel covers their genes of interest. Observation of the highest transcript counts and the highest number of uniquely expressed genes per cell with CosMx may be due to the transcript detection chemistry and probe design strategies.

Tissue preparation and coverage of the whole tissue area were challenging. CosMx required less training to mount tissue samples to Superfrost slides than MERFISH and Xenium for mounting tissue to circular and Xenium-specific slides, respectively. After tissue samples were mounted to assay-specific slides, the selection of the FOV under the guidance of a pathologist using morphology marker staining was required with CosMx. This step reduced the coverage area on this platform. In this study, we used TMAs constructed from multiple samples; thus, CosMx did not cover some of the tissue cores owing to the selection of FOVs. We analyzed whole tissue cores mounted to slides using MERFISH and Xenium.

Negative control and blank probes were necessary for estimating the background signal of the assays. We observed high expression of negative control probes in the CosMx assays. We could not include MERFISH in this comparison owing to a lack of negative control probes. High expression of negative controls represented the background or noise level in the CosMx assays. Later, we calculated FDRs on each platform using negative control and blank probes separately for each assay to estimate the error rates on each platform. The FDRs for CosMx and MERFISH were high in all TMAs. Xenium displayed low expression of negative controls in all TMAs; thus, the FDR for Xenium with both negative control and blank probes was low. Overall, a lack of negative control and blank probes in the panel design limited our ability to evaluate the signal efficiency of detected gene expressions with ST assays. In addition, the sensitivity and specificity of probe design are critical for ST assays. While high expression of negative controls increases the false discovery rates, non-specific probe bindings can cause a high background signal, and high transcript counts per cell, which could decrease the accuracy of cell type annotations.

We evaluated the correlation of ST data with bulk RNA-seq and DSP-WTA data to understand whether there was a similar gene expression pattern between these different assays. CosMx and MERFISH showed a reduced correlation coefficient with bulk RNA-seq than Xenium-UM and Xenium-MM in the older TMA (ICON2). This suggests that aging in TMA blocks may reduce the detection of RNA more in CosMx and MERFISH than Xenium-UM and Xenium-MM. However, all ST platforms displayed a similar correlation coefficient with bulk RNA-seq with the newly constructed TMA (MESO2). There were no differences between the correlation coefficients of Xenium-UM and Xenium-MM assays with old and new TMAs. It is important to note that even though there are tissue processing differences between bulk RNA-seq data, which is performed with extracted RNA from fresh frozen tissue and is sequencing based, and ST data which is performed with FFPE tissue and is imaging-based, the correlation between these assays displayed moderate similarity. DSP WTA data is sequencing-based RNA-seq from FFPE tissue samples but lacks an RNA extraction step in the sample preparation protocol. Having DSP data from FFPE tissue without an RNA extraction step supports the stronger correlations between ST data and DSP WTA data in both old and new TMAs. However, we cannot rule out the impact of differences in the location of the tumor used to generate the bulk RNA-seq, given the archival nature of the samples selected.

Accomplishing accurate cell segmentation has been a hurdle with single-cell imaging–based ST platforms. Each platform tries to solve the problem of cell segmentation using different strategies. We found that cell-segmentation algorithms with morphology marker staining that benefited from nucleus, cell membrane, and cytoplasm staining to identify boundaries of nucleus and cell boundaries (Methods) performed better than the cell boundary expansion method did. However, these algorithms depended on the quality of the morphology marker staining. When a problem in the staining step was encountered, cell segmentation was inconsistent owing to the background for the morphology marker staining and generated cells without transcript counts as we observed with the MERFISH assay. Xenium-UM was the only method without morphology marker staining, and it had inaccurate cell segmentation, especially in tissue regions with air spaces. Although we reduced the cell size for Xenium-UM to no more than 7 μm in the cell segmentation step, the cells were larger than in other assays with morphology marker staining. Inaccurate cell segmentation causes the transcripts to be assigned incorrectly to neighboring cells. Cell segmentation accuracy of ST assays may be increased using additional data in the cell segmentation algorithms besides the morphology marker staining.

Precise cell type annotation was crucial for downstream analysis and was affected by all previously mentioned steps above, including panel design and cell segmentation. We performed a cluster-based manual cell type annotation approach to annotate cell types with lung adenocarcinoma samples when scRNA-seq results of patients were not available. We considered different cell markers to annotate cell types manually on each platform owing to differences in panel design. The noise level in the CosMx assay due to non-specific probe attachment eliminated the distinct, highly expressed cell-type-specific genes in the clusters; thus, we could not separate B cells from T cells and labeled a cluster as unknown because of the expression of multiple distinct cell markers in CosMx. MERFISH had low gene expression; thus, one of the clusters in MERFISH did not have a clear cell type indicator and was labeled as unknown. The low noise level in Xenium helped us to easily annotate clusters with a cell type based on available gene expression. Better cell segmentation with Xenium-MM than Xenium-UM increased the level of cell type annotation from coarse cell lineages to cell subtypes for T cells. We tried to annotate lung adenocarcinoma samples with public reference datasets and an automated cell type annotation algorithm using the TransferData() option in Seurat, but the results were inadequate owing to differences between the cohorts of the reference dataset and the ST data. For the pleural mesothelioma samples, we benefited from having matched scRNA-seq data to annotate the ST data via Seurat co-embedding and label transfer. CosMx and MERFISH had low prediction scores for some cell types owing to high noise levels or low gene expression; hence, we labeled the cell types as unknown as we did previously. Having reference scRNA-seq data from the same patients increased the refinement of identified cell types compared to only predicted high-level lineage phenotyping from ST gene expressions. Cell type annotation is essential for the examination of cell-cell interactions and other downstream analyses.

Co-expression of disjoint gene pairs in the cells highlighted the importance of cell segmentation in ST assays. All ST assays had cells that co-expressed gene pairs. MERFISH had the lowest percentage of cells with co-expression of gene pairs compared to the other ST platforms. This could be related to cell segmentation errors. In addition, the cell type annotation step in the ICON dataset struggled to separate B cells from T cells in CosMx, and to separate T cell subtypes in CosMx, MERFISH and Xenium-UM. In some instances, the high percentage of co-expressions of disjoint gene pairs could be explained biologically, for example T cells can co-express CD4 and CD8A outside the thymus in pathogenic states such as cancer¹⁸.

Furthermore, we performed a histological evaluation of the phenotype patterns on each platform identified by pathologists, skilled in immune profiling, to reveal any issues that may have been hidden in the large amount of data produced by these platforms. For this assessment, we leveraged H&E staining and mIF with serial sections of samples. The cellular and architectural patterns of tumor cells and nonmalignant cells, including immune cells, are very well known by pathologists and immunologists¹⁹. We performed this evaluation to validate the data obtained with these platforms, understand the limitations of the outputs of ST platforms, and, most importantly, set the basis for a workflow that can integrate bioinformatics and immune and molecular information with histological categorization of tissues for immune-oncology research.

Although an H&E stain’s ability to show the difference between a CD4-positive and a CD8-positive cell is null, its power to distinguish cell types that have been staples of histology for centuries can be easily examined to increase or gain some confidence in the cell type annotations produced to match the expected cell types in every experimental run. Also, our assessment of the phenotype patterns on ST data was supported by reviewing key biomarkers on mIF images of serial sections of samples to assist pathological evaluation. The ST platforms differed in the number of genes and probes per gene and in gene selection itself, resulting in differences in their ability to produce a dataset that yielded consistent results of their final annotations when compared with each other. Xenium-MM had the highest F1-scores for every phenotype interrogated, whereas MERFISH had the lowest. Only Xenium allowed us to obtain an H&E staining in the same section used for the assay at the time of our experimental run; thereby facilitating histological evaluation of morphological features was performed with H&E staining of first section of TMAs. These simplified cell type annotations can be challenging to assess without immunology and pathology experience and depending on the experimental design and populations of interest. We recommend visual inspection of the cell in a simplified annotation that encompasses histologically evaluable cell types to evaluate the results of ST data that do not stray from gold standards. This approach can be easily performed on all ST platforms, as they all have the basic lineage gene markers required to perform simplified annotations.

This work has limitations; it was conducted in samples from a single institution, lacks the inclusion of a broader number of tumor types, whole tissue sections, field of view selection in CosMx or samples with special tissue processing protocols such as decalcification, these are variables that strongly affect assay performance of any tissue-based assays including single cell ST. In addition, the lack of negative controls or blank probes in some assays limited our ability to properly evaluate the quality of the produced transcript counts. The assays cannot be performed using the same tissue sections, thus, performing ST assays on adjacent serial sections of TMAs limited the replication of tissue patterns. Also, our study did not include the same tissue type for different ages of TMAs due to the lack of availability of these materials. Therefore, the differences between the ages of TMAs are only informative. Further studies considering these special pre-analytical variables are warranted.

The spatial transcriptomics field is evolving rapidly. Multiple commercial companies released ST assays with more gene coverage in their panels (i.e: CosMx 6K, Xenium 5K) that are being tested by researchers. Moving forward, we propose that increasing the number of probes per gene to get specific transcript counts per gene would improve cell type annotation capabilities. Finally, integration of protein assays with single-cell spatial transcriptomic assays within the same plane or a serial plane can increase the strength of cell type annotations and downstream data analyses, refining our understanding of tissue architecture and cell-cell interactions.

Methods

This study was approved by the MD Anderson Institutional Review Board and was conducted according to the principles of the Declaration of Helsinki for mesothelioma (Lab08-0380) and lung adenocarcinoma (PA15-1112).

Panel design

A predesigned CosMx 1,000-plex panel, MERFISH 500-plex immuno-oncology panel, and Xenium 289-plex human lung panel were selected for this study. Fifty custom genes were added to the Xenium panel: ARG1, B2M, CALB2, CCL18, CCL19, CCL2, CCL21, CCL3, CCL4, CCL5, CCL8, CD276, CD33, CD44, CEACAM8, CR2, CXCL11, ENTPD1, FAP, FCER2, HLA-DRB1, HSPA6, ICOS, IDO1, IFNG, IGHG1, IGHG4, ITGAX, KRT19, LAMP3, MSLN, NCAM1, NT5E, PTPRC, TCF7, TGFB1, TGFB2, TGFB3, TIGIT, TMEM173, TNFRSF9, TNFSF4, TNFSF9, TOX, TRDC, TREM1, VEGFA, VSIR, VTCN1, and ZNF683.

Sample preparation

Four TMAs of FFPE tissue from surgically resected tumor samples obtained from patients with lung adenocarcinoma (n = 22; 2 TMAs, 53 cores [ICON1 and ICON2]) or pleural mesothelioma (n = 22; 2 TMAs, 49 cores [MESO1 and MESO2]) were used. TMA construction was performed using 1-mm-diameter cores from FFPE tumor tissue blocks, with up to three cores per sample. ICON TMA tissues were collected from 2016 to 2018, whereas MESO TMA tissues were collected from 2020 to 2022. All samples were obtained at The University of Texas MD Anderson Cancer Center.

TMA cores in the ICON set occupied an area of 17.0 mm  ×  13.5 mm in the TMA receiver block, whereas TMA cores in the pleural mesothelioma set occupied an area of 12 mm  ×  7 mm.

Tissue preparation for ST assays

FFPE TMA blocks were sectioned and prepared for each assay according to the ST platform manufacturers’ instructions, as described briefly below.

CosMx

CosMx uses Superfrost Plus micro slides for the placement of tissue. NanoString guidelines required that tissue be placed in the center of a slide in a 20 mm × 15 mm area, avoiding excess paraffin or tissue on the slide area outside of the scanner area to prevent processing of tissue in a fiducial area²⁰. ICON TMAs were scored to select the area of interest, and two TMA rows were excluded for further analysis owing to scanning-area constraints. Pleural mesothelioma TMA blocks were only scored to avoid excess paraffin outside of the scanning area; no cores were excluded. After scoring, sections were cut at 4-5 μm and placed in nuclease-free water, excess tissue was removed using a brush, and the selected tissue was aligned at the center of the slide in a 20 mm × 15 mm area. Slides were then dried at room temperature overnight before storage at 4 °C and then were shipped within a week to the NanoString Technology Access Program. Four slides were sent to NanoString for this project.

MERFISH

The MERFISH assay is used with special circular slides provided by the manufacturer and requires preparation with an FFPE fiducial premix solution before sectioning in a specific protocol with training required by the provider²¹. Tissue sections were prepared at a thickness of 4-5 μm, floated in a nuclease-free water bath, and separated for pickup. The tissue was then picked up, making sure it was within a 12.6 mm × 15.4 mm area at the center of the slide with the help of a brush. The ICON2 TMA was fitted within the area after excluding one column of tissue from the mounting area. Pleural mesothelioma TMA blocks were only scored to avoid excess paraffin outside the scanning area; no cores were excluded. Also, one TMA (MESO2) was fitted on a single slide. The tissue sections were dried in an oven at 60 °C for 10 min and were then immediately ready for shipping. Slides cushioned in Petri dishes were shipped to Vizgen within 2 weeks of sectioning. The program was restricted to three slides per project.

Xenium

Special slides had to be purchased by the manufacturer for tissue placement. These slides had an area of 12 mm × 24 mm marked by fiducials. After scoring the blocks for the area of interest, each block was sectioned at 4–5 μm, floated in a nuclease-free water bath, and separated before picking up the tissue, making sure no part of the tissue or paraffin was touching the fiducials. The ICON2 TMA was scored to select the area of interest (matching the area with the CosMx assay), two TMA rows were excluded for further analysis, and only one TMA (ICON2) fit within the fiducials. Pleural mesothelioma TMA blocks were only scored to avoid excess paraffin outside the scanning area, no cores were excluded, and two TMAs were fitted on a single slide. Next, the slides were incubated at 42 °C for 3 h and dried overnight at room temperature inside a desiccator before shipping²². Within 1 week, the slides were sent to the 10x Genomics Catalyst program²³, which was restricted to only two slides per project. Both unimodal and multimodal segmentation methods were used with Xenium.

Data delivery and bioinformatics workflow

The three ST platforms differed in their scanning capabilities and data delivery packages, which required different pipelines and workarounds for downstream analysis. A presentation meeting for the data delivery, consisting of the quality control parameters, representative images of the tissues analyzed, and analysis results with UMAP and phenotype annotations from each assay, was directed by the platform manufacturers’ in-house pathology and data science teams who performed the assays.

CosMx

CosMx included whole tissue imaging only for the FOV selection. were manually placed by an operator with a maximum area of 545 μm x 545 μm (at the moment of the experimental run) and selected according to the specific needs and aims of the project. For morphology visualization, mIF with DAPI, PanCk, CD3, and CD45 was performed using the “CosMx® Human Universal Cell Segmentation Kit (RNA)” (Part number 121500020). The NanoString Spatial Molecular Imaging Technology Access Program had an FOV limit of 25 per slide; however, for this project, the vendor selected a total of 165 FOVs on four slides. For data delivery, NanoString provided the outputs of their in-house pipelines including whole tissue images for FOV selection, high-resolution FL images per FOV, raw channel images of each morphology stain, raw data that included metadata, expression matrix, FOV positions, polygon files that could be used with third-party software, cell composite, cell labels, cell overlay files and Seurat objects, analysis results, napari input files with a custom CosMx plug-in to visualize analysis results, and summary PowerPoint files. Tissue morphology, the captured probes, the FOVs selected for analysis, cell segmentation, and analyzed data from the NanoString in-house pipeline were examined, and morphology scans were exported as OME-TIFF files for use in different digital image analysis suites using scripts within the custom CosMx plug-in in napari.

MERFISH

The MERFISH platform has a maximum scan capture area of 1 cm², which must be selected by the user. For data delivery, Vizgen provided the outputs of their in-house pipelines, including cellpose²⁴ cell-segmentation results in parquet format with selected regions for data analysis using the best morphology stains to generate cell-by-gene and transcript-by-location matrices; low-resolution whole slide images and high-resolution mIF images, consisting of the DAPI nuclear stain and a cocktail of morphology markers labeled as PolyT, Cellbound1, Cellbound2, and Cellbound3 (MERSCOPE Cell Boundary Stain Kit, PART NUMBER 10400118); raw images of samples in OME-TIFF file format; data analysis results; and MERFISH input files to explore data using the MERSCOPE Visualizer desktop application.

Xenium

The Xenium in situ platform provided full tissue imaging within a scanning area of 12 mm × 24 mm. For the Xenium-UM assay, slides were stained with only a nuclear stain (DAPI); Later, cell segmentation was performed using an expansion algorithm. For the Xenium-MM assay, morphology marker staining with a combination of nuclear (DAPI, 5 mg/ml), “Boundary” (ATP1A1, E-cadherin, and CD45), and “Interior” (18S ribosomal RNA) stain cocktails were performed to identify nucleus, cell membrane boundaries and cytoplasm, later these staining results and nuclear expansion algorithm up to 5 μm assisted the multi-modal cell-segmentation algorithm²⁵. For data delivery, the 10X Genomics Catalyst program provided superimposed H&E-stained images of the TMAs and the raw system outputs, which included the transcript locations without any filtering, as well as segmentation files and analyzed data using their own pipelines to visualize data in Xenium Explorer, which allowed for visualization of the transcripts, and the morphological staining, which included stains for DAPI and a cocktail of visualization markers.

Data analysis workflow

Cell segmentation

Cell-segmentation results for CosMx (enhanced cellpose algorithm²⁴), MERFISH (cellpose cell segmentation), and Xenium-MM (custom deep learning models²⁵) were provided by the manufacturers, and their outputs were used in the data analysis. Cell segmentation algorithms of CosMx, MERFISH and Xenium-MM generated cell segmentation results with guidance of morphology marker staining. However, the cell-segmentation algorithm of Xenium-UM was rerun with modified parameters using the 10X Genomics Xenium Ranger pipeline (version 1.7.0.2) to eliminate larger cells (final parameters: --expansion-distance = 7, --dapi-filter = 75).

Preprocessing, clustering, and annotation

Quality control and data filtering were performed for each ST dataset based on criteria suggested by the ST platform manufacturers. For CosMx, cells with areas more than five times the geometric mean of the cell area and fewer than 30 transcript counts were filtered. For MERFISH, Xenium-UM, and Xenium-MM, cells with fewer than 10 transcript counts per cell were filtered. These filtering steps eased cell type annotation while generating gaps between cells.

Normalization, dimensional reduction, and cell type annotation

Standard log-normalization was performed for each dataset using the NormalizeData function in Seurat (version 5)²⁶. The dimensionality of each dataset was reduced using the RunPCA function with default parameters and including all targeted genes. TMAs containing samples of the same cancer type were integrated with harmony²⁷. The dimensionality was further reduced to two dimensions using the RunUMAP function, and cells were clustered using the FindNeighbors function and the FindClusters function at 0.8 resolution with the Louvain algorithm.

Cells in the ICON TMAs were annotated manually by identifying differentially expressed genes in each cluster using the FindAllMarkers function (Wilcoxon test implemented using the Presto package²⁸). The following markers were used to annotate the clusters: EPCAM and PIGR for tumor cells; CD19, MS4A1 and BANK1 for B cells; MZB1 and JCHAIN for plasma cells; CD3E for T cells, CD68 and CD163 for macrophages when the gene expression information was available for the ST platform.

scRNA-seq was performed from fresh tumor (n = 11) and adjacent normal (n = 5) tissue samples from patients with pleural mesothelioma. Data was preprocessed with Seurat²⁶ standard scRNA-seq pipeline and annotated clusters manually based on highly differentially expressed genes in each cluster. Cells were annotated using the label transfer method including the FindTransferAnchors and TransferData functions in Seurat with default parameters from scRNA-seq data of pleural mesothelioma data for the MESO TMAs. Results were evaluated by performing differentially expressed genes in the predicted cell types.

Bulk RNA-seq and DSP WTA concordance analysis

Bulk RNA-seq data was generated from frozen tumor tissue samples of patients that were also in the ICON and MESO TMAs. Data was processed with STAR aligner²⁹ to align raw reads to hg19 version of Human reference genome, featureCounts³⁰ and normalized with varianceStabilizingTransformation (VST)³¹ method. Later, we correlated normalized read counts of common genes of bulk RNA-seq and ST data with Pearson’s correlation.

DSP WTA data was generated using another serial section of TMAs that were used in this manuscript. DSP WTA data was segmented as tumor and TME sections by pathologists with morphology marker staining. Data was normalized with RUVg³² normalization method. We added the normalized gene expressions of DSP WTA from Tumor and TME segments of same tissue cores. Later, we correlated normalized read counts of common genes of DSP WTA and ST data with Pearson’s correlation. Figures were generated using ggplot2³³ (version 3.5.1), tidyverse³⁴ (version 2.0.0), ggbreak³⁵ (version 0.1.4), reshape2³⁶ (version 1.4.4) and ggpubr³⁷ (version 0.6.0).

Statistics and reproducibility

Wilcoxon signed rank test was performed with paired datasets and Mann–Whitney U test with unpaired dataset. No statistical method was used to predetermine sample size. No data were excluded from the analyses. The experiments were not randomized. Investigators were blinded in the data analysis.

Pathological evaluation of cell segmentation and phenotyping

After performing cell-phenotyping annotations for all assays, each dataset was organized and compared using a best possible result approach to evaluating morphology across the ST platforms. Specifically, the data for each platform were compared by three independent pathologists, agnostic to all assay information, using the ICON2 TMA, which consisted of cores obtained exclusively from patients with non-small cell lung cancer. Cores without tumor tissue were excluded. For the evaluation, the resulting clusters of the phenotyping were merged into simplified annotation groups that consisted of different types of cells that can be identified easily with basic histology, with the assistance of mIF staining for groups harder to identify solely on morphology by H&E staining. The simplified annotated images consisted of the following phenotypes: tumor cells independent of cluster (cyan), T cells (red), plasma cells (purple), macrophages (orange), fibroblasts and smooth muscle cells (yellow), endothelial cells (green), and other cells (gray or maroon). For CosMx, the cell type annotation was compared to the H&E stained section that was closest to the section of ST data, and mIF imaging of the GeoMx DSP WTA assay stained for SYTO13 (DNA, Nanostring, 121301310, concentration 500 nM with 1:10 dilution), panCK (PanCK AF532: Clone: AE1 + AE3; Novus NBP2-33200AF532; nanoString 121300320-020223; concentration at 0.5 µg/ml with 1:40 dilution), CD3E (CD3 AF647: Clone: UMAB54; Origene AC211369; CG223985F005AF; concentration at 0.5 µg/ml with 1:2000 dilution) and CD68 (CD68 AF594: Clone: KP1; Santa Cruz AF594; E3024; dilution 0.5 µg/ml with 1:400 dilution). We validated the standard fluorescence assay of the GeoMx DSP assay using human tonsil tissue. Antibodies were validated for use in human tissue^38,39. We also optimized visualization settings by adjusting black and white signal intensity thresholds in the HALO (Indica Labs, version 3.4) digital image analysis software per core (M.M.A., L.M.S.S.) to maximize signal-to-noise ratio in accordance with expected histological patterns. These settings of visualization were shared among all pathologists for evaluation. For MERFISH, a similar arrangement was used, but the closest H&E-stained section to ST data was a Xenium H&E for the cores where it was available; otherwise, the image came from the next section nearest to ST data. For Xenium, the post-Xenium H&E staining was used for scoring the comparison. Scanning settings for HE images not originated by the Xenium assay were scanned in a Leica Systems Aperio AT2 scanner, at 40X without z stacking.

Accuracy scores of 1–5 (1 = best, 5 = worst) were assigned by each pathologist in evaluating the accuracy of the cell-segmentation mask produced by each assay and previously described morphological features. The pathologists were blinded to the diagnosis, tissue core number and position, and each other’s scores during the evaluation. The pathologists’ scores for each ST platform were averaged, with no significant discrepancies.

Calculation of FDRs and F1-scores

The CosMx panel included 10 negative control probes, the MERFISH panel included 50 blank probes, and the Xenium panel included 20 negative control probes, 41 negative control code words, and 141 blank code words. FDR values were calculated for each ST platform using negative control and blank probes individually and the following formula^14,15,40:

1

$${FDR}_c=\frac{N_c}{N_{tol}}\times \frac{G_{tol}}{G_c}\times 100$$

In this formula, N_tol represents the total read count, and G_tol denotes the total number of probes. For assays with negative control probes, N_c and G_c refer to the total negative control read counts and number of negative control probes, respectively. Similarly, for assays with blank probes, N_c and G_c corresponded to the total blank read counts and number of blank probes, respectively.

Three independent pathologists evaluated cell segmentation and cell type annotation results of bioinformatics pipelines to give scores from 1 to 5 using H&E staining and mIF images individually by following same guidelines. The average scores of the pathologists were normalized by setting precision calls to 1, and the cell type annotation scores (recalls) were calculated by taking the reciprocal of the normalized average scores. These values were subsequently used to compute F1-scores.

F1-scores were calculated for cell segmentation and each cell type, using the average scores of the three independent pathologists as precision scores and the cell type annotation results as recall with the following formula⁴¹:

2

$$F1-score=\frac{2\left(Precision\times Recall\right)}{\left(Precision+Recall\right)}$$

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.

Acknowledgements

This study was supported by Lung SPORE grant P50 CA070907 from the NIH to J.H., the Aileen M. Dillon & Lee M. Bourg Mesothelioma Fund, the Re & RM Kennedy Lung Cancer Fund, the Fleming Endowed Fund, the MD Anderson Rare Tumor Initiative, the MD Anderson Translational Molecular Pathology-Immunoprofiling Lab (TMP-IL) Moon Shots Platform to C.H. and L.M.S.S., NCI grant U24CA224285 (to the MD Anderson Cancer Center CIMAC) to C.H. and I.W. and the CCSG Research Histology Core Laboratory and the Advanced Technology Genomics Core supported by the NIH/NCI P30CA016672 award. We thank the MD Anderson Department of Thoracic and Cardiovascular Surgery nurses and staff for their support of this study. The authors acknowledge Donald Norwood, members of the Editing Services, Research Medical Library for editing this manuscript. Figure 1 is created in BioRender. Ozirmak lermi, N. (2025) https://BioRender.com/y68w242.

Author contributions

L.M.S.S., C.H., K.C., N.O.L., and M.M.A. conceptualized and designed the experiments. S.H., A.S., K.K., W.L., S.B., M.J., L.M.S.S., RTI Team, J.H., J.Z., B.S., T.C., A.T., M.A., R.M., D.G., M.G.R., X.T., and M.M.A. prepared and organized data. A.S., S.H., M.M.A., I.L., L.H., L.M.S.S., K.T., RTI Team, J.H., A.W., J.Z., B.S., T.C., A.T., M.A., R.M., D.G., M.G.R., and X.T. generated data. N.O.L., J.D., Q.L. and K.C. performed bioinformatic data analyses. I.L., L.H., A.S., S.H., L.M.S.S., and M.M.A. performed pathology data analyses. N.O.L. and M.M.A. prepared figures and legends and wrote the manuscript. K.T., B.S-E., I.W., C.H., L.M.S.S. administered the project. C.H., L.M.S.S., I.W., B.S-E. acquired funding. C.H., K.C., L.M.S.S. supervised the project. All authors reviewed, read, and agreed to publish the manuscript. N.O.L. and M.M.A. contributed equally and all reserve the right to list themselves first on their curricula vitae.

Peer review

Peer review information

Nature Communications thanks Seunghee Kim-Schulze and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.

Data availability

The data that support the findings of this study, which were CosMx SMI, MERFISH and Xenium raw data have been deposited in the Gene Expression Ominibus (GEO) database (https://www.ncbi.nlm.nih.gov/geo/) under accession code GSE299786, GSE299886 and GSE300007, respectively.  are provided with this paper.

Code availability

All data analysis was conducted using publicly available software, packages, and tools in R (version 4.3.1) (https://www.r-project.org/), as described in the Methods section. The code used to perform the analyses and generate results in this study is publicly available and has been deposited in GitHub repository at https://github.com/KChen-lab/ST_Comparison under GPL-3.0 license, and Zenodo (https://zenodo.org/records/15880732)⁴².

Competing interests

C.H. declares research funding to institution from Sanofi, BTG, Iovance, Obsidian, KSQ, EMD Serono, Takeda, Genentech, BMS, Summit Therapeutics, Artidis, Immunogenesis and Novartis; scientific advisory board member of Briacell with stock options; personal fees from Regeneron outside the scope of the submitted work. L.M.S.S. declares research funding to institution from Theolytics, advisory role/consulting fees from BioNTech, travel support for participation in 10x Genomic Pathology Day event and participation in NanoString Roadshow event, both unrelated to this work. M.A. declares research funding to institution from Genentech, Nektar Therapeutics, Merck, GlaxoSmithKline, Novartis, Jounce Therapeutics, Bristol Myers Squibb, Eli Lilly, Adaptimmune, Shattuck Lab, Gilead, Verismo therapeutics, Lyell; scientific advisory board member of GlaxoSmithKline, Shattuck Lab, Bristol Myers Squibb, AstraZeneca, Insightec, Regeneron, Genprex; personal fees from AstraZeneca, Nektar Therapeutics, SITC; participation of safety review committee for Nanobiotix-MDA Alliance, Henlius outside the scope of the submitted work. J.Z. declares research funding from Johnson and Johnson, Helius, Merck, Novartis and Summit, honoraria and consulting fees from AstraZeneca, BeiGene, Catalyst, GenePlus, Helius, Innovent, Johnson and Johnson, Novartis, Takeda and Varian outside the submitted work. D.G. has served on scientific advisory committees for Sanofi, Menarini Ricerche, Onconova, and Eli Lilly, and has received research support from Takeda, NGM Biopharmaceuticals, Boehringer Ingelheim and AstraZeneca. T.C. has received over the past 24 months speaker fees/honoraria (including travel/meeting expenses) from ASCO Post, AstraZeneca, Bio Ascend, Bristol Myers Squibb, Clinical Care Options, IDEOlogy Health, Medical Educator Consortium, Medscape, OncLive, PEAK Medicals, PeerView, Physicians’ Education Resource, Targeted Oncology; advisory role/consulting fees (including travel/meeting expenses) from AstraZeneca, Bristol Myers Squibb, Genentech, Merck, oNKo-innate, Pfizer, and RAPT Therapeutics; institutional research funding from AstraZeneca and Bristol Myers Squibb. All other authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41467-025-63414-1.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. Marx, V. Method of the Year: spatially resolved transcriptomics. Nat. Methods; 2021; 18, pp. 9-14.1:CAS:528:DC%2BB3MXnsFWhsA%3D%3D [DOI: https://dx.doi.org/10.1038/s41592-020-01033-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33408395]

2. Black, S et al. CODEX multiplexed tissue imaging with DNA-conjugated antibodies. Nat. Protoc.; 2021; 16, pp. 3802-3835.1:CAS:528:DC%2BB3MXhsVOksLvL [DOI: https://dx.doi.org/10.1038/s41596-021-00556-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34215862][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8647621]

3. Gröbner, SN et al. The landscape of genomic alterations across childhood cancers. Nature; 2018; 555, pp. 321-327.2018Natur.555.321G [DOI: https://dx.doi.org/10.1038/nature25480] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/29489754]

4. Chen, KH; Boettiger, AN; Moffitt, JR; Wang, S; Zhuang, X. RNA imaging. Spatially resolved, highly multiplexed RNA profiling in single cells. Science; 2015; 348, [DOI: https://dx.doi.org/10.1126/science.aaa6090] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25858977][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4662681]aaa6090.

5. Tian, L; Chen, F; Macosko, EZ. The expanding vistas of spatial transcriptomics. Nat. Biotechnol.; 2023; 41, pp. 773-782.1:CAS:528:DC%2BB38XisFCgur3P [DOI: https://dx.doi.org/10.1038/s41587-022-01448-2] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36192637]

6. He, S et al. High-plex imaging of RNA and proteins at subcellular resolution in fixed tissue by spatial molecular imaging. Nat. Biotechnol.; 2022; 40, pp. 1794-1806.1:CAS:528:DC%2BB38XisFygu7bM [DOI: https://dx.doi.org/10.1038/s41587-022-01483-z] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36203011]

7. Janesick, A et al. High resolution mapping of the tumor microenvironment using integrated single-cell, spatial and in situ analysis. Nat. Commun.; 2023; 14, pp. 8353-8353.2023NatCo.14.8353J1:CAS:528:DC%2BB2cXlslw%3D [DOI: https://dx.doi.org/10.1038/s41467-023-43458-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/38114474][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10730913]

8. Moffitt, J. R. et al. Molecular, spatial, and functional single-cell profiling of the hypothalamic preoptic region. Science362 (2018).

9. Wang, H. et al. Systematic benchmarking of imaging spatial transcriptomics platforms in FFPE tissues. bioRxivhttps://doi.org/10.1101/2023.12.07.570603 (2023).

10. Cook, D. P. et al. A comparative analysis of imaging-based spatial transcriptomics platforms. bioRxivhttps://doi.org/10.1101/2023.12.13.571385 (2023).

11. Hartman, A. & Satija, R. Comparative analysis of multiplexed in situ gene expression profiling technologies. bioRxivhttps://doi.org/10.1101/2024.01.11.575135 (2024).

12. Ren, P. et al. Systematic benchmarking of high-throughput subcellular spatial transcriptomics platforms. bioRxivhttps://doi.org/10.1101/2024.12.23.630033 (2024).

13. Pang, J-MB et al. Spatial transcriptomics and the anatomical pathologist: molecular meets morphology. Histopathology; 2024; 84, pp. 577-586. [DOI: https://dx.doi.org/10.1111/his.15093] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37991396]

14. Benjamini, Y; Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Ser. B: Methodol.; 1995; 57, pp. 289-300.1325392 [DOI: https://dx.doi.org/10.1111/j.2517-6161.1995.tb02031.x]

15. Svozil, J. & Baerenfaller, K. in Methods in Enzymology Ch. 21 (ed Shukla, A. K.) (Academic Press, 2017).

16. Genomics x. Best Panelkit Available: Xenium Panels - 10x Genomics (2024).

17. Goutte, C. & Gaussier E. A probabilistic interpretation of precision, recall and F-score, with implication for evaluation. In: Advances in Information Retrieval (eds Losada, D. E. & Fernández-Luna, J. M.) (Springer Berlin Heidelberg, 2005).

18. Nah, EH; King, DE; Craig, FE. CD4 and CD8 antigen coexpression: a flow cytometric study of peripheral blood, bone marrow, body fluid, and solid lymphoreticular specimens. Arch. Pathol. Lab Med.; 1997; 121, pp. 381-384.1:STN:280:DyaK2s3osFSgtg%3D%3D [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/9140307]

19. Giraldo, NA; Peske, JD; Sautès-Fridman, C; Fridman, WH. Integrating histopathology, immune biomarkers, and molecular subgroups in solid cancer: the next step in precision oncology. Virchows Arch.: Int. J. Pathol.; 2019; 474, pp. 463-474. [DOI: https://dx.doi.org/10.1007/s00428-018-02517-1]

20. CosMx SMI Slide Preparation for RNA Assays MAN-10184-05. Bruker (2024).

21. MERSCOPE® User Guide:Formalin-Fixed Paraffin-Embedded Tissue Sample Preparation. Vizgen (2024).

22. Xenium slide preparation. 10X Genomics (2024).

23. Xenium Catalyst Program. 10X Genomics (2024).

24. Stringer, C; Wang, T; Michaelos, M; Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nat. methods; 2021; 18, pp. 100-106.1:CAS:528:DC%2BB3cXis1Sgs77K [DOI: https://dx.doi.org/10.1038/s41592-020-01018-x] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33318659]

25. Xenium In Situ Multimodal Cell Segmentation: Workflow and Data Highlights. 10X Genomics (2024).

26. Hao, Y et al. Dictionary learning for integrative, multimodal and scalable single-cell analysis. Nat. Biotechnol.; 2024; 42, pp. 293-304.1:CAS:528:DC%2BB3sXhtFSisbvO [DOI: https://dx.doi.org/10.1038/s41587-023-01767-y] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/37231261]

27. Korsunsky, I et al. Fast, sensitive and accurate integration of single-cell data with Harmony. Nat. methods; 2019; 16, pp. 1289-1296.1:CAS:528:DC%2BC1MXitFOqsr7N [DOI: https://dx.doi.org/10.1038/s41592-019-0619-0] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/31740819][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6884693]

28. Korsunsky I., Nathan A., Millard N., Raychaudhuri S. Presto scales Wilcoxon and auROC analyses to millions of observations. bioRxiv 653253 (2019).

29. Dobin, A et al. STAR: ultrafast universal RNA-seq aligner. Bioinformatics; 2013; 29, pp. 15-21.1:CAS:528:DC%2BC38XhvV2gsbnF [DOI: https://dx.doi.org/10.1093/bioinformatics/bts635] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23104886]

30. Liao, Y; Smyth, GK; Shi, W. featureCounts: an efficient general purpose program for assigning sequence reads to genomic features. Bioinformatics; 2014; 30, pp. 923-930.1:CAS:528:DC%2BC2cXltFGqu7c%3D [DOI: https://dx.doi.org/10.1093/bioinformatics/btt656] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/24227677]

31. Love, MI; Huber, W; Anders, S. Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2. Genome Biol.; 2014; 15, [DOI: https://dx.doi.org/10.1186/s13059-014-0550-8] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25516281][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4302049]550.

32. Risso, D; Ngai, J; Speed, TP; Dudoit, S. Normalization of RNA-seq data using factor analysis of control genes or samples. Nat. Biotechnol.; 2014; 32, pp. 896-902.1:CAS:528:DC%2BC2cXhsVSqtLnI [DOI: https://dx.doi.org/10.1038/nbt.2931] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/25150836][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4404308]

33. Valero-Mora, PM. ggplot2: elegant graphics for data analysis. J. Stat. Softw. Book Rev.; 2010; 35, pp. 1-3.

34. Wickham, H et al. Welcome to the Tidyverse. J. Open Source Softw.; 2019; 4, 1686.2019JOSS..4.1686W [DOI: https://dx.doi.org/10.21105/joss.01686]

35. Xu, S et al. Use ggbreak to effectively utilize plotting space to deal with large datasets and outliers. Front. Genet.; 2021; 12, pp. 774846-774846.1:CAS:528:DC%2BB38XktF2lur0%3D [DOI: https://dx.doi.org/10.3389/fgene.2021.774846] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/34795698][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8593043]

36. Hadley W. Reshaping Data with the reshape Package Reshaping Data with the reshape Package. J. Statistical Softw.21 (2007).

37. Kassambara A. ggpubr: ‘ggplot2’ Based Publication Ready Plots. (2023).

38. Solis Soto L. M., Haymaker C. Biospecimen Research Database - Spatial transcriptomics: NanoString GeoMx Digital Spatial Profiling Analytical Validation of Immuno-oncology Protein Targets (MD Anderson). Biorepositories and Biospecimen Research Branch, National Cancer Institute (2024).

39. Zimmerman, SM et al. Spatially resolved whole transcriptome profiling in human and mouse tissue using Digital Spatial Profiling. Genome Res.; 2022; 32, pp. 1892-1905.1:CAS:528:DC%2BB2cXit1CmsLvK [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/36100434][PubMedCentral: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9712633]

40. Soric, B. Statistical “Discoveries” and Effect-Size Estimation. J. Am. Stat. Assoc.; 1989; 84, pp. 608-610.

41. Goutte C., Gaussier E. A Probabilistic Interpretation of Precision, Recall and F-score, with Implication for Evaluation (Springer, 2005).

42. Ozirmak Lermi N. KChen-lab/ST_Comparison: Comparison of imaging based single-cell resolution spatial transcriptomics profiling platforms using formalin-fixed paraffin-embedded tumor samples (v1.0.0). Zenodo, https://doi.org/10.5281/zenodo.15880732, (2025).