Spatially resolved tissue imaging to analyze the tumor immune microenvironment: beyond cell-type densities

Abstract

Introduction

The tissue immune microenvironment is associated with key aspects of tumor biology. The interaction between the immune system and cancer cells has predictive and prognostic potential across different tumor types. Spatially resolved tissue-based technologies allowed researchers to simultaneously quantify different immune populations in tumor samples. However, bare quantification fails to harness the spatial nature of tissue-based technologies. Tumor-immune interactions are associated with specific spatial patterns that can be measured. In recent years, several computational tools have been developed to increase our understanding of these spatial patterns.

Topics covered

In this review, we cover standard techniques as well as new advances in the field of spatial analysis of the immune microenvironment. We focused on marker quantification, spatial intratumor heterogeneity analysis, cell‒cell spatial interaction studies and neighborhood analyses.

Correspondence to Dr Carlos E De Andrea; [email protected]

WHAT IS ALREADY KNOWN ON THIS TOPIC

• The are currently several technologies that can measure the tumor immune microenvironment with spatial resolution. Bare quantification of immune markers has been the standard analytical approach. However, in recent years, analytical methods that make use of the spatial nature of these technologies have grown in popularity.

WHAT THIS STUDY ADDS

• This review summarizes the current state of spatially resolved analytical pipelines. It describes the principles of basic marker quantification, heterogeneity analysis, cell-to-cell interaction analysis and neighborhood analysis. Examples of successful implementation of these analytical strategies are provided.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

• The analytical pipelines reviewed in the present work can maximize the yield of spatially resolved technologies.

Introduction

The composition of the tumor immune microenvironment (TIME) critically determines cancer biology and can affect the sensitivity of patients to various therapies.¹ Several different cell types make up the TIME, including lymphocytes, macrophages, myeloid cells, dendritic cells, and neutrophils. Complex interactions between these cells and neoplastic cells can influence diverse important processes in malignant tissue, such as malignant cell growth, angiogenesis, metastasis, immune evasion and therapeutic resistance.^2–4

Single cellular and extracellular matrix components of the TIME and their associated spatial organization have been used to identify prognostic and predictive biomarkers. The spatial organization of the TIME can be defined by the spatial relationships established between immune and cancer cells derived from their localization within tissue. Therefore, the ability to characterize the TIME in an analytically valid and reproducible way has been of great interest in the field of oncology, particularly since the widespread clinical use of immunotherapy.^{5 6} The use of comprehensive methods to investigate immune cell populations, such as single-cell RNA-seq (scRNA-seq),⁷ highly multiparametric flow cytometry and cytometry by time of flight,^{8 9} has revolutionized our understanding of the TIME. However, these technologies require tissue disaggregation, which leads to loss of architecture and spatial organization. The analysis of the spatial features of immune and cancer cells represents both an analytical challenge and a window of opportunity, as this information is present in formalin-fixed paraffin-embedded (FFPE) tissue sections that are stable and accessible and may help explain tumor immunobiology and predict treatment-specific outcomes.¹⁰ In this review, we discuss critical milestones in the development of spatial immune microenvironment analysis strategies, focusing on tissue-based multiplexed and hyperplexed technologies. The principles discussed in this review can also be applied to more comprehensive spatially resolved technologies such as spatial transcriptomics. We also discuss computational advances in the analysis of complex cell–cell interactions and the identification of cellular neighborhoods as well as the associated challenges.

Origins of image analysis and multiplexed imaging techniques

Tissue biomarkers have been employed in biomedical research and in the clinic for a long time. Biomarkers can be broadly classified as prognostic or predictive. Prognostic biomarkers provide relevant tumor behavior information about the probability of survival regardless of the treatment received. On the other hand, predictive biomarkers are able to predict outcomes in a treatment-specific fashion and are therefore very helpful for deciding patient management. Tumor morphology and histological classifications are indeed the most basic tissue biomarkers. As immunohistochemical and immunofluorescence (IF) methods have advanced, it has become possible to discern cell populations and measure their densities within tissue sections. Immunohistochemistry (IHC) has allowed the quantification of immune cell populations and immunomodulatory molecules that have prognostic and predictive power. It is, therefore, important to measure biomarkers in an analytically valid and reproducible manner.

For decades, semiquantitative scoring systems based on pathologists’ estimations using light microscopy have served as the conventional approach for IHC analysis. Moreover, these evaluator-dependent scoring methods are currently the gold standard for assessing several clinically relevant biomarkers, such as PD-L1 expression.¹¹ In addition to other preanalytical and analytical factors,¹² it is known that the “manual scoring” by pathologists is subjective, has limited reproducibility and lacks a true quantitative output.¹³ Therefore, there have been efforts to develop analytical quantification methods.^{14 15} Although several of these methods can evaluate relevant clinical biomarkers,^16–18 they have not been widely adopted in clinical practice.

No single biomarker has demonstrated absolute prognostic or predictive power, which supports the need to integrate data from multiple markers and/or nonredundant metrics. Therefore, simultaneous measurements of diverse markers in a single slide and spatial associations among cell subpopulations could improve biomarker discovery. In addition to the single marker IHC and IF, novel techniques enable researchers to investigate the spatial architecture of the TIME in a more comprehensive manner. Multiplexed and hyperplexed IHC-based,^{19 20} IF-based,^21–23 mass spectrometry-based²⁴ and spatial transcriptomics/proteomics-based^25–27 technologies increase the number of simultaneously identified proteins or mRNA transcripts, and therefore, the number of potentially quantified immune and cancer cell populations (figure 1). In addition, the incorporation of functional markers allows the interrogation of specific cell states or biological processes within the cell populations of interest, increasing the significance and dimensionality of the data.

Figure 1. Top: Overview of the design, performance and analysis of a tissue imaging experiment to analyze the tissue immune microenvironment (TIME). Bottom: Table summarizing the main technologies used to perform spatial TIME analyses. Figure partially created with BioRender.com. The costs are approximately defined as follows: low<US$500/sample; mid=US$500-US$1500/sample; and high>US$1500/sample. FFPE, formalin-fixed paraffin-embedded; FFT, fresh frozen tissue; IF, immunofluorescence; QC, quality control.

However, these technologies are associated with challenges related to immune marker selection, assay validation, protocol optimization and analytical platform selection. First, multiplexed assays require complex development, control and validation processes prior to implementation.²⁸ Preanalytical considerations, including time from tissue collection to tissue fixation (cold ischemia time), fixation and tissue processing methods, should be carefully planned. In addition, the storage conditions and age of FFPE tissue blocks can influence the antigenicity of target markers.^{29 30} Second, the scientific question to be addressed should guide the multiplexed platform and the panels of markers to be analyzed. Platforms differ in their relevant technical characteristics, such as spatial resolution, multiplexing capacity, and workaround time. Furthermore, the source of tissue samples dramatically influences the results. For example, a tissue microarray allows the analysis of different samples in a single FFPE tissue section, ameliorating technical variability issues. On the other hand, whole slide images can include larger tissue areas and enable a deeper understanding of intratumor spatial heterogeneity. Last, different analysis pipelines will determine the computing power requirements. A standard multiplex marker assessment pipeline is shown in figure 1.

Single-plex and multiplex marker quantification analysis pipeline

Tissue/cell segmentation and quantification of immune populations are generally the initial steps in most multiplex tissue-based analysis pipelines. Different analysis strategies have unique advantages and are subject to different limitations. Marker quantification relative to the measured area (marker expression per square millimeter), cell density (cell per tissue area such as square millimeter) or the fraction of cells relative to the total number of analyzed cells (proportion of cells) are commonly used metrics. The quantification of individual immune cell populations can be used to calculate their prognostic and predictive potential, to identify biological differences among tumor types or to analyze their variation longitudinally over time in the same tumor/patient after therapeutic interventions. In addition, information on the global infiltration of various immune populations is needed to design complex spatial analysis strategies.

Pixel-based marker quantification

Digital images are numeric arrays of pixels that contain information on the cells present in the tissue. The pixel information present in these arrays will vary according to variations in cell type composition. Therefore, quantifying normalized pixel intensities without considering individual cell units is a commonly employed and effective strategy for analyzing multiplex tissue images. Pixel-based analyses usually begin with image preprocessing steps that are followed by marker normalization to ensure intraslide and interslide comparability. Next, different tissue compartments or areas are defined according to the pixel intensity of specific markers. For example, cytokeratin can be used to define the pixels belonging to the cancer cell compartment, vimentin can be used to identify the surrounding stromal area, or CD3 can be used to delineate the T-cell compartment. Compartment definition usually requires setting adequate marker thresholds and may also require one or several rounds of image preprocessing or filtering to accommodate specific compartment shapes, smooth out compartment edges and remove any artifactual holes or defects in the compartment mask. Subsequently, the pixel intensity of the marker of interest is averaged across the selected tissue compartment and divided by the total compartment area. This metric captures both the number of positive cells/structures that express the marker and their intensity in positive areas. Complex colocalization hierarchies can be used to analyze specific colocalized markers. The main advantages of pixel-based quantification are its simplicity, high precision, objective nature, and reproducibility, in addition to its simple computational requirements. In addition, pixel-based scoring is not restricted to intracellular markers and is well suited for quantifying extracellular immune mediators, such as cytokines and other relevant macromolecules, and complex biological structures, such as neutrophil extracellular traps (NETs), which are webs of extracellular nuclear DNA associated with proteins extruded by dying neutrophils.³¹ However, pixel-based approaches that quantify immune markers using average fluorescence intensity scores that are not standardized, lack the interpretability of cell-based quantification and are affected by the size of the tissue compartment being analyzed. These drawbacks limit its use in subsequent spatial cell‒cell interaction analyses (see table in figure 2). By using pixel-based quantitative IF (QIF) platforms, such as automated quantitative analysis (AQUA),³² we have investigated the immune microenvironment of a variety of tumors.^{33 34} For example, pixel-based QIF was used to analyze various immune populations in glioblastomas treated with immunotherapy. Changes in the mean fluorescence intensity of several immune markers were noted.³³

Figure 2. Overview of the two main strategies (pixel-based and cell-based) used to quantify immunofluorescence markers. Top: Flow chart comparing the analytical pipelines. Bottom: Table displaying the feature comparison between methods.

Cell-based marker quantification

Cells are considered the basic biological unit of tissue and generally consist of three identifiable structural components: the cell membrane, the cytoplasm, and the nucleus. Image analysis algorithms can be applied to segment the membrane and nuclear compartments of cells, which allows single-cell high-dimensional morphology analysis. In contrast to pixel-based quantification, after image preprocessing steps, cell segmentation processes are carried out before proceeding with the analysis. Although still in use, algorithms based on watershed segmentation,^{35 36} active contour segmentation,³⁷ graph cut segmentation³⁸ or Voronoi tessellation³⁹ (among others) have recently been superseded by deep learning-based segmentation models.⁴⁰ Most of these algorithms rely on nuclear markers such as DAPI (4',6-diamidino-2-phenylindole) or Hoechst stain to perform nuclear segmentation. Each detected nucleus is then expanded to provide an approximation of the full cell area, constrained by the distance and proximity of neighboring cells. A list of several features derived from cell morphometry and intensity measurements is then calculated for each cell. Alternatively, segmentation methods that use nuclear and membrane markers to feed deep learning-based classifiers, such as the DeepCell algorithm, have been developed as powerful cell segmentation tools.⁴¹ They run pretrained networks (eg, Mesmer) to generate cell masks.⁴¹ Quality checks of the cell outlines can be visualized by overlaying the cell masks over the original images.

After cell segmentation and feature extraction, the cells are labeled according to their marker expression profile. Expression-matrix approaches rely on the annotation of the average intensity of different markers for each of the cells in the sample. Intensity thresholds can then be specified manually by researchers or by distribution histogram-based methods, such as the Otsu method or Renyi entropy.^{42 43} Other alternatives not requiring thresholding include various clustering techniques (hierarchical, density-based, and graph-based techniques, such as the Louvain or Leiden algorithms). Alternatively, user-supervised, image-based machine learning algorithms such as CellSighter⁴⁴ can be trained to classify cells in experiments. These algorithms work directly with images, considering other morphometric variables in addition to marker intensity in the cell segmentation process. After major cell lineage identification, proteins of interest not used for the phenotyping process can be quantitatively scored within the defined cell population (such as granzyme B, PD-1 or PD-L1).

The main advantage of cell-based approaches is that they provide biologically interpretable metrics. In addition, these methodologies allow subsequent complex cell‒cell interaction analysis pipelines. Cell-based pipelines are the mainstays of commonly employed multiplex IF analysis software, such as QuPath,⁴⁵ Halo (Indica Labs), Phenoplex (Visiopharm) and InForm (Akoya Biosciences). However, the evaluation of individual cell populations using tissue segmentation strategies is subject to prominent associated challenges. These include difficulties in systematically obtaining accurate single-cell identification/phenotyping across tissues and samples, the use of extremely labor-intensive morphology-assisted quality control steps to ascertain adequate performance, the presence of bias secondary to dissimilar marker expression levels/patterns and cell size or morphology, the loss of valuable marker information in the thresholding process through binarization, and the oversight of information present between individual cells in interstitial/extracellular areas that may have biological relevance. Moreover, cell segmentation strategies have associated computational costs. As a result of increased complexity and variability in results between methods, reproducibility may be an issue.

Cell-based quantification has grown in popularity in recent years. For example, a cell-based strategy has been used to quantify CD8+T cells in various tumor types and to correlate cell density with the presence of NETs.⁴⁶

Quantifying spatial intratumor heterogeneity

Tumors are subjected to evolutionary pressure that leads to intertumor and intratumor heterogeneity (ITH).⁴⁷ In recent years, ITH has received increasing attention due to its association with cancer progression and response to treatment, including its response to checkpoint inhibitors.^48–52 ITH quantification is a field of active research. From a genetic perspective, multiple methods have been developed to quantify different tumor subclones as a proxy of ITH.^53–56 In addition, efforts have been made to analyze ITH at the transcriptomic level.^{57 58} At the spatial level, tumors also display heterogeneity. However, analyzing spatial-ITH (s-ITH) is a complex task. At the spatial level, tumors can deviate from perfect homogeneity in several different ways (figure 3), implying that measuring heterogeneity requires the integration of several different analytical strategies. Furthermore, s-ITH refers not only to cancer cells but also to multiple types of immune and nonimmune cells (such as fibroblasts and endothelial cells). In addition to variations in intrinsic cell properties and cell density, s-ITH should also reflect the diversity of spatial configurations present in the sample.

Figure 3. Spatial intratumor heterogeneity analysis (s-ITH). Top: (A) Spatial heterogeneity can display different patterns. Some patterns are associated with high global heterogeneity while others are not. (B) Flow chart with the main outline of an s-ITH analytical pipeline. Bottom: Table describing different commonly used s-ITH methods. Figure partially created with BioRender.com.

Broadly, s-ITH can be analyzed globally (considering a tumor as a whole population) or locally (considering local spatial communities within the tumor). Although cells are the basic unit of ITH analysis at the genomic or transcriptomic level, no consensus exists on the spatial unit of measure used to perform local s-ITH. Most local s-ITH approaches are based on the selection of different areas by the researcher or the segmentation of the image into areas of constant size (image tiling or tessellation). Relevant metrics that capture the density or proportion of various cellular phenotypes are estimated in each of these areas. Metrics to measure s-ITH differ in their complexity and robustness. The metric choice can heavily influence the data structure, properties, and results of the analysis. ITH metrics can be further divided into statistical, image-based, and graph-based metrics. Statistical methods represent some of the simplest ITH metrics, including dispersion estimators such as variance, standard deviation (SD), range, and coefficient of variation. More intricate methods, such as entropy metrics⁵⁹ or mixed effects models,⁶⁰ can also be applied. Image-based methods analyze the spatial distribution of single markers to approach the level of spatial homogeneity or clustering. Textural image analysis⁶¹ and spatial autocorrelation metrics⁶² are examples of image-based approaches (see table in figure 3). Other approaches represent tissue images as graphs, where cells are nodes and edges are cell distances. Each cell or node is assigned to a specific cell type. Graph modularity scores measure how different cell types are spatially arranged into homogeneous or heterogeneous communities.⁶³

Excluding entropy and graph-based metrics, all other s-ITH metrics are restricted to analyzing single cell types. In addition, heterogeneity measurement is heavily influenced by the analyzed tissue area and the size and number of tiles used to implement the analysis. Furthermore, the number of cell phenotypes and their density also impact the metric results.

These types of analyses revealed the impact of s-ITH on survival in patients treated with immunotherapy. For example, Lopez de Rodas et al combined a local tessellation approach to calculate a global per-sample Rao quadratic index (an entropy metric) to classify non-small cell lung cancer (NSCLC) samples into high-heterogeneity and low-heterogeneity groups. The authors demonstrated that patients with tumors displaying high heterogeneity had worse survival after treatment with checkpoint inhibitors.⁶⁴

Simple spatial cell‒cell interaction

Compared with other multiplexed technologies, image-based multiplexed platforms allow direct interrogation of cell‒cell spatial mapping. Cell–cell spatial interactions are required for many immunologically relevant processes, such as cell migration, antigen presentation and cell cytotoxicity. Spatial Cartesian mapping can reveal biologically and clinically relevant complex immune architectures that cannot be identified by cell population quantification.

Most spatial interaction analysis pipelines start by measuring the distance between all the cells from two populations of interest, obtaining a cell-to-cell distance matrix. Basic spatial interactions include calculating the average distance, the average minimum distance, or the average maximum distance between two cell populations (figure 4). Other approaches require the user to define biologically relevant interaction distances and select a reference and target cell. Afterward, the number of target cells within the defined distance from every cell of origin is calculated. Alternatively, other approaches, such as the G-cross function,⁶⁵ infer the probability of encountering a target cell across the distance radius. Notably, most Euclidean cell‒cell distance metrics obtained from segmented tissue samples are highly dependent (and inversely correlated) on the density of the cells being analyzed. Thus, careful analysis using correlation functions or multivariate models should be incorporated to ascertain their contribution beyond cell counts. Alternatively, to account for the influence of the total cell density on spatial correlation metrics, several methods have been developed. Ripley’s K function⁶⁶ (or similar variants including L, F and G functions) can be used to calculate the colocalization of one or several cell populations within the distance radius K compared with a complete spatial randomness distribution. This random background is usually obtained from a Poisson process, by homogeneous distribution sampling or by iteratively permuting cell labels. In addition, random distributions and colocalization metrics can be calculated considering tissue hierarchies (eg, tumor and stromal compartments).⁶⁷ One of the main limitations of these approaches is that the colocalization distance threshold must be selected. This is usually an arbitrary choice, as only rarely is there enough evidence to support an explicit biologically relevant distance threshold. To address this limitation, numerous radius sizes can be interrogated using cross-K functions, as implemented in the R package SPIAT.⁶⁸ Moreover, distance analysis is computationally intensive and is usually performed for restricted cell populations. Despite being a powerful technique, analyses restricted to two populations may not be sufficient to capture the complexity of spatial interaction architectures.

Figure 4. Simple cell spatial interaction analysis and complex (neighborhood) interaction analysis. Top left: Table showing the different methods used to analyze cell-to-cell spatial interactions. Top right: Explanatory diagrams of simple cell-to-cell spatial interaction analysis. Bottom: Explanatory diagram of closest neighbors and UTAG neighborhood analysis approaches. UTAG, unsupervised discovery of tissue architecture with graphs.

For example, using radius interaction analysis, it has been shown that close contacts and interactions (defined as having a radius of less than 45 µm) between PD-1+CD8+T cells and PD-L1+cancer cells are associated with a poor prognosis in patients with endometrial carcinomas.⁶⁹

Multiple spatial interactions: cellular neighborhoods

The variety of cell types present in the TIME requires the visualization and quantification of several immune and cancer cell markers in a single tissue section. Many immune-relevant functions within the TIME are organized into highly complex tridimensional structures that are characterized by a conserved composition and spatial distribution of immune cells. These conserved, distinct structures can be described as two-dimensional (2D) representations using cellular neighborhoods. For example, the organization of several B and T-cell lymphocyte populations as well as dendritic cells into tertiary lymphoid structures is associated with favorable prognosis in several cancer types.^{70 71} Systematic spatial mapping of various cell types in a nonrandom fashion is the basis of cellular neighborhood analysis.

Broadly, cellular neighborhood analysis strategies typically incorporate machine or deep learning strategies and can be classified as unsupervised or weakly supervised. Unsupervised approaches do not take into account image metadata to identify tissue neighborhoods. Among these algorithms, one of the most popular unsupervised neighborhood profiling algorithms requires the annotation of the closest neighbors surrounding every single cell in the sample. These closest neighbor patterns are then used to feed a clustering algorithm to find reproducible spatial colocalization patterns.⁷² This approach has been used to decompose the immune microenvironment in colorectal carcinomas using the codetection by indexing (CODEX)⁷³ labeling technique as well as to characterize complex cellular neighborhoods in NSCLC.⁷⁴ Other algorithms, such as unsupervised discovery of tissue architecture with graphs (UTAG),⁷⁵ are based on adjacency and feature matrices. In this approach, a distance threshold to consider two cells as spatially related must be defined by the researcher. UTAG allows the discovery of ‘tissue microanatomical domains’ that can be used to decompose tissue into meaningful spatial interaction domains. Other approaches inspired by natural language processing, such as SpatialLDA,⁷⁶ analyze the spatial neighbors of cells to infer different spatial microenvironments (called “topics”), just as groups of words are usually associated with specific topics in a text document. This approach has been applied to identify novel spatially recurrent cell‒cell microenvironments associated with recurrence in colorectal cancer.⁷⁷

Approaches such as that implemented in CytoMAP⁷⁸ analyze tumor neighborhoods in a grid-like manner. The cell composition and marker intensity were calculated for all the tiles in the sample, and the results were clustered using self-organizing maps.

Recently, weakly supervised deep learning methods, such as NaroNet,⁷⁹ have been implemented to perform cellular neighborhood analysis. This trend is driven by image metadata (including clinical information, molecular data, and histologic data) used as training labels to obtain biologically relevant spatial patterns. These patterns are specific for the selected labels. In contrast to other methods that consider a single definition of neighborhood, NaroNet’s spatial interaction networks are arranged in hierarchies of increasing size and complexity (local phenotypes, neighborhoods, and areas). This approach allows a broader analysis of complex cell‒cell interactions. Using NaroNet, complex cell‒cell spatial interaction patterns associated with endometrial cancer relapse have been analyzed,⁸⁰ identifying that CD8+T cells, Tregs and macrophage-rich neighborhoods are associated with better clinical outcomes.

The choice of the neighborhood approach usually relies on the scientific hypothesis to be addressed. Unsupervised approaches are good options for pattern discovery studies while weakly supervised strategies can be employed to increase the clinical relevance of results. The notion that neighborhood analysis is complex and computationally intensive should also be considered. In addition, all algorithms require the adjustment of hyperparameters, which may be performed in a user predefined manner or through cross-validation methods. These parameters influence the ability of the algorithm to identify different spatial patterns present in the tissue. Although less prone to bias, hyperparameter cross-validation increases the computational costs of the pipeline. However, flawed hyperparameter adjustment may lead to reproducibility issues. A major limitation of computationally complex neighborhood analysis relies on the numerous data required to train stable and reproducible models. Neighborhood identification methods that rely on deep learning may require an exceedingly large number of image examples to be trained and validated. Although this issue may partly be alleviated with the use of synthetic images created through several different methods,^{68 81} it still represents a major limitation to its widespread implementation. In addition, there is a lack of uniformity in the computational rules and hyperparameters necessary to capture biological or clinically relevant information. These models are also generally subject to the same limitations of single-cell segmentation analysis described above. To overcome some of these limitations, a recent model called ‘Graph of Graphs’ has incorporated a manifold learning approach to achieve accurate classification of tumors from patients with lung cancer sensitive and resistant to PD-1 axis blockers. The authors used a segmentation-independent strategy in which tissue microdomains or “patches” from tumor slides stained with a 29-marker imaging mass cytometry panel were used to generate multimarker correlation vectors that represent the local tissue organization.⁸²

Future directions

Although multiplex and hyperplex spatial technologies have demonstrated good predictive and prognostic capabilities, most of these technologies are restricted to research applications. Translation of these spatial analyses to the clinic is challenging. The approval of in vitro diagnostic tests (IVDs) by regulatory agencies requires a stringent analysis of assay validity, reproducibility, sensitivity, and specificity. Despite several PDL1 immunohistochemical assays and the analysis of lymphocyte infiltration patterns (the immunoscore),⁸³ no other tissue-based immune biomarker has gained IVD approval. Further efforts are required to accelerate the implementation of spatial tissue-based analyses in routine clinical practice.⁸⁴

Assessment of the TIME to advance the field of immune oncology demands the implementation of increasingly complex techniques that are capable of simultaneously measuring not only protein markers but also RNA transcripts, DNA sequences and epigenetic landmarks.⁸⁵ Of these, spatial transcriptomics has already been applied in the field of oncology,^{86 87} and its potential in the field of immune-oncology has already been recognized.⁸⁸ Spatial transcriptomics exceeds the number of total markers that can be measured per cell, allowing the identification of complex cell phenotypes. However, spatial transcriptomics is a heterogeneous group of techniques that differ in terms of transcriptome coverage and spatial resolution. For example, Visium Spatial Gene Expression (10x Genomics) allows the interrogation of the entire genome but does not allow direct single-cell spatial resolution. On the other hand, other spatial transcriptomics technologies, such as GeoMx (NanoString), allow the interrogation of a limited set of transcripts in specific spatial tumor compartments. Furthermore, the massive amount and complex structure of the data generated by these techniques pose a great challenge. First, handling such an amount of data usually requires great computational power and carefully planned strategies to reduce data dimensionality. The data distributions yielded by these technologies are usually zero-inflated. This means that most measured transcript or protein counts in each cell are equal to zero and require adequate statistical approaches.⁸⁹ Moreover, specific challenges regarding data acquisition, normalization, image segmentation and cell phenotyping are relevant and have been reviewed.⁹⁰ Tailored solutions for analyzing gene expression spatial patterns, including SpatialDE⁹¹ and SPARK,⁹² are available. Due to the vast number of cell phenotypes identified in these experiments, neighborhood analyses are more complicated than less comprehensive technologies, and therefore, advances in the field are needed. Given the specific strengths and limitations of different spatial omics approaches, pipelines that integrate the results of several of these technologies (including multiplex and hyperplex technologies and simple stains such as hematoxylin and eosin) will be needed. There are already examples of pipelines that integrate scRNAseq, chromatin accessibility and spatial proteomic data.⁹³ These massive data sources are associated with challenges regarding data interpretation from a biological and clinical perspective.

As the amount of data obtained from spatial experiments increases, there is an increasing need for publicly available pipelines to address different analytical challenges. Open-source pipelines will enable widespread adoption, faster pipeline improvement and transparency. Some of the most popular open-source data analysis pipelines applied to spatially resolved data are summarized in figure 5.

Figure 5. Table summarizing the open-source analytical pipelines used in spatial data analysis. UTAG, unsupervised discovery of tissue architecture with graphs.

Further developments to analyze the TIME from tissue slides stained with HE are required and are under active development. HE-stained slides are the basis of pathological diagnosis and are therefore readily available and associated with clinical information. Consequently, the development of HE-based computational methods to analyze the TIME should be a priority in the field of immune-oncology. The use of deep neuronal network-based approaches (such as convolutional neural networks (CNNs)) for different tasks, such as image segmentation and classification, has already shown good results and has been reviewed extensively.⁹⁴ CNNs have also been used to investigate certain aspects of the TIME, such as TIL infiltration patterns in tumor slides stained with HE,⁹⁵ to determine associations with histological and molecular subtypes. Similar approaches have also been employed to quantify the proximity between cancer cells and TILs in breast carcinomas.⁹⁶ Other deep learning approaches may also play a role in future HE-based AI (Artificial Intelligence) applications. For example, graph-based algorithms have been successfully implemented to analyze the tumor-infiltrating lymphocyte distribution of breast cancer, melanoma and NSCLC.^97–99

Currently used techniques analyze the TIME in a 2D manner. However, the TIME is a 3D structure with dynamic temporal variations. 3D tissue imaging techniques and videorecording (4D imaging) are being developed and will likely require more sophisticated analysis tools. The use of 3D image reconstruction has already been implemented to analyze the tertiary lymphoid structures present in colorectal carcinomas.¹⁰⁰

Conclusion

Spatial patterns are intimately linked to key immune processes occurring in the tumor microenvironment. Therefore, spatial analysis is required to obtain a complete understanding of cancer-cell and immune-cell interactions. Substantial advances in the field have allowed the potential of tissue-based imaging technologies to be harnessed. Future efforts will be required to make these approaches widespread, improve the analytical pipelines and data management strategies and ultimately support biomedical research and improve patient care.

We appreciate the members of CEDA and IM laboratories for the critical appraisal of this work.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

Ethics statements

Patient consent for publication

Not applicable.

Footnote

X @itlive

Contributors ALJ, EMW, IM and CEDA wrote the manuscript. ALJ and EMW prepared the figures. ALJ, IM and CEDA conceived the original idea of the manuscript. All authors supervised the writing of the manuscript. Guarantor: ALJ.

Funding This work was supported by Instituto de Salud Carlos III and Fondo Europeo de Desarrollo Regional (ISCIII and FEDER; grants PI21/01547—MDL and CEDA), MCIN/AEI/10.13039/501100011033 and European Union-NextGenerationEU/PRTR (grant CPP21-008390—MDL and CEDA), and Fundacio La Marato de TV3 (IM, MDL and CEDA).

Competing interests No, there are no competing interests.

Provenance and peer review Not commissioned; externally peer reviewed.

References

¹ Bagaev A, Kotlov N, Nomie K, et al. Conserved pan-cancer microenvironment subtypes predict response to immunotherapy. Cancer Cell 2021; 39: 845–65. doi:10.1016/j.ccell.2021.04.014

² Leibovici J, Itzhaki O, Huszar M, et al. The tumor microenvironment: part 1. Immunotherapy 2011; 3: 1367–84. doi:10.2217/imt.11.111

³ Binnewies M, Roberts EW, Kersten K, et al. Understanding the tumor immune microenvironment (TIME) for effective therapy. Nat Med 2018; 24: 541–50. doi:10.1038/s41591-018-0014-x

⁴ Hinshaw DC, Shevde LA. The tumor microenvironment innately modulates cancer progression. Cancer Res 2019; 79: 4557–66. doi:10.1158/0008-5472.CAN-18-3962

⁵ Gibney GT, Weiner LM, Atkins MB. Predictive biomarkers for checkpoint inhibitor-based immunotherapy. Lancet Oncol 2016; 17: e542–51. doi:10.1016/S1470-2045(16)30406-5

⁶ Gjoerup O, Brown CA, Ross JS, et al. Identification and utilization of biomarkers to predict response to immune checkpoint inhibitors. AAPS J 2020; 22: 132. doi:10.1208/s12248-020-00514-4

⁷ Christodoulou M-I, Zaravinos A. Single-cell analysis in immuno-oncology. IJMS 2023; 24: 8422. doi:10.3390/ijms24098422

⁸ Iyer A, Hamers AAJ, Pillai AB. Cytof for the masses. Front Immunol 2022; 13: 815828. doi:10.3389/fimmu.2022.815828

⁹ Maby P, Corneau A, Galon J. Phenotyping of tumor infiltrating immune cells using mass-cytometry (CyTOF). Methods Enzymol 2020; 632: 339–68. doi:10.1016/bs.mie.2019.07.025

¹⁰ Fu T, Dai L-J, Wu S-Y, et al. Spatial architecture of the immune microenvironment orchestrates tumor immunity and therapeutic response. J Hematol Oncol 2021; 14: 98. doi:10.1186/s13045-021-01103-4

¹¹ Paver EC, Cooper WA, Colebatch AJ, et al. Programmed death ligand-1 (PD-L1) as a predictive marker for immunotherapy in solid tumours: a guide to immunohistochemistry implementation and interpretation. Pathology 2021; 53: 141–56. doi:10.1016/j.pathol.2020.10.007

¹² Taylor CR. An exaltation of experts: concerted efforts in the standardization of immunohistochemistry. Hum Pathol 1994; 25: 2–11. doi:10.1016/0046-8177(94)90164-3

¹³ Taylor CR, Levenson RM. Quantification of immunohistochemistry--issues concerning methods, utility and semiquantitative assessment II. Histopathology 2006; 49: 411–24. doi:10.1111/j.1365-2559.2006.02513.x

¹⁴ Fritz P, Hönes J, Lutz D, et al. Quantitative immunohistochemistry: standardization and possible application in research and surgical pathology. Acta Histochem Suppl 1989; 37: 213–9.

¹⁵ Fritz P, Wu X, Tuczek H, et al. Quantitation in immunohistochemistry. A research method or a diagnostic tool in surgical pathology? Pathologica 1995; 87: 300–9.

¹⁶ Bloom K, Harrington D. Enhanced accuracy and reliability of HER-2/neu immunohistochemical scoring using digital microscopy. Am J Clin Pathol 2004; 121: 620–30. doi:10.1309/Y73U-8X72-B68T-MGH5

¹⁷ Szeszel MK, Crisman CL, Crow L, et al. Quantifying estrogen and progesterone receptor expression in breast cancer by digital imaging. J Histochem Cytochem 2005; 53: 753–62. doi:10.1369/jhc.4A6600.2005

¹⁸ Galon J, Mlecnik B, Bindea G, et al. Towards the introduction of the “Immunoscore” in the classification of malignant tumours. J Pathol 2014; 232: 199–209. doi:10.1002/path.4287

¹⁹ van der Loos CM. Chromogens in multiple immunohistochemical staining used for visual assessment and spectral imaging: the colorful future. J Histotechnol 2010; 33: 31–40. doi:10.1179/his.2010.33.1.31

²⁰ Akturk G, Sweeney R, Remark R, et al. Multiplexed immunohistochemical consecutive staining on single slide (MICSSS): multiplexed chromogenic IHC assay for high-dimensional tissue analysis. Methods Mol Biol 2020; 2055: 497–519. doi:10.1007/978-1-4939-9773-2_23

²¹ Stack EC, Wang C, Roman KA, et al. Multiplexed immunohistochemistry, imaging, and quantitation: a review, with an assessment of tyramide signal amplification, multispectral imaging and multiplex analysis. Methods 2014; 70: 46–58. doi:10.1016/j.ymeth.2014.08.016

²² Goltsev Y, Samusik N, Kennedy-Darling J, et al. Deep profiling of mouse splenic architecture with CODEX multiplexed imaging. Cell 2018; 174: 968–81. doi:10.1016/j.cell.2018.07.010

²³ Lin J-R, Izar B, Wang S, et al. Highly multiplexed Immunofluorescence imaging of human tissues and tumors using t-cycif and conventional optical microscopes. Elife 2018; 7: e31657. doi:10.7554/eLife.31657

²⁴ Giesen C, Wang HAO, Schapiro D, et al. Highly multiplexed imaging of tumor tissues with subcellular resolution by mass cytometry. Nat Methods 2014; 11: 417–22. doi:10.1038/nmeth.2869

²⁵ Ståhl PL, Salmén F, Vickovic S, et al. Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science 2016; 353: 78–82. doi:10.1126/science.aaf2403

²⁶ Liu Y, Yang M, Deng Y, et al. High-spatial-resolution multi-omics sequencing via deterministic barcoding in tissue. Cell 2020; 183: 1665–81. doi:10.1016/j.cell.2020.10.026

²⁷ Hernandez S, Lazcano R, Serrano A, et al. Challenges and opportunities for immunoprofiling using a spatial high-plex technology: the nanostring geomx® digital spatial profiler. Front Oncol 2022; 12: 890410. doi:10.3389/fonc.2022.890410

²⁸ Taube JM, Akturk G, Angelo M, et al. The society for Immunotherapy of cancer statement on best practices for multiplex immunohistochemistry (IHC) and immunofluorescence (IF) staining and validation. J Immunother Cancer 2020; 8: e000155. doi:10.1136/jitc-2019-000155

²⁹ Engel KB, Moore HM. Effects of preanalytical variables on the detection of proteins by immunohistochemistry in formalin-fixed, paraffin-embedded tissue. Arch Pathol Lab Med 2011; 135: 537–43. doi:10.5858/2010-0702-RAIR.1

³⁰ Agrawal L, Engel KB, Greytak SR, et al. Understanding preanalytical variables and their effects on clinical biomarkers of oncology and immunotherapy. Seminars in Cancer Biology 2018; 52: 26–38. doi:10.1016/j.semcancer.2017.12.008

³¹ Masucci MT, Minopoli M, Del Vecchio S, et al. The emerging role of neutrophil extracellular traps (nets) in tumor progression and metastasis. Front Immunol 2020; 11: 1749. doi:10.3389/fimmu.2020.01749

³² McCabe A, Dolled-Filhart M, Camp RL, et al. Automated quantitative analysis (AQUA) of in situ protein expression, antibody concentration, and prognosis. J Natl Cancer Inst 2005; 97: 1808–15. doi:10.1093/jnci/dji427

³³ Schalper KA, Rodriguez-Ruiz ME, Diez-Valle R, et al. Neoadjuvant nivolumab modifies the tumor immune microenvironment in resectable glioblastoma. Nat Med 2019; 25: 470–6. doi:10.1038/s41591-018-0339-5

³⁴ Henick BS, Villarroel-Espindola F, Datar I, et al. Quantitative tissue analysis and role of myeloid cells in non-small cell lung cancer. J Immunother Cancer 2022; 10: e005025. doi:10.1136/jitc-2022-005025

³⁵ Najman L, Schmitt M. Geodesic saliency of watershed contours and hierarchical segmentation. IEEE Trans Pattern Anal Machine Intell 1996; 18: 1163–73. doi:10.1109/34.546254

³⁶ Li G, Liu T, Nie J, et al. Segmentation of touching cell nuclei using gradient flow tracking. J Microsc 2008; 231: 47–58. doi:10.1111/j.1365-2818.2008.02016.x

³⁷ Caselles V, Kimmel R, Sapiro G. Geodesic active contours. Int J Comput Vis 1997; 22: 61–79. doi:10.1023/A:1007979827043

³⁸ Lesko M, Kato Z, Nagy A. Live cell Segmentation in fluorescence microscopy via graph cut. 2010 20th international conference on pattern recognition. Istanbul, Turkey: IEEE; 2010. Available: http://ieeexplore.ieee.org/document/5597303/

³⁹ Jones TR, Carpenter A, Golland P. Voronoi-based Segmentation of cells on image Manifolds. In: Liu Y, Jiang T, Zhang C, eds. Computer Vision for Biomedical Image Applications. Berlin, Heidelberg: Springer, 2005: 535. Available: http://link.springer.com/10.1007/11569541_54

⁴⁰ Maška M, Ulman V, Delgado-Rodriguez P, et al. The cell tracking challenge: 10 years of objective benchmarking. Nat Methods 2023; 20: 1010–20. doi:10.1038/s41592-023-01879-y

⁴¹ Greenwald NF, Miller G, Moen E, et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat Biotechnol 2022; 40: 555–65. doi:10.1038/s41587-021-01094-0

⁴² Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst, Man, Cybern 1979; 9: 62–6. doi:10.1109/TSMC.1979.4310076

⁴³ Sahoo P, Wilkins C, Yeager J. Threshold selection using renyi’s entropy. Pattern Recognition 1997; 30: 71–84. doi:10.1016/S0031-3203(96)00065-9

⁴⁴ Amitay Y, Bussi Y, Feinstein B, et al. Cellsighter: a neural network to classify cells in highly multiplexed images. Nat Commun 2023; 14: 4302. doi:10.1038/s41467-023-40066-7

⁴⁵ Bankhead P, Loughrey MB, Fernández JA, et al. Qupath: open source software for digital pathology image analysis. Sci Rep 2017; 7: 16878. doi:10.1038/s41598-017-17204-5

⁴⁶ de Andrea CE, Ochoa MC, Villalba-Esparza M, et al. Heterogenous presence of neutrophil extracellular traps in human solid tumours is partially dependent on IL ‐8. J Pathol 2021; 255: 190–201. doi:10.1002/path.5753

⁴⁷ Merlo LMF, Pepper JW, Reid BJ, et al. Cancer as an evolutionary and ecological process. Nat Rev Cancer 2006; 6: 924–35. doi:10.1038/nrc2013

⁴⁸ Burrell RA, McGranahan N, Bartek J, et al. The causes and consequences of genetic heterogeneity in cancer evolution. Nature 2013; 501: 338–45. doi:10.1038/nature12625

⁴⁹ Van den Eynde M, Mlecnik B, Bindea G, et al. The link between the multiverse of immune microenvironments in metastases and the survival of colorectal cancer patients. Cancer Cell 2018; 34: 1012–26. doi:10.1016/j.ccell.2018.11.003

⁵⁰ Wolf Y, Bartok O, Patkar S, et al. UVB-induced tumor heterogeneity diminishes immune response in melanoma. Cell 2019; 179: 219–35. doi:10.1016/j.cell.2019.08.032

⁵¹ Vitale I, Shema E, Loi S, et al. Intratumoral heterogeneity in cancer progression and response to immunotherapy. Nat Med 2021; 27: 212–24. doi:10.1038/s41591-021-01233-9

⁵² Wolf Y, Samuels Y. Intratumor heterogeneity and antitumor immunity shape one another bidirectionally. Clin Cancer Res 2022; 28: 2994–3001.

⁵³ Navin N, Kendall J, Troge J, et al. Tumour evolution inferred by single-cell sequencing. Nature 2011; 472: 90–4. doi:10.1038/nature09807

⁵⁴ Oesper L, Mahmoody A, Raphael BJ. Theta: inferring intra-tumor heterogeneity from high-throughput DNA sequencing data. Genome Biol 2013; 14. doi:10.1186/gb-2013-14-7-r80

⁵⁵ Roth A, Khattra J, Yap D, et al. Pyclone: statistical inference of clonal population structure in cancer. Nat Methods 2014; 11: 396–8. doi:10.1038/nmeth.2883

⁵⁶ Jiang Y, Qiu Y, Minn AJ, et al. Assessing intratumor heterogeneity and tracking longitudinal and spatial clonal evolutionary history by next-generation sequencing. Proc Natl Acad Sci U S A 2016; 113: E5528–37. doi:10.1073/pnas.1522203113

⁵⁷ Park Y, Lim S, Nam J-W, et al. Measuring intratumor heterogeneity by network entropy using RNA-seq data. Sci Rep 2016; 6: 37767. doi:10.1038/srep37767

⁵⁸ Gavish A, Tyler M, Greenwald AC, et al. Hallmarks of transcriptional Intratumour heterogeneity across a thousand tumours. Nature 2023; 618: 598–606. doi:10.1038/s41586-023-06130-4

⁵⁹ Nunes A, Trappenberg T, Alda M. The definition and measurement of heterogeneity. Transl Psychiatry 2020; 10: 299. doi:10.1038/s41398-020-00986-0

⁶⁰ Everett R, Flores KB, Henscheid N, et al. A tutorial review of mathematical techniques for quantifying tumor heterogeneity. Math Biosci Eng 2020; 17: 3660–709. doi:10.3934/mbe.2020207

⁶¹ Haralick RM, Shanmugam K, Dinstein I. Textural features for image classification. IEEE Trans Syst, Man, Cybern 1973; SMC-3: 610–21. doi:10.1109/TSMC.1973.4309314

⁶² Moran PAP. Notes on continuous stochastic phenomena. Biometrika 1950; 37: 17–23. doi:10.1093/biomet/37.1-2.17

⁶³ Alcalá-Corona SA, Sandoval-Motta S, Espinal-Enríquez J, et al. Modularity in biological networks. Front Genet 2021; 12: 701331. doi:10.3389/fgene.2021.701331

⁶⁴ Lopez de Rodas M, Nagineni V, Ravi A, et al. Role of tumor infiltrating lymphocytes and spatial immune heterogeneity in sensitivity to PD-1 axis blockers in non-small cell lung cancer. J Immunother Cancer 2022; 10: e004440. doi:10.1136/jitc-2021-004440

⁶⁵ Barua S, Fang P, Sharma A, et al. Spatial interaction of tumor cells and regulatory T cells correlates with survival in non-small cell lung cancer. Lung Cancer 2018; 117: 73–9. doi:10.1016/j.lungcan.2018.01.022

⁶⁶ Ripley BD. The second-order analysis of stationary point processes. J Appl Probab 1976; 13: 255–66. doi:10.2307/3212829

⁶⁷ Keren L, Bosse M, Marquez D, et al. A structured tumor-immune microenvironment in triple negative breast cancer revealed by multiplexed ion beam imaging. Cell 2018; 174: 1373–87. doi:10.1016/j.cell.2018.08.039

⁶⁸ Feng Y, Yang T, Zhu J, et al. Spatial analysis with SPIAT and spaSim to characterize and simulate tissue microenvironments. Nat Commun 2023; 14: 2697. doi:10.1038/s41467-023-37822-0

⁶⁹ López-Janeiro Á, Villalba-Esparza M, Brizzi ME, et al. The association between the tumor immune microenvironments and clinical outcome in low-grade, early-stage endometrial cancer patients. J Pathol 2022; 258: 426–36. doi:10.1002/path.6012

⁷⁰ Cabrita R, Lauss M, Sanna A, et al. Tertiary lymphoid structures improve immunotherapy and survival in melanoma. Nature 2020; 577: 561–5. doi:10.1038/s41586-019-1914-8

⁷¹ Helmink BA, Reddy SM, Gao J, et al. B cells and tertiary lymphoid structures promote immunotherapy response. Nature 2020; 577: 549–55. doi:10.1038/s41586-019-1922-8

⁷² Schürch CM, Bhate SS, Barlow GL, et al. Coordinated cellular neighborhoods orchestrate antitumoral immunity at the colorectal cancer invasive front. Cell 2020; 183: 838. doi:10.1016/j.cell.2020.10.021

⁷³ Agasti SS, Wang Y, Schueder F, et al. DNA-barcoded labeling probes for highly multiplexed exchange-PAINT imaging. Chem Sci 2017; 8: 3080–91. doi:10.1039/c6sc05420j

⁷⁴ Sorin M, Rezanejad M, Karimi E, et al. Single-cell spatial landscapes of the lung tumour immune microenvironment. Nature 2023; 614: 548–54. doi:10.1038/s41586-022-05672-3

⁷⁵ Kim J, Rustam S, Mosquera JM, et al. Unsupervised discovery of tissue architecture in multiplexed imaging. Nat Methods 2022; 19: 1653–61. doi:10.1038/s41592-022-01657-2

⁷⁶ Chen Z, Soifer I, Hilton H, et al. Modeling multiplexed images with spatial-LDA reveals novel tissue microenvironments. J Comput Biol 2020; 27: 1204–18. doi:10.1089/cmb.2019.0340

⁷⁷ Lin J-R, Chen Y-A, Campton D, et al. High-plex immunofluorescence imaging and traditional histology of the same tissue section for discovering image-based biomarkers. Nat Cancer 2023; 4: 1036–52. doi:10.1038/s43018-023-00576-1

⁷⁸ Stoltzfus CR, Filipek J, Gern BH, et al. Cytomap: a spatial analysis toolbox reveals features of myeloid cell organization in lymphoid tissues. Cell Rep 2020; 31: 107523. doi:10.1016/j.celrep.2020.107523

⁷⁹ Jiménez-Sánchez D, Ariz M, Chang H, et al. Naronet: discovery of tumor microenvironment elements from highly multiplexed images. Med Image Anal 2022; 78: 102384. doi:10.1016/j.media.2022.102384

⁸⁰ Jiménez-Sánchez D, López-Janeiro Á, Villalba-Esparza M, et al. Weakly supervised deep learning to predict recurrence in low-grade endometrial cancer from multiplexed immunofluorescence images. NPJ Digit Med 2023; 6: 48. doi:10.1038/s41746-023-00795-x

⁸¹ Jimenez-Sanchez D, Ariz M, De Andrea CE, et al. Synplex: in silico modeling of the tumor microenvironment from multiplex images. IEEE Trans Med Imaging 2023; 42: 3048–58. doi:10.1109/TMI.2023.3273950

⁸² Lin Y-WE, Shnitzer T, Talmon R, et al. Graph of graphs analysis for multiplexed data with application to imaging mass cytometry. PLoS Comput Biol 2021; 17: e1008741. doi:10.1371/journal.pcbi.1008741

⁸³ Galon J, Lanzi A. Immunoscore and its introduction in clinical practice. Q J Nucl Med Mol Imaging 2020; 64: 152–61. doi:10.23736/S1824-4785.20.03249-5

⁸⁴ Locke D, Hoyt CC. Companion diagnostic requirements for spatial biology using multiplex immunofluorescence and multispectral imaging. Front Mol Biosci 2023; 10: 1051491. doi:10.3389/fmolb.2023.1051491

⁸⁵ Bressan D, Battistoni G, Hannon GJ. The dawn of spatial omics. Science 2023; 381: eabq4964. doi:10.1126/science.abq4964

⁸⁶ Berglund E, Maaskola J, Schultz N, et al. Spatial maps of prostate cancer transcriptomes reveal an unexplored landscape of heterogeneity. Nat Commun 2018; 9: 2419. doi:10.1038/s41467-018-04724-5

⁸⁷ Wu Y, Yang S, Ma J, et al. Spatiotemporal immune landscape of colorectal cancer liver metastasis at single-cell level. Cancer Discov 2022; 12: 134–53. doi:10.1158/2159-8290.CD-21-0316

⁸⁸ Elhanani O, Ben-Uri R, Keren L. Spatial profiling technologies illuminate the tumor microenvironment. Cancer Cell 2023; 41: 404–20. doi:10.1016/j.ccell.2023.01.010

⁸⁹ Agarwal DK, Gelfand AE, Citron-Pousty S. Zero-inflated models with application to spatial count data. Environ Ecol Stat 2002; 9: 341–55. doi:10.1023/A:1020910605990

⁹⁰ Park J, Kim J, Lewy T, et al. Spatial omics technologies at multimodal and single cell/subcellular level. Genome Biol 2022; 23: 256. doi:10.1186/s13059-022-02824-6

⁹¹ Svensson V, Teichmann SA, Stegle O. Spatialde: identification of spatially variable genes. Nat Methods 2018; 15: 343–6. doi:10.1038/nmeth.4636

⁹² Sun S, Zhu J, Zhou X. Statistical analysis of spatial expression patterns for spatially resolved transcriptomic studies. Nat Methods 2020; 17: 193–200. doi:10.1038/s41592-019-0701-7

⁹³ Chen S, Zhu B, Huang S, et al. Integration of spatial and single-cell data across modalities with weakly linked features. Nat Biotechnol 2023. doi:10.1038/s41587-023-01935-0

⁹⁴ Bera K, Schalper KA, Rimm DL, et al. Artificial intelligence in digital pathology - new tools for diagnosis and precision oncology. Nat Rev Clin Oncol 2019; 16: 703–15. doi:10.1038/s41571-019-0252-y

⁹⁵ Saltz J, Gupta R, Hou L, et al. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep 2018; 23: 181–93. doi:10.1016/j.celrep.2018.03.086

⁹⁶ Yuan Y. Modelling the spatial heterogeneity and molecular correlates of lymphocytic infiltration in triple-negative breast cancer. J R Soc Interface 2015; 12: 20141153. doi:10.1098/rsif.2014.1153

⁹⁷ Basavanhally AN, Ganesan S, Agner S, et al. Computerized image-based detection and grading of lymphocytic infiltration in her2+ breast cancer histopathology. IEEE Trans Biomed Eng 2010; 57: 642–53. doi:10.1109/TBME.2009.2035305

⁹⁸ Failmezger H, Muralidhar S, Rullan A, et al. Topological tumor graphs: a graph-based spatial model to infer stromal recruitment for immunosuppression in melanoma histology. Cancer Res 2020; 80: 1199–209. doi:10.1158/0008-5472.CAN-19-2268

⁹⁹ Wang X, Barrera C, Bera K, et al. Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (Tils) predict clinical benefit for immune checkpoint inhibitors. Sci Adv 2022; 8: eabn3966. doi:10.1126/sciadv.abn3966

¹⁰⁰ Lin J-R, Wang S, Coy S, et al. Multiplexed 3d atlas of state transitions and immune interaction in colorectal cancer. Cell 2023; 186: 363–81. doi:10.1016/j.cell.2022.12.028