Brief Communication

https://doi.org/10.1038/s41587-022-01467-z

Modeling intercellular communication in tissues

using spatial graphs of cells
David S. Fischer , Anna C. Schaar1,3,4 and Fabian J. Theis
1,2,4 1,2,3 ✉

Models of intercellular communication in tissues are based on 69 µm across the six datasets (Fig. 1c), showing that cell–cell depen-
molecular profiles of dissociated cells, are limited to recep- dencies appear on length scales characteristic of molecular mecha-
tor–ligand signaling and ignore spatial proximity in situ. We nisms of cell communication. NCEMs outperformed nonspatial
present node-centric expression modeling, a method based baseline models consistently by an average ΔR2 (Online Methods)
on graph neural networks that estimates the effects of niche of 0.016 (Fig. 1c). As expected, the ΔR2 is small compared to the
composition on gene expression in an unbiased manner from baseline model R2 that characterizes between-cell-type variance
spatial molecular profiling data. We recover signatures of (0.39–0.79) because cell type identity accounts for a large fraction
molecular processes known to underlie cell communication. of variance in single-cell gene expression assays17. The inferred
Cells interact on multiple length-scales through direct contact of length scales were robust to data downsampling (Extended Data
surface-bound receptors and ligands, tight junctions and mechani- Fig. 3a), out-of-domain data from an unseen genetic knockout con-
cal effects, and through indirect mechanisms, including soluble dition (Extended Data Fig. 3b–e), to simulated segmentation errors
factors. Usually, these communication events cannot be directly (Extended Data Fig. 3f,g) and to removal of the interaction terms
observed but are critical to understand emergent phenomena in tis- from the linear model (Supplementary Fig. 2). The spatial effect on
sue niches1. Molecular signatures of sender and receiver cell types model performance varies between target cell types, suggesting that
are used to infer latent cell communication events in a tissue through cell-type-specific molecular mediators of cell–cell dependency are
co-occurrence of ligand and receptor expression across putatively captured (Supplementary Fig. 3).
communicating cell types2,3 and through gene expression signatures NCEMs can be extended to spot transcriptomics if
in the receiving cell2,4. Here we propose node-centric expression within-cell-type variation can be recovered from spot transcrip-
models (NCEM) to improve cell communication inference through tomics datasets in deconvolution analyses18,19. Here NCEMs model
the use of spatial graphs of cells to constrain axes of cellular com- the expression variation within cell types across spots as a function
munication. We infer cell communication from image-structured of the inferred spot composition (Fig. 1d and Methods). We con-
molecular profiling assays of RNA or proteins with subcellular reso- sidered data from lymph nodes18,19 (Extended Data Fig. 4a,b) for
lution (Fig. 1a). We defined an NCEM as a graph neural network which a deconvolution was previously demonstrated with cell2loca-
that predicts a cell’s observed gene expression vector from its cell tion19. Linear NCEMs were substantially better at predicting gene
type label and its niche5 (Fig. 1b and Methods). Cell–cell dependen- expression states of cell types in particular spots than a nonspatial
cies may be caused by diverse molecular mechanisms not limited baseline model, both globally (Fig. 1e) and for each cell type (Fig. 1f).
to ligand–receptor-based communication. Therefore, we consider The inferred couplings were stable to moderate subsampling of the
the effects of niche composition on all genes in an unbiased man- transcriptomics spots in the training data (Extended Data Fig. 4d).
ner. Previous mathematical models of cell–cell interactions in spa- We found putatively interacting ligand–receptor pairs for almost all
tial data differed in at least one out of the following central design type couplings in CellPhoneDB3 and NicheNet2 analyses of matched
choices that constitute an NCEM (Supplementary Table 1): they did single-cell RNA sequencing (scRNA-seq) data, thus demonstrat-
not represent statistical dependencies of gene expression6,7, did not ing the need for a quantitative description of statistical couplings
model cell communication events6–8, did not work on targeted pro- in niches (Extended Data Fig. 4e–g). We also identified spatial
tocols with limited ligand and receptor gene capture8–10 or relied on dependencies between entire niches when modeling spot graphs
leave-one-gene hold-outs4,9, which can result in false discoveries of (Supplementary Fig. 4).
dependencies (Extended Data Fig. 1 and Methods). Next we interpreted the spatial dependencies in the MERFISH
We demonstrate cell communication inference with NCEMs on brain data. We found that L2/3 intratelencephalic (IT) cells depend
six datasets measured with MERFISH11,12, CODEX13, MIBI-TOF14, on the presence of Sncg expressing cells, vascular leptomenin-
MELC15 and chip cytometry16 (Extended Data Fig. 2 and Methods). geal cells, and L4/5 cells in their niche (Extended Data Fig. 5 and
On average, intracell-type variance accounted for 40.6% of the total Supplementary Fig. 5). These associations are reproduced by
variance (Supplementary Fig. 1 and Methods). We defined the CellPhoneDB (Supplementary Fig. 6a,b). The L2/3 IT cell subclus-
screened neighborhood sizes such that they cover the range of aver- ters are spatially localized in distinct areas of the primary motor
age node degrees of the given dataset (Extended Data Fig. 2b). We cortex12. Indeed, the relative performance of NCEM is spatially
fit a linear model of gene expression based on a niche represented structured (Extended Data Fig. 5f and Supplementary Fig. 5e). We
as interaction terms between the receiver cell type and the presence quantified these dependencies between cell types as ‘cell type cou-
of each (sender) cell type in the neighborhood (Methods). Linear plings’, the number of significant gene-wise coefficients of the cell
NCEMs were most predictive on an intermediate length scale of type pair in an NCEM fit (Extended Data Fig. 5g and Methods).

Institute of Computational Biology, Helmholtz Zentrum München, Neuherberg, Germany. 2TUM School of Life Sciences Weihenstephan, Technical
1

University of Munich, Freising, Germany. 3Department of Mathematics, Technical University of Munich, Garching bei München, Germany. 4These authors
contributed equally: David S. Fischer, Anna C. Schaar. ✉e-mail: [email protected]

332 Nature Biotechnology | VOL 41 | March 2023 | 332–336 | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication

a Gene Resolution Niche

expression of cell 2 Cell type
Immune cell 2 4 Immune cell

Segmentation Repressed cancer cell Spatial Cancer cell

3
Malignant cancer cell graph 1 Epithelial cell
Epithelial cell of cell types
Interaction

b
4 Self Niche Repression
2
Gene
f( 1 , ( , , )) =
expression 3
1

cell type
Sender
f( 2 , ( , , )) =
f
NCEM f
Linear NCEM Effect
f f( 3 , ( , , )) = summary by
2 4 cell-type pair
Cell types f( 4 , ( , , )) = Receiver
and spatial cell type
context 3
1

c
Fetal liver wild Colon
Brain (MERFISH) Cancer (MIBI-TOF) Tonsils (MELC) Cancer (CODEX)
type (MERFISH) (chip cytometry)
0.460 0.630 Example
0.760 cell radius
Variance explained (R 2)

0.565 0.525
0.805
0.440 0.628 Model:
0.520
0.560 0.750 Baseline
0.420
0.515 0.800 NCEM
0.626
0.555 Split:
0.510 0.740 0.400
0.795 0.624 1
0.550 0.505
0.380 2
0.730 3

10
50
200
1,000
0

10

200

2,500

10
50
200
1,000
0

10

200

6,000

10
50
200
1,000
0
5
10
50
200

* * * * * *
Resolution (μm)

d e f
es
yp

Baseline
Spots Cell types genes 0.800
lt

Variance explained (R 2 )
el
C

NCEM
Spot i
0.700
i
Spots

Spots

0.600

A Abundance Expression 0.500

Baseline

NCEM

B cells
T Treg
NKT
Macrophages
DC
CD8 T cells
CD4 T cells
FDC
T TfR
T TIM
ILC
Monocytes
NK
VSMC
Mast
Endo

Fig. 1 | Node-centric expression models capture statistical dependencies between cells in space. a, Spatial graphs of cells are based on segmentation
masks of cells in spatial molecular profiling data. Resolution is the radius of neighborhood used to define a niche. Numbers label cells and are used
in Fig. 1b. b, NCEMs describe the gene expression observation of a cell as a function (f) of its spatial neighborhood (niche). c, Linear models capture
neighborhood dependencies in spatially resolved single-cell data. Shown are the R2 values between predicted and observed expression vectors on held-out
test cells by resolution for six datasets. Green line, 10 µm; baseline (blue points with cross-validation split indicated as shape), a nonspatial linear model;
bracket (*), significant difference in paired t-test. d, Variation in deconvoluted expression vectors over spots for a given cell type can be attributed to spot
composition with a linear NCEM. A, spot adjacency matrix. e,f, NCEM performance on deconvoluted data. Shown are the R2 values between predicted
and observed gene expression vectors for held-out test spots of a linear NCEM in comparison to a baseline model that does not use the spot composition
information. The performance is shown across the entire test set (e) and split by cell type (f) (n = 3 cross-validation splits). For each box in (e,f), the
centerline defines the median, the height of the box is given by the interquartile range (IQR), the whiskers are given by 1.5 × IQR and outliers are given as
points beyond the minimum or maximum whisker.

Nature Biotechnology | VOL 41 | March 2023 | 332–336 | www.nature.com/naturebiotechnology 333

Brief Communication NaturE BiotEchnology

a b

Endo
0 0 0

Subcluster

FD
1

DC
2
1 1 lls

C
ce
UMAP2

2 2
3 T
4 3 3 8
CD

IL
C
UMAP1 Ma
cro s
B cells pha cell
ges 4T
CD4 T cells 6 CD
CD8 T cells 8
DC 8

Abundance in neighborhood
5
Endo
FDC Mast B cells
ILC 6 6 4
Macrophages
Mast
3
Monocytes 4 4 VS
tes MC
NK ocy
2 Mon
NKT
T TIM 2 2
T TfR
NK

T
1

Tr
T Treg

eg
VSMC

TT
T
NK
0 1 2 3 4 0 1 2 3 0 1 2 3

fR
T TIM
B cell subclusters FDC subclusters Mast subclusters

c d e f Correlation
Fold change Fold change
–0.10 –0.15 0 0.05 0.10 –0.4 –0.2 0 0.2 0.4 0 0.5 1.0

14
LTF LTF
CXCL13
FTL FTL
12 B cells
–log10 FDR-corrected P values

PFN1 PFN1 FDCSP

CR2 CD4 T cells
CR2
RPS25 CD8 T cells
RPS25
RPL13A RPL13A 10 T Treg
T TfR
CXCL14 CXCL14
NK
IGFBP3 IGFBP3 8 ILC

Sender
CXCL13 CXCL13 IGHG3
NKT
EEF1A1 EEF1A1 Monocytes
IGFBP7 IGFBP7 6 FDC
RPSA RPSA DC
CD52 CD52 Macrophages
RPS27 RPS27 4 Endo
FDCSP FDCSP IGHA1 VSMC
IGLC3 IGLC3 Mast
IGHA1 IGHA1 2
T TIM
IGHG1 IGHG1
IGHG3

Macrophages
CD4 T cells
CD8 T cells

Monocytes
IGHG3 0
B cells

T Treg

VSMC

T TIM
T TfR

Endo

Mast
NKT

FDC
ILC

DC
NK
T TIM
T TfR
T Treg
B cells

ILC

VSMC
CD4 T cells
CD8 T cells
DC
Endo

Macrophages
Mast
Monocytes
NK
NKT

–0.10–0.05 0 0.05 0.10

CD4 T cells
CD8 T cells

Endo
FDC

Mast
Monocytes

NKT
T TIM
T TfR
T Treg
VSMC
Macrophages
ILC
DC

NK

log fold change

Sender

Sender cell type Receiver cell type

g h Linear NCEM NL-NCEM IND NL-NCEM LR

Receptor space Ligand space
0.900
Variance explained (R 2)

i 0.800
i
f( , )= Ŷ
0.700
0.715
0.950
0.800
0.710
0.900
Graph 0.705 0.795
attention 0.850
Receptor Ligand network Decoder 0.700 0.790
genes Cells genes 0.800
All genes
G = (A, YR , YL) Ŷ
0
5
10
50
200

0
5
10
50
200

0
5
10
50
200

Cell i Cell i
i i
* * *
Cells

YR YL
Cells

A Y Resolution (μm)

Example Model: Split:

cell radius Baseline 1
Cell i NCEM 2
3

334 Nature Biotechnology | VOL 41 | March 2023 | 332–336 | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication
Fig. 2 | Node-centric expression models identify interacting cell types. a, Top: UMAPs with subclusterings of spot-wise gene expression of B cells, FDCs
and mast cells from the Visium lymph node dataset. Bottom: the average abundance of cell types per neighborhood after deconvolution of the subclusters.
b, Type coupling analysis with edge width proportional to the L1 norm of the vector of fold changes of differentially expressed genes for each pair of sender
and receiver cell types. Only edges with at least 200 genes are shown. c,d, Sender effect (c) and receiver effect (d) analysis of the FDC–B cell signaling
axis. Shown is the estimated fold change that the sender cell type on the x axis induces in the gene on the y axis in receiving B cell (c) and conversely
for receiver cell types given sender FDCs (d). e, Volcano plot of differentially expressed genes of B cells in the neighborhood of FDC. f, Sender similarity
analysis based on a correlation of the coefficient vectors of each sender type with respect to B cell receivers. g, A graph kernel to specifically model
receptor activity of a cell given the observed ligand expression in its niche extends NCEM to ligand–receptor modeling. h, R2 values between predicted and
observed expression vectors for held-out test cells of linear (NCEM), nonlinear (NL-NCEM IND) and ligand–receptor-kernel NCEMs (NL-NCEM LR) by
resolution on imputed MERFISH fetal liver data. Green line, 10 µm; baseline (blue points with cross-validation split indicated as shape), a nonspatial linear
model; bracket (*), significant difference in paired t-test.

We discovered a dependency of CD8 T cells on multiple other cell compared to categorical cell type labels. We observed differential
types in the chip cytometry colon data (Extended Data Fig. 6) and receptor signaling as differential latent neuron activation of Kit11 in
a dependency of CD8 T cells on proximity to the tumor–immune sinusoidal endothelial cells (SECs) depending on their proximity to
boundary14 in colorectal cancer (Extended Data Fig. 7). arterial endothelial cells (AECs) (Supplementary Data 2).
Similarly, we interpreted NCEM fits on the deconvoluted Visium NCEMs are linear and nonlinear graph neural networks and
lymph node data. We identified a bidirectional dependency of B CVAEs that model cell communication events in spatial omics
cells and follicular dendritic cells (FDCs) that is indicative of posi- assays (Supplementary Table 2). We identified statistical depen-
tive feedback between both cell types in germinal center organiza- dencies between cells on physiologically relevant length scales and
tion20 (Fig. 2a,b and Extended Data Fig. 4c). Similarly, we found interpreted fits based on model parameters. The statistical identifi-
evidence for a dependency of mast cells on B cells (Fig. 2b) and ability of cell type couplings may improve with increased capture
a mast cell subcluster associated with niches enriched in B cells of niche heterogeneity, through the inclusion of three-dimensional
(Fig. 2a). The FDC subcluster associated with niches enriched in data, by increasing the number of cells measured and by increas-
B cells (cluster 3) showed increased expression of Cxcl13, a key ing the variation in the training data through perturbations of
chemo­kine for germinal centers20 (Extended Data Fig. 4c), support- niche structure. Uncertainty in segmentation of cells or nuclei can
ing the association of these couplings with germinal centers. We be improved on the level of the spatial measurement22 or may be
further dissected these couplings based on the gene-wise effects of addressed in model extensions. We found that linear NCEMs per-
all senders on one receiver type (‘receiver effect analysis’, Fig. 2c) form well in the presented tasks and are promising candidates for
and of one sender on all receivers (‘sender effect analysis’, Fig. 2d), cell communication inference. The complexity of the graph neural
which contextualizes differential expression results of the FDC–B network used in the NCEM defines the complexity of the motifs of
cell axis (Fig. 2e and Supplementary Data 1). Multiple T cell clusters cell communication that can be discovered and may be expanded
had a similar effect on B cells in a ‘sender similarity analysis’ given more complex datasets. CVAE–NCEMs may be used to model
(Fig. 2f), in which we correlated the coefficient vectors of sender cell-intrinsic variation together with niche effects. Similarly, the
cell types that correspond to B cell receivers, which demonstrates graph kernels tailored to ligand–receptor signaling presented here
conservation of cell type identity in the sender profile. provide constrained latent variables that explain extrinsic variation
In contrast to linear NCEMs, nonlinear NCEMs can account for and could be used together with variables for cell-intrinsic varia-
weighted or higher-order interactions between cell types (Extended tion. Nonlinear NCEMs learn a cellular representation within the
Data Fig. 8a). As for linear NCEMs, we found resolution-dependent spatial graph23, and we demonstrated that these representations
prediction performance profiles in nonlinear NCEMs (Extended can model niches and may be exploited for unsupervised analysis
Data Fig. 8b and Methods) and a dependency between L4/5 IT and of tissue structures.
L2/3 IT cells (Extended Data Fig. 8c and Methods). Notably, the
nonlinear models did not outperform linear models in gene expres- Online content
sion prediction, which suggests that the spatial dependencies in the Any methods, additional references, Nature Research report-
given datasets are well described by linear models (Extended Data ing summaries, source data, extended data, supplementary infor-
Fig. 8b). Next, we considered a conditional variational autoencoder mation, acknowledgements, peer review information; details of
(CVAE) version of an NCEM to model cell-intrinsic latent states. author contributions and competing interests; and statements of
We conditioned the distribution over node expression states on a data and code availability are available at https://doi.org/10.1038/
graph embedding of the niche and the cell type (Extended Data s41587-022-01467-z.
Fig. 9a). Even though CVAE–NCEM attained much higher predictive
performance in reconstruction tasks (Extended Data Fig. 9b and Received: 1 July 2021; Accepted: 11 August 2022;
Supplementary Fig. 7a), these models did not consistently capture Published online: 27 October 2022
spatial dependencies because niche states were represented in latent
variables (Extended Data Fig. 9c–f and Supplementary Fig. 7b–e). References
The interpretation of spatial dependencies inferred on targeted 1. Palla, G., Fischer, D. S., Regev, A. & Theis, F. J. Spatial components of
spatial molecular profiling assays is constrained by the limited molecular tissue biology. Nat. Biotechnol. 40, 308–318 (2022).
2. Browaeys, R., Saelens, W. & Saeys, Y. NicheNet: modeling intercellular
capture of ligand–receptor pairs. We imputed the cell-wise gene communication by linking ligands to target genes. Nat. Methods 17,
expression in the MERFISH fetal liver data using corresponding 159–162 (2020).
scRNA-seq data21 (Extended Data Fig. 10). We designed a graph 3. Efremova, M., Vento-Tormo, M., Teichmann, S. A. & Vento-Tormo, R.
kernel of cell-wise receptor activity to directly model ligand–recep- CellPhoneDB: inferring cell–cell communication from combined
tor interactions between neighboring cells (Fig. 2g). The peak expression of multi-subunit ligand–receptor complexes. Nat. Protoc. 15,
1484–1506 (2020).
predictive performance of the ligand–receptor nonlinear NCEM 4. Arnol, D., Schapiro, D., Bodenmiller, B., Saez-Rodriguez, J. & Stegle, O.
was much higher compared to the nonlinear NCEM (R2 of 0.799 Modeling cell-cell interactions from spatial molecular data with spatial
and 0.947), demonstrating the increased complexity of the input variance component analysis. Cell Rep. 29, 202–211 (2019).

Nature Biotechnology | VOL 41 | March 2023 | 332–336 | www.nature.com/naturebiotechnology 335

Brief Communication NaturE BiotEchnology
5. Kipf, T. N. & Welling, M. Semi-supervised classification with graph 17. Wolf, F. A., Angerer, P. & Theis, F. J. SCANPY: large-scale single-cell gene
convolutional networks. Proceedings of the 5th International Conference on expression data analysis. Genome Biol. 19, 15 (2018).
Learning Representations (2017). 18. Lopez, R. et al. DestVI identifies continuums of cell types in spatial
6. Dries, R. et al. Giotto: a toolbox for integrative analysis and visualization of transcriptomics data. Nat Biotechnol. 40, 1360–1369 (2022).
spatial expression data. Genome Biol. 22, 78 (2021). 19. Kleshchevnikov, V. et al. Cell2location maps fine-grained cell types in spatial
7. Palla, G. et al. Squidpy: a scalable framework for spatial omics analysis. transcriptomics. Nat. Biotechnol. 40, 661–671 (2022).
Nat. Methods 19, 171–178 (2022). 20. Ansel, K. M. et al. A chemokine-driven positive feedback loop organizes
8. Yuan, Y. & Bar-Joseph, Z. GCNG: graph convolutional networks for inferring lymphoid follicles. Nature 406, 309–314 (2000).
gene interaction from spatial transcriptomics data. Genome Biol. 21, 300 (2020). 21. Biancalani, T. et al. Deep learning and alignment of spatially
9. Cang, Z. & Nie, Q. Inferring spatial and signaling relationships between cells resolved single-cell transcriptomes with Tangram. Nat. Methods 18,
from single cell transcriptomic data. Nat. Commun. 11, 2084 (2020). 1352–1362 (2021).
10. Garcia-Alonso, L. et al. Mapping the temporal and spatial dynamics of the 22. Lohoff, T. et al. Integration of spatial and single-cell transcriptomic data
human endometrium in vivo and in vitro. Nat. Genet. 53, 1698–1711 (2011). elucidates mouse organogenesis. Nat. Biotechnol. 40, 74–85 (2022).
11. Lu, Y. et al. Spatial transcriptome profiling by MERFISH reveals fetal liver 23. Hetzel, L., Fischer, D. S., Günnemann, S. & Theis, F. J. Graph representation
hematopoietic stem cell niche architecture. Cell Discov. 7, 47 (2021). learning for single cell biology. Curr. Opin. Syst. Biol. 28, 100347 (2021).
12. Zhang, M. et al. Molecular, spatial and projection diversity of neurons in
primary motor cortex revealed by in situ single-cell transcriptomics. Preprint Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in
at bioRxiv https://www.biorxiv.org/content/10.1101/2020.06.04.105700v1. published maps and institutional affiliations.
abstract (2020). Open Access This article is licensed under a Creative Commons
13. Schürch, C. M. et al. Coordinated cellular neighborhoods orchestrate Attribution 4.0 International License, which permits use, sharing, adap-
antitumoral immunity at the colorectal cancer invasive front. Cell 183, tation, distribution and reproduction in any medium or format, as long
838 (2020). as you give appropriate credit to the original author(s) and the source, provide a link to
14. Hartmann, F. J. et al. Single-cell metabolic profiling of human cytotoxic the Creative Commons license, and indicate if changes were made. The images or other
T cells. Nat. Biotechnol. 39, 186–197 (2021). third party material in this article are included in the article’s Creative Commons license,
15. Pascual-Reguant, A. et al. Multiplexed histology analyses for the phenotypic unless indicated otherwise in a credit line to the material. If material is not included in
and spatial characterization of human innate lymphoid cells. Nat. Commun. the article’s Creative Commons license and your intended use is not permitted by statu-
12, 1737 (2021). tory regulation or exceeds the permitted use, you will need to obtain permission directly
16. Jarosch, S. et al. Multiplexed imaging and automated signal quantification in from the copyright holder. To view a copy of this license, visit http://creativecommons.
formalin-fixed paraffin-embedded tissues by ChipCytometry. Cell Rep. org/licenses/by/4.0/.
Methods 1, 100104 (2021). © The Author(s) 2022

336 Nature Biotechnology | VOL 41 | March 2023 | 332–336 | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication
Methods MERFISH-scRNA-seq integration. We integrated scRNA-seq with MERFISH
Data. Fetal liver (MERFISH). Lu et al.11 measured wild type (WT) and Tet2−/− fetal data to impute the full gene expression in the spatial graph of cells measured
livers with MERFISH in 140 (WT) and 195 (Tet2−/−) images across four WT in MERFISH. We performed this integration between the MERFISH fetal liver
fetal livers at E14.5 and two Tet2−/− knockout fetal livers at E14.5, with 132 genes (WT) data and scRNA-seq of E14.5 whole fetal liver cells sequenced by 10x
observed in 40,864 (WT) and 54,970 (Tet2−/−) cells. We used cell types as originally Genomics platform11, which is available as GSE172127 on GEO. The scRNA-seq
annotated by Lu et al.: AEC, erythroid cell, erythroid progenitor, hepatocyte, dataset contains 9,448 cells across 28,692 features. We performed quality control
megakaryocyte, macrophage, myeloid and SEC. We removed cells with an unknown and removed cells with fewer than 500 detected genes, genes expressed in less
label from the dataset. We scaled model outputs by the node size in the respective than three cells and cells with more than 10% of the transcripts coming from
output layer of each model class to mitigate count noise (Supplementary Fig. 8f). mitochondrial genes from the dataset. We applied Tangram21 to generate a spatially
resolved representation of the scRNA-seq fetal liver dataset21. We used 131 out of
Brain (MERFISH). Zhang et al.12 measured mouse primary motor cortex with 132 genes from the MERFISH fetal liver (WT) data, which were both present in the
MERFISH in 64 images across two mice, with 254 genes observed in 284,098 MERFISH and the scRNA-seq dataset, to perform the mapping.
cells. We used the cell types originally annotated by Zhang et al.: astrocytes,
endothelial, L2/3 IT neurons, L4/5 IT, L5/6 near-projecting neurons, L5 IT, L5 Variance decomposition into inter- and intra-cell-type variance. The variance
pyramidal tract neurons, L6 cortico-thalamic projection neurons, L6 IT, L6 IT of a single-cell resolved dataset can be decomposed into intercell-type variance,
Car3, L6b, Lamp5, microglia, oligodendrocyte precursor cells, oligodendrocytes, intracell-type variance and gene variance. The gene variance is independent of
perivascular macrophages, pericytes, parvalbumin, smooth muscle cells, Sncg, cell type definitions and can therefore be considered separately from intra- and
somatostatin (Sst), Sst Chodl, vascular leptomeningeal cells, vasoactive intestinal inter-cell-type variance:
polypeptide, and other cells, where L identifies the layer (L1–L6) of the distinctive J 
N  2
laminar structure based on cytoarchitectural features (Extended Data Fig. 2a). yi,j − ȳ

Parvalbumin, Sst, vasoactive intestinal polypeptide, Sncg and Lamp5 define five i j

subclasses of GABAergic cells. We removed cells labeled as ‘other’ from the dataset. N J   2 N J  2 N J  2
We used an identifier for the respective mouse as domain information. = yi,j − ȳk(i),j + ȳk(i),j − ȳj + ȳj − ȳ
i j i j i j
        
Colon (chip cytometry). Jarosch et al.16 measured an inflamed colon with chip intracell− type variance intercell− type variance gene variance

cytometry in two images from one patient, with 19 genes observed in 11,321 (1)
cells. We used the cell types originally annotated by Jarosch et al.: B cells, CD4
T cells, CD8 T cells, GATA3+ epithelial, Ki67 high epithelial, Ki67 low epithelial, where yi,j is the gene expression of cell i out of N and gene j out of J, x̄k,j is the mean
lamina propria cells, macrophages, monocytes, PD-L1+ cells, intraepithelial expression of each gene j in each cell type k, k(i) is the cell type of cell i, ȳj is the
lymphocytes, muscular cells and other lymphocytes (Extended Data Fig. 2a). We mean expression of each gene j and ȳ is the mean of the dataset.
coarsened the cell type annotation by combining Ki67 high epithelial and Ki67 low
epithelial to a joined annotation of Ki67 epithelial. We log-transformed the gene Simulations. Segmentation errors. We simulated segmentation errors in which
expression values for use in the analyses presented here to mitigate count noise the segment boundary between two neighboring cells is misplaced (Extended
(Supplementary Fig. 8b). Data Fig. 3f,g). We selected a fraction of cells (10% or 50%) in the MERFISH fetal
liver data at random, selecting one neighbor at random for each selected cell, and
Cancer (MIBI-TOF). Hartmann et al.14 measured colorectal carcinoma and healthy transferred a fraction of the total molecular abundance vector of the selected cell to
adjacent tissue with MIBI-TOF in 58 images across four individuals, with 36 genes its neighbor.
observed in 63,747 cells. We used the cell types originally annotated by Hartmann
et al.: endothelial, epithelial, fibroblast, CD11c myeloid, CD68 myeloid, CD4 Spatial dependencies. We simulated single-cell resolved spatial data from scratch
T cells, CD8 T cells and other immune cells (Extended Data Fig. 2a). The cohort by using the cell graph from the chip cytometry colon data (Extended Data Fig. 2)
in this dataset includes two patients with colorectal carcinoma and two healthy and the simulated node-wise gene expression vectors. We modeled cell types using
controls. We scaled the model outputs by cell-wise size factors. the number of genes originally defined in the dataset and drew a mean expression
value for each gene from a uniform distribution between 0 and 10. We considered
Tonsils (MELC). Pascual-Reguant et al.15 measured tonsils from patients two scenarios of dependencies between cells: (1) a dataset without spatial
undergoing tonsillectomy with multiepitope ligand cartography (MELC), an dependencies in which all cells are drawn from one cell type and are independent
immunohistochemistry approach, in one image across one patient, with 51 and identically distributed and (2) a dataset with spatial dependencies in which
genes observed in 9,512 cells. We used the cell types originally annotated by cells belong to either one of two cell types, where we introduced dependencies on
Pascual-Reguant et al.: B cells, endothelial cells, innate lymphoid cell (ILC), the presence of the respective other cell type in the neighborhood in 50% of the
monocytes/macrophages/dendritic cells (DC), natural killer (NK) cells, plasma genes, with strong effect sizes drawn from a uniform distribution between 4 and 6.
cells, T cytotoxic cells, T helper cells (Extended Data Fig. 2a). We removed cells We fitted NCEMs, Misty9 and SVCA4 on both simulated datasets (Supplementary
labeled as ‘other’ from the dataset. Fig. 2). The simulated images contained over 4,500 cells per image. To reduce the
runtime for SVCA for these samples we cropped both images in the lower-right
Cancer (CODEX). Schürch et al.13 measured advanced-stage colorectal cancer corner to create images with approximately 850 cells each.
with CODEX in 140 images across 35 patients, with 57 genes observed in 272,266
cells. We used the cell types originally annotated by Schürch et al.: B cells, CD11b+ Models. The inputs to NCEMs are (1) a gene expression matrix Y ∈ RN×J where
monocytes, CD11c+ dendritic cells, CD11b+ CD68+ macrophages, CD163+ N is the number of cells and J is the number of genes, (2) a matrix of observed
macrophages, CD68+ macrophages, CD68+ macrophages GzmB+, CD68+ CD163+ cell types Xl ∈ RN×L where L is the number of unique cell type labels and (3)
macrophages, CD3+ T cells, CD4+ T cells, CD4+ T cells CD45RO+, CD4+ T cells a matrix specifying the batch assignments Xc ∈ RN×C of C distinct batches or
GATA3+, CD8+ T cells, NK cells, T regs, adipocytes, dirt, granulocytes, immune domains, such as images or patients. We denote the adjacency matrix of connected
cells, immune cells/vasculature, lymphatics, nerves, plasma cells, smooth muscle, cells as A ∈ RN×N , which is calculated based on the spatial proximity of cells
stroma, tumor cells, tumor cells/immune cells and vasculature (Extended Data Fig. per image. For linear models and models with an indicator aggregator, we used
2a). We removed cells with an annotation of dirt or an undefined label from the a binary adjacency matrix Aij = 1 if d(xa , xb ) ≤ δ max where d(·, ·) describes the
dataset. We merged the macrophage subclusters (CD11b+ CD68+ macrophages, euclidean distance between nodes a, b ∈ N in space and δmax is the neighborhood
CD163+ macrophages, CD68+ macrophages, CD68+ macrophages GzmB+ and size (resolution), and Aij = 0 otherwise. For models using graph convolutions,
CD68+ CD163+ macrophages) and the CD4+ T cell subclusters (CD4+ T cells, CD4+ we normalized A such that all rows sum to one: D−1A where D is the diagonal
T cells CD45RO+ and CD4+ T cells GATA3+). We scaled model outputs by the node degree matrix. The output of NCEMs is Ŷ ∈ RN×J , a reconstruction of the
node size in the respective output layer of each model class to mitigate count noise gene expression matrix. In selected datasets, we applied∑size factor scaling to
J
y
(Supplementary Fig. 8e). the network output Y using the size factors sfi = 1
∑ N ∑J
j ij
. The global data
y′ ′
N i ′ j′ i j

Lymph node (Visium). We performed deconvolution with cell2location on a 10x

19 handling per dataset is reported in Supplementary Table 3, model hyperparameters
Visium lymph node dataset based on a scRNA-seq dataset from the same tissue for linear models are reported in Supplementary Table 4, and the parameters for
as previously described19. The cell type labels used for deconvolution were B cells, nonlinear and CVAE models in Supplementary Table 5.
DC, endothelial, follicular dendritic cells, ILC, macrophage, mast, monocytes, NK,
natural killer T cells (NKT), CD4 T cells, CD8 T cells, T cells (TIM3), T follicular Loss functions. We use a Gaussian log-likelihood, ll, loss as an
regulatory cells fr, T regulatory cells reg) and vascular smooth muscle cells. optimization objective for linear and nonlinear models with
N ∑ J (√ (y −ŷ )2
ll(y) = N1∗J (−log 2πσ j − 0.5 ij σ 2 ij ) over cells i and genes j, where
∑ )
Dataset partitions. We randomly selected 10% of all nodes across all images and i j j

patients as the test set. From the remaining nodes, 10% of all nodes are selected as σj is the predicted standard deviation of a gene (Supplementary Fig. 8). The loss
the validation set. function of CVAE model is the negative data log-likelihood in addition to the

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology
Kullback–Leibler divergence between the variation posterior qϕ(z)
) and the prior Ligand–receptor NCEM (NL-NCEM-LR). Here we consider a specific
p(z) on the latent variables: llCVAE = −ll(y) + DKL qϕ (z)||p(z) . NL-NCEM with a tailored graph kernel. This graph kernel embeds each
(
cell into a receptor dimensional latent space z based on the receptor gene
Optimization. We ran grid searches to find the optimal set of hyperparameters for expression on the index cell, ligand gene expression on neighboring cells,
each dataset where the batch size is the number of images per dataset. We selected and the adjacency matrix A, which ∑ encodes the set of neighbors ) MN of cell i:
the number of nodes evaluated per image per batch to improve convergence. We fenc : zik = g(Ai , Yi,r(k) , Y:,l(k) ) = Mm fR (Yi,r(k) ) ∗ fL Ym,r(k) . Here, r(k) and l(k)
(
trained all models with the Adam optimizer: linear models with 0.05, the remaining encode the gene index of receptor and ligand that correspond to ligand–receptor
models with multiple learning rates of {0.5, 0.05, 0.005}. Additionally, we used a pair k. The latent state and graph-level predictors Xc are input to a fully connected
learning scheduler on the validation loss with a patience of 20 epochs, which reduces layer stack that decodes gene expression fdec : pθ (Yi |zi , Xc ). The corresponding
the learning rate by a factor of 0.5, so lrnew = lr ∗ 0.5 and early stopping with a nonspatial baseline is a nonlinear model (NL) that receives receptor expression
patience of 100 epochs. The exact description of all grid searches in code are supplied of the index cell as bottleneck activation and has the same decoder. This baseline
in the benchmarking repository (Code Availability). We trained linear models for model is not nested in the nonlinear ligand–receptor NCEM (NL-NCEM-LR)
hypothesis testing using ordinary least squares estimators on the full dataset. but models a baseline which imputes all genes’ expression based on ligand gene
expression within the cell.
Linear NCEM. The linear nonspatial baseline model infers a reconstruction
Ŷ from a node’s cell type and respective domain information via Ŷ= XDβ, Conditional variational autoencoder NCEM. A conditional variational autoencoder
where XD is the design matrix and β ∈ R(L+C)×J are the parameters learned NCEM (CVAE–NCEM) learns a distribution over node states Y based on a
by the model. The design matrix of nonspatial baseline models is given by node-wise latent space z. The nonspatial CVAE null model contains the cell
XD = (Xl , Xc ) ∈ RN×(L+C). The spatial counterpart model, the linear NCEM, type and graph-level predictors as a condition in the variational posterior and
has access to an additional spatial sender–receiver interaction matrix. First, the likelihood model. In CVAE–NCEM, the conditions are the cell type labels
we computed the binary sender cell presence in the neighborhood of each Xl, some graph-level predictors Xc and the local graph embedding g(A, Xl). The
cell XS = 1(AXl >0) ∈ {0, 1}N×L, where 1(·) represents an indicator function. encoder is given by fenc : qϕ (zi |Yi , Xli , g(A, Xl )i , Xci ) and the decoder is defined by
To generate a matrix representation of sender–receiver cell interactions, fdec : pθ (Yi |zi , Xli , g(A, Xl )i , Xci ). A CVAE–NCEM for a full dataset depends on
we compute the interaction terms between the cell type of the index cell both the niche and the type of the cell itself. This setting presents the challenge of
and the presence of each cell type in its neighborhood as the outer product
2 encountering a nonidentifiability between variance attributed to latent variables,
between Xl and XS. The resulting interaction matrix is XTS ∈ {0, 1}N×L , cell type conditions and neighborhood context. In this study, we consider the
and the design matrix for the linear model with interaction terms is given CVAE–NCEM trained on the molecular vectors of a single target cell type as a
2
by XD = (Xl , XTS , Xc ) ∈ RN×(L+L +C). This design matrix can be related to function of the full neighborhood context to remove the nonidentifiability with
a graph neural network: Xl and Xc are node-wise condition vectors that can respect to cell type variation and focus on the nonidentifiability between latent
be appended to a local graph embedding centered on an index cell, and XTS variables and neighborhoods.
is equivalent to an outer product of the one-hot-encoded representation of
an index cell with the projection obtained from a single-layer graph neural Model evaluation. We evaluated model performance using the coefficient of
∑
network that embeds one-encoded cell type feature vectors with a feature-wise J
(yij −ŷij )2
max pooling operator across the neighborhood without the index cell. This determination: R2i = 1 − ∑J
j
2 for cells i and over genes j. We selected
j yij −ȳij )
(
projection is a cell-type-dimensional indicator for the presence of each cell the best performing models based on R2 on a validation dataset and showed this
type in the neighborhood. Linear NCEMs perform parameter inference on metric evaluated on test data in the manuscript. The performance of CVAEs is
2
Ŷ = XDβ where β ∈ R(L+L +C)×J . We also considered an NCEM without additionally assessed in style transfer tasks. In style transfer, the gene expression
interaction terms which does not have receiver-specific sender effects but only state and neighborhood of a reference node a from the source domain is encoded
global sender effects, which account for the presence of senders in the niche to estimate the latent states of this node. This latent representation is then decoded
via XD = (Xl , XS , Xc ) ∈ RN×(L+L+C). We evaluated significance of coefficients to the target domain of cell b, which implies conditioning the decoding on the
corresponding to the interaction matrix XTS with a Wald test. target neighborhood:

za ∼ qϕ z|Ya , Xla , g(A, Xl )a , Xca (2)

( )
Linear NCEM for deconvoluted spot transcriptomics. The baseline model is the
same as for the standard linear NCEM. The corresponding NCEM treats the
spot as a neighborhood and uses the deconvoluted cell type abundances per spot
Ŷb = pθ za , Xlb , g(A, Xl )b , Xcb (3)
( )
XF ∈ R(N∗L)xL as a vector-shaped neighborhood summary, replacing a kernel on
a graph. Note that (N*L) is the number of spots times the number of cell types:
where a, b are cell indices, qϕ is the amortized variational posterior and pθ is the
this model treats every type- and spot-wise gene expression vector, a result of the
decoder network. See also Conditional variational autoencoder NCEM for details
deconvolution, as an observation. The overall design matrix of the linear model
on the notation.
includes the interaction between the target cell type and the spot composition, and
2
spot-wise covariates: XD ∈ R(N∗L)×(L+L +C). Note that here, the spot composition Unsupervised analysis. We used uniform manifold approximation and
is the same for all L gene expression prediction problems per spot. As for the linear projection (UMAP) to embed the cells in two dimensions for visualization of
NCEM, we again fit a linear model to this design matrix to predict deconvoluted high-dimensional data.
gene expression. One can define a corresponding nonlinear model that uses the We computed the UMAP of B cell, FDC and mast cell substates (Fig. 2a)
deconvoluted cell type abundances per spot XF ∈ RN×L as a vector-shaped node based on 50 principal components (PCs) and k = 500. We computed the UMAP
feature space. Note that N is the number of spots in this nonlinear model. These of the scRNA-seq reference dataset of lymph nodes (Extended Data Fig. 4b)
feature vectors can be connected based on spot proximity in a graph embedding of based on 50 PCs with k = 100. We computed the UMAP of the MERFISH brain
spots fenc : qϕ (zs |g(A, Xl )s ). The cell-type-wise gene expression decoder for spot s data12 matrix (Extended Data Fig. 5a) based on the first 35 PCs and the k-nearest
and cell type k is then fdec : pθ (Ysk |zs , Xk , Xcs ), where Xk is a one-hot embedding of neighbor graph with k = 10. We computed the UMAP of L2/3 IT neurons in
the cell type k. slice 153 (Extended Data Fig. 5a) and slice 162 (Supplementary Fig. 5a) of the
MERFISH brain dataset based on the first 40 PCs with k = 40 and performed
Nonlinear NCEM. NCEMs include nonlinear models that encode the Louvain community detection using Scanpy17 to define stable L2/3 IT substates.
neighborhood through a graph neural network (NL-NCEM) and decode We computed UMAPs of CD8 T cells in area 1 in the chip cytometry dataset
expression vectors. The corresponding nonspatial baseline model is a nonlinear (Extended Data Fig. 6) based on the gene expression matrix directly and k = 22,
model (NL) that predicts expression from cell type and graph-level predictors. and UMAPs of CD8 T cells in image 1, 5, 8 and 16 of the MIBI-TOF cancer
A local graph embedding is given by fenc : qϕ (zi |Xli , g(A, Xl )i , Xci ), which dataset (Extended Data Fig. 7) based on the gene expression directly and k = 60.
We performed Louvain community detection of the latent space in CVAE and
encodes the cell type labels Xl, some graph-level predictors Xc and the local
CVAE–NCEM IND models (Extended Data Fig. 9d,e and Supplementary
graph embedding g(A, Xl), based on the adjacency matrix A, into a latent
Fig. 7c,d) based on the latent space using k = 80 for the MERFISH brain dataset
state z. The latent state of cell i is input to a fully connected layer stack given
and k = 250 for the chip cytometry colon dataset.
by fdec : pθ (Yi |zi , Xli , Xci ). If one uses an indicator embedding function as
We performed cluster enrichment with Fisher’s exact test. Each
described in the section Linear NCEM and all hidden layers are removed from contingency table is composed of two categorical variables. The first
the NL-NCEM, a single linear transformation of the input remains, which is variable describes the binary assignment of cells to one L2/3 IT subcluster.
equivalent to the linear NCEM. Alternatively, g(A, Xl) can be a graph embedding The second variable describes the presence of a source cell type in their
learned by a graph-convolutional network (GCN)5. A one-layer GCN is given neighborhood. We performed Benjamini and Hochberg false discovery rate
by g(A, Xl ) = softmax(ReLU(ĀXl W)), where W ∈ RL×H is a weight matrix, H correction (FDR) of cluster enrichment P values. A similar approach was used for
is the dimension of the learned node representation and Ā is the normalized the cluster enrichment analysis of CD8 T cells in the chip cytometry colon and the
adjacency matrix. MIBI-TOF cancer datasets.

Nature Biotechnology | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication
Type coupling analysis. We performed type coupling analysis, sender effect Code availability
and receiver effect analysis based on a Wald test on the parameters estimates of All models described here are implemented in a Python package available at
linear NCEM obtained on the full dataset as ordinary least squares estimates. We https://github.com/theislab/ncem. All benchmarking and analysis codes are
performed FDR-correction of the resulting P values using the Benjamini-Hochberg provided at https://github.com/theislab/ncem_benchmarks. Tutorials for model
correction method. The coupling measure between sender and receiver cells usage are available from https://github.com/theislab/ncem_tutorials.
is the L1-norm of coefficients of significant coefficients that correspond to the
specific receiver–sender interaction term in the linear model, or the number of
differentially expressed genes. The sender effect and receiver effect analysis consists
of the set of coefficients and their significance for a particular sender and receiver,
References
24. Tabula Sapiens Consortium et al. The Tabula Sapiens: a multiple-organ,
respectively. The sender similarity analysis is a hierarchical clustering of the
single-cell transcriptomic atlas of humans. Science 376, eabl4896 (2022).
Pearson product-moment correlation coefficients of coefficient vectors of sender
cell types for one defined receiver cell type.
Acknowledgements
Differential receptor activity in NL-NCEM-LR. We used a t-test to obtain a We thank S. Richter, M. Lotfollahi, V. Kleshchevnikov, S. Günnemann, C. M. Schürch,
ranking for highly differential receptor signaling in SEC depending on the presence M. Zhang, X. Zhuang, S. Jarosch and D. Busch for valuable discussion and feedback
of neighboring AEC. We used the neighborhood size that corresponded to the best on this project. In particular, we want to thank S. Jarosch and D. Busch for sharing
performing resolution of the NL-NCEM-LR model. the chip cytometry colon dataset prepublication. We thank L. Hetzel, G. Palla and
L. Zappia for their valuable feedback on this manuscript. This work was supported
Subsampling robustness analysis. We randomly subsampled the spatial by the German Federal Ministry of Education and Research (BMBF) under grant
transcriptomics spots from the Visium lymph node data to 5%, 25%, 50% and no. 01IS18036B and no. 01IS18053A, by the Bavarian Ministry of Science and the
75% of all spots across three cross validations. We deconvoluted the resulting Arts in the framework of the Bavarian Research Association ‘ForInter’ (Interaction
subsampled slide with cell2location and used this inferred spot composition as of human brain cells), by the Wellcome Trust grant no. 108413/A/15/D and by the
input to the NCEM type coupling analysis for spot-transcriptomic data. In order Helmholtz Association’s Initiative and Networking Fund through Helmholtz AI (grant
to assess the robustness with respect to identified putative dependencies, we no. ZT-I-PF-5-01). D.S.F. acknowledges support from a German Research Foundation
computed the R2 between the inferred coefficient vectors over genes for each cell (DFG) fellowship through the Graduate School of Quantitative Biosciences Munich
type pair between the fit to the complete data and the fit to the subsampled slide. (QBM) (GSC 1006 to D.S.F.) and by the Joachim Herz Foundation. A.C.S. has been
funded by the German Federal Ministry of Education and Research (BMBF) under
CellPhoneDB and NicheNet. We inferred putatively communicating ligand– grant no. 01IS18036B.
receptor pairs in lymph nodes using CellPhoneDB as implemented in squidpy7
on scRNA-seq data24 with n = 53,275 cells on the 10,000 most variable genes. Author contributions
We quantified sender–receiver interactions as the number of significant ligand– D.S.F. and F.J.T. conceived the project. D.S.F. and A.C.S. performed the analysis and
receptor pairs at an FDR-corrected P value of 0.05. Additionally, we considered the wrote the code. D.S.F., A.C.S. and F.J.T. wrote the manuscript.
presence of nonzero expression of cognate ligand–receptor pairs (Extended Data
Fig. 4e). We performed the CellPhoneDB analysis shown in Supplementary Fig. 6
based on n = 1,000 permutations. Additionally, we used randomly subsampled data Funding
for the analysis of MERFISH brain12 10% with n = 27,655, MIBI TOF cancer14 40% Open access funding provided by Helmholtz Zentrum München - Deutsches
with n = 25,498 and CODEX cancer13 10% with n = 25,186. Forschungszentrum für Gesundheit und Umwelt (GmbH)
We defined the 5,000 most variable genes per receiver cell type as target genes
in a NicheNet analysis. For the following cell types, we limited the number of Competing interests
highly variable genes to the number given in brackets depending on the respective F.J.T. consults for Immunai Inc., Singularity Bio B.V., CytoReason Ltd and Omniscope
intracell-type heterogeneity: DC (500), endothelial (1,500), erythrocyte (250), HSC Ltd, and has ownership interest in Dermagnostix GmbH and Cellarity. The remaining
(1,000), macrophages (4,000), mast (1,000), monoctyes (2,000), myeloid (2,000), authors declare no competing interests.
neutrophil (400), stromal cells (1,500) and T Treg (3,000). We defined all remaining
genes as background genes for NicheNet. We selected the top-100-ranked ligands
from NicheNet and thresholded the putative ligands to be expressed in at least 5% Additional information
of all sender cells. Extended data are available for this paper at https://doi.org/10.1038/s41587-022-01467-z.
Supplementary information The online version contains supplementary material
Reporting summary. Further information on research design is available in the available at https://doi.org/10.1038/s41587-022-01467-z.
Nature Research Reporting Summary linked to this article.
Correspondence and requests for materials should be addressed to Fabian J. Theis.

Data availability Peer review information Nature Biotechnology thanks Qing Nie, Tommaso Biancalani
The MERFISH fetal liver11, MERFISH brain12, MIBI TOF cancer14, MELC tonsils15, and the other, anonymous, reviewer(s) for their contribution to the peer review of
CODEX cancer13, chip cytometry colon16 and Visium lymph node19 datasets are this work.
publicly available (Methods). Reprints and permissions information is available at www.nature.com/reprints.

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology

Extended Data Fig. 1 | Benchmarking cell–cell dependency inference on simulated data. Shown are fits of NCEMs (a, b), MistyR (c, d), and SVCA (e,
f) on simulated data with simulated dependencies between cells (a, c, e) and without simulated dependencies between cells (b, d, f). The simulation is
described in the Methods.

Nature Biotechnology | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication

Extended Data Fig. 2 | Cell-type centric summary statistics of the considered datasets. (a) Cell-type frequencies by dataset. Shown is a barplot with the
number of cells in each cell-type for MERFISH – brain data, chip cytometry – colon data, MIBI TOF – cancer data, MELC – tonsils data, CODEX – cancer
data, MERFISH - fetal liver wild type and MERFISH - fetal liver Tet2−/−. (b) Mean node degree (number of neighbors) by resolution in µm and dataset
for MERFISH – brain data (n = 284,098 cells), chip cytometry – colon data (n = 11.321 cells), MIBI TOF – cancer data (n = 63,747 cells), MELC – tonsils
data (n = 9,512 cells), CODEX – cancer data (n = 272,266 cells), MERFISH - fetal liver wild type (n = 40,864 cells) and MERFISH - fetal liver Tet2−/−
(n = 54,970 cells). For each box in (b), the centerline defines the median, the height of the box is given by the interquartile range (IQR), the whiskers are
given by 1.5 * IQR and outliers are given as points beyond the minimum or maximum whisker.

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology

Extended Data Fig. 3 | See next page for caption.

Nature Biotechnology | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication
Extended Data Fig. 3 | NCEM robustness with respect to data perturbation. (a) Robustness of length scales of spatial dependencies to data down-
sampling. Shown are the R2 values between predicted expression vectors and observed expression vectors for held-out test cells of linear models by
resolution in μm with cross validation indicated as point shape and line style, showing the relative performance of NCEM model and baseline model. The
downsampling was performed on the full set of images. (b) Out-of-domain generalization of NCEM fits across genotypes. We translated a linear NCEM
fit on wild type data to Tet2-/- knockout data and found a similar spatial dependency structure as in a linear NCEM fit on the knockout data alone. (c)
Comparison of baseline and optimal resolution linear NCEM fits between the transferred wild type model and the knockout fit on knockout test data,
based on R2 values between predicted expression vectors and observed expression vectors (n = 4,740 cells). (d) Comparison of true and predicted gene-
wise mean expression for different models on knockout evaluation data. (e) Comparison of R2 values attained by NCEM and baseline model in transfer
and in within-domain prediction task. (f) Concept of simulation of segmentation errors. Cells from a measured spatial graph are sampled at random
and a fraction of their molecular counts is transferred to neighboring cells, simulating a misplacement of a segmentation boundary between both cells
(Methods). (g) Robustness in terms of segmentation errors for baseline and NCEM linear model on the chip cytometry - colon dataset for 10% and 50%
of all nodes in the dataset and different strengths of augmentation (n = 3 cross-validation splits). For each box, the centerline defines the median, the
height of the box is given by the interquartile range (IQR), the whiskers are given by 1.5 * IQR and outliers are given as points beyond the minimum or
maximum whisker.

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology

Extended Data Fig. 4 | Robustness of cell communication inference on deconvoluted spot transcriptomics data. (a) 10x Visium slide of a lymph node
with the spot-wise abundance of B cells, follicular dendritic cells (FDC), and mast cells inferred with cell2location superimposed. (b) UMAP of cells in a
matched scRNA-seq data set of human lymph nodes, spleen and tonsils with cell types superimposed. (c) Type coupling heatmap of the Visium – lymph
node dataset, with edge width proportional to the number of differentially expressed genes at a false-discovery-rate-corrected p-value threshold of
0.05 for each pair of sender and receiver cell types. Only edges with at least 200 genes are shown. (d) Violin plot of Cxcl13 expression per cell for FDC
subclusters in the Visium - lymph node data. (e) Robustness of type coupling analysis in a Visium slide on human lymph nodes. Shown are R2 between
inferred cell type coupling vectors of randomly subsampled spots and the complete data for subsampling ratios of 5 %, 25 %, 50% and 75 % in three cross
validations (n = 256 type couplings in each boxplot). For each box, the centerline defines the median, the height of the box is given by the interquartile
range (IQR), the whiskers are given by 1.5 * IQR and outliers are given as points beyond the minimum or maximum whisker. (f-h) Correlation of measures
of cell communication events between pairs of cell types compared with type coupling scores from NCEM on the tabular sapiens lymph node dataset.
Shown are CellphoneDB permutation test results with the number of ligand-receptor pairs with positive mean expression (f), number of ligand-receptor
pairs with a FDR-corrected p-value below a threshold of 0.05 (g) and the number of ligands associated with a pair of cell types as identified by NicheNet
(h) (Methods). Each point is one pair of cell types. The vertical line indicates the threshold for showing edges in Fig. 2b.

Nature Biotechnology | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication

Extended Data Fig. 5 | See next page for caption.

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology
Extended Data Fig. 5 | Cell heterogeneity can be attributed to niche composition. (a) Spatial cell type distribution in the mouse brain. Shown are a
UMAP of molecular embedding of all cells in slide 153 (n = 7439 cells) with the cell type superimposed, followed by slice 153 of mouse brain in the
MERFISH – brain dataset with the spatial allocation of all cell types superimposed, field of view number 486 of the same slice with poly(A) RNA channel
superimposed at central z-plane (z = 4.5 µm), and the spatial proximity graph of the same field of view with a resolution of 100 µm. (b) UMAPs of
molecular embedding of L2/3 IT cells with molecular subclustering superimposed (colors as in b). (c) Distribution of cell-wise difference of R2 between
NCEM and non-spatial baseline model by molecular subcluster (L2/3 IT 0: n = 316, L2/3 IT 1: n = 314, L2/3 IT 2: n = 313, L2/3 IT 3: n = 133, L2/3 IT 4:
n = 128). The centerline of the boxplots defines the median, the height of the box is given by the interquartile range (IQR), the whiskers are given by 1.5
* IQR and outliers are given as points beyond the minimum or maximum whisker. (d) UMAPs of molecular embedding of all L2/3 IT cells in an example
image (n = 1204 cells) showing if a given cell-type is present in the neighborhood. The underlying neighborhoods were defined at the optimal resolution
defined in Fig. 1d (100 µm). (e) Heatmap of fold change and false-discovery rate corrected p-values of cluster enrichment of binary neighborhood labels,
where fold changes are the ratio between the relative neighboring source cell-type frequencies per subtype cluster and the overall source cell-type
frequency in the image. (f) Model performance on L2/3 IT cells in space on slice 153 of mouse brain in the MERFISH – brain dataset with L2/3 IT sub-
states (first panel), L2/3 IT, L4/5 IT, Sncg, and VLMC (second panel) and the difference of R2 between the NCEM at resolution of 100 µm and the best
nonspatial baseline model (third panel) superimposed. (g) Type coupling analysis of MERFISH – brain data, showing the number of differentially expressed
genes at a false-discovery-rate-corrected p-value threshold of 0.05 for each pair of sender and receiver cell types.

Nature Biotechnology | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication

Extended Data Fig. 6 | Attributing cell heterogeneity to niche composition in inflamed colon. (a) Area 1 of chip cytometry – colon dataset with cell-types
superimposed. (b) UMAPs of molecular embedding of CD8 T cells only with molecular subclustering superimposed (colors as in c). (c) Distribution of
cell-wise difference of R2 between spatial model non nonspatial baseline model by molecular sub-cluster (CD8 T cells 0: n = 74, CD8 T cells 1: n = 58,
CD8 T cells 2: n = 41, CD8 T cells 3: n = 37, CD8 T cells 4: n = 24). The centerline of the boxplots defines the median, the height of the box is given by the
interquartile range (IQR), the whiskers are given by 1.5 * IQR and outliers are given as points beyond the minimum or maximum whisker. (d) UMAPs of
molecular embedding of all CD8 T cells in area 1 (n = 234 cells) showing if a given cell-type is present in the neighborhood. The underlying neighborhoods
were defined at the best performing resolution identified in Fig. 1c (40 µm). (e) Heatmap of fold change and false-discovery rate corrected p-values
of cluster enrichment of binary neighborhood labels, where fold changes are the ratio between the relative neighboring source cell-type frequencies
per subtype cluster and the overall source cell-type frequency in the image. (f) Area 1 of colon in the chip cytometry – colon dataset with CD8 T cell
sub-states (left) and the difference of R2 between the NCEM interaction model at resolution of 40 µm and the best nonspatial baseline model (right). (g)
Type coupling analysis, showing the number of differentially expressed genes at a false-discovery-rate-corrected p-value threshold of 0.05 for each pair of
sender and receiver cell types.

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology

Extended Data Fig. 7 | Attributing cell heterogeneity to niche composition in colorectal cancer. (a) Field of view 16 of MIBI TOF – cancer dataset with
the spatial allocation of all cell-types superimposed. (b) UMAPs of molecular embedding of CD8 T cells only with molecular sub-clustering superimposed
(colors as in c). (c) Distribution of cell-wise difference of R2 between spatial model non-spatial baseline model by molecular sub-cluster (CD8 T cells
0: n = 304, CD8 T cells 1: n = 293, CD8 T cells 2: n = 278, CD8 T cells 3: n = 247, CD8 T cells 4: n = 207). The centerline of the boxplots defines the
median, the height of the box is given by the interquartile range (IQR), the whiskers are given by 1.5 * IQR and outliers are given as points beyond the
minimum or maximum whisker. (d) UMAPs of molecular embedding of all CD8 T cells in area 1 (n = 1,329 cells) showing if a given cell type is present
in the neighborhood. The underlying neighborhoods were defined at the optimal resolution identified in Fig. 1d (13 µm). (e) Heatmap of fold change
and false-discovery rate corrected p-values of cluster enrichment of binary neighborhood labels, where fold changes are the ratio between the relative
neighboring source cell-type frequencies per subtype cluster and the overall source cell-type frequency in the image. (f) Field of view 1, 5, 8 and 16 of colon
in the MIBI TOF – cancer dataset with CD8 T cell sub-states, cell type assignments to epithelial and T cells, and the difference of R2 between the NCEM
interaction model at a resolution of 13 µm and the best nonspatial baseline model (scale bar 50 µm). (g) Type coupling analysis, showing the number of
differentially expressed genes at a false-discovery-rate-corrected p-value threshold of 0.05 for each pair of sender and receiver cell types.

Nature Biotechnology | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication

Extended Data Fig. 8 | See next page for caption.

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology
Extended Data Fig. 8 | Nonlinear models of spatial dependencies of expression states. (a) A node-supervised model in which the label is the expression
vector of a cell and the input consists of categorical cell type assignments and a spatial proximity graph. This model can also be viewed as a nonlinear
regression model: a local graph embedding of each cell is reconstructed to a cell-wise expression state. The forward pass for a cell i is shown. (b) Inferred
nonlinear spatial dependencies. Shown are the R2 values for held-out test data of nonlinear models by resolution in µm with cross validation indicated
as point shape and line style and comparatively mean performance of linear model in Fig. 1d. Linear (interaction) (gray line): linear model with interaction
effects; NL: nonlinear model; IND: the graph kernel is an indicator function across cell types in the neighborhood (yellow lines); GCN: the graph kernel is a
graph convolution, a linear embedding of the cell types in the neighborhood (teal lines); split (point shapes): cross-validation split; bracket (*): significant
difference in paired t-test between baseline model and best spatial model with (MERFISH – brain dataset pIND = 0.033, chip cytometry – colon dataset
pGCN = 0.026, MIBI-TOF – cancer dataset pIND = 0.005 and pGCN = 0.036). (c) Heatmap of cumulative gradients (saliency) of gene expression prediction
of L2/3 IT with respect to the input cells, aggregated by the sender cell type clusters, on test data. Shown is a cumulative gradient matrix of L2/3 IT
predictions by source cell type and image (n = 64 images). The cumulative absolute gradients are derived from the absolute gradients tensor across each
cell’s molecular vector prediction with respect to the cells in the neighborhood (source cells) per image, by taking a sum across the molecular output
features and by taking a sum across source cells of the same type. We aggregated these saliency maps per sender-receiver cell type pair as SALS ∈ RL L
∗

, where L is the number of distinct cell types in the model. Non-normalized saliencies will show a pattern similar to the contact frequency matrix as cell
types with frequent connections will skew the learned importance of cell connections. Therefore, we normalized the saliencies by the absolute number
nab of occurrences of each cell type pair: SALSnorm
ab
= n1 SALSab. For each box in (c), the centerline defines the median, the width of the box is given by the
ab
interquartile range (IQR), the whiskers are given by 1.5 * IQR and outliers are given as points beyond the minimum or maximum whisker.

Nature Biotechnology | www.nature.com/naturebiotechnology

NaturE BiotEchnology Brief Communication

Extended Data Fig. 9 | Modeling intrinsic and extrinsic variation in deep latent variable models. (a) A node generative network (CVAE–NCEM) is a
conditional variational autoencoder in which the condition is not a constant but a graph embedding, which is also learned. The forward pass for a cell i
through the model is shown. (b) Latent variable models improve reconstructive performance. Shown are the R2 values of held-out test data based on the
forward pass model evaluation from chip cytometry – colon data for linear models, encoder–decoder models, and variational autoencoders for both NCEM
and nonspatial models (n=3 cross-validation splits). baseline: a nonspatial linear model of gene expression per cell-type; NCEM interactions: linear model
with interaction effects; NL: nonlinear model; IND: the graph kernel is an indicator function across cell-types in the neighborhood; GCN: the graph kernel is
a graph convolution, a linear embedding of the cell-types in the neighborhood. (c) Neighborhood transfer performance of NCEM and nonspatial models.
Shown is the R2 over cells in the test set for models trained on predicting muscular cells and Lamina propria cells for both CVAE and CVAE–NCEMs trained
on neighborhoods with different radii with optimization algorithm as color (n=3 cross-validation splits). Plain: normal CVAE training; aggressive: aggressive
encoder training. For each box in (b, c), the centerline defines the median, the height of the box is given by the interquartile range (IQR), the whiskers
are given by 1.5 * IQR and outliers are given as points beyond the minimum or maximum whisker. (d–f) Latent variables of CVAE–NCEM are confounded
with neighborhood conditions. (d) UMAP of molecular embedding in the CVAE–NCEM IND latent space of muscular cells in an example image (n = 1,149
cells) with molecular sub-clustering superimposed (muscle 0: n = 315, muscle 1: n = 287, muscle 2: n = 238, muscle 3: n = 183, muscle 4: n = 126). (e)
UMAPs of molecular embedding in the CVAE–NCEM IND latent space of all muscle cells in the same image with superimposed binary label of presence
of a given cell-type, as defined in the title, in the neighborhood. The underlying neighborhoods were defined at a resolution of 100 µm. (f) Heatmap of fold
change and false-discovery corrected p-values of cluster enrichment of binary neighborhood labels, where fold changes are the ratio between the relative
neighboring source cell-type frequencies per subtype cluster and the overall source cell-type frequency in the image.

Nature Biotechnology | www.nature.com/naturebiotechnology

Brief Communication NaturE BiotEchnology

Extended Data Fig. 10 | Modeling ligand–receptor signaling with NCEM. (a) UMAP of cells in MERFISH – fetal liver data. (b) Imputation of MERFISH data
with scRNA-seq increases the number of genes that can be modeled with NCEMs, including receptor genes, ligand genes, and ligand–receptor pairs. (c)
Distribution of selected marker genes that are both observed in scRNA-seq and in MERFISH over cell types.

Nature Biotechnology | www.nature.com/naturebiotechnology