Articles

Robust enumeration of cell subsets from tissue

expression profiles
Aaron M Newman1,2,10, Chih Long Liu1,2,10, Michael R Green2,3,9, Andrew J Gentles3,4, Weiguo Feng5, Yue Xu6,
Chuong D Hoang6, Maximilian Diehn1,5,7 & Ash A Alizadeh1–3,7,8

We introduce CIBERSORT, a method for characterizing cell (SVR), a machine learning approach highly robust with respect
composition of complex tissues from their gene expression to noise10 (Online Methods and Supplementary Discussion).
profiles. When applied to enumeration of hematopoietic Unlike previous methods, SVR performs a feature selection, in
subsets in RNA mixtures from fresh, frozen and fixed tissues, which genes from the signature matrix are adaptively selected
including solid tumors, CIBERSORT outperformed other methods to deconvolve a given mixture (Supplementary Fig. 1). An
© 2015 Nature America, Inc. All rights reserved.

with respect to noise, unknown mixture content and closely empirically defined global P value for the deconvolution is then
related cell types. CIBERSORT should enable large-scale analysis determined (Fig. 1a).
of RNA mixtures for cellular biomarkers and therapeutic targets
(http://cibersort.stanford.edu/). Design and validation of a leukocyte signature matrix
To assess the feasibility of leukocyte deconvolution from bulk
Changes in cell composition underlie diverse physiological states tumors, we designed and validated a leukocyte gene signature
of metazoans and their complex tissues. For example, in malig- matrix, termed LM22. It contains 547 genes that distinguish
nant tumors, levels of infiltrating immune cells are associated 22 human hematopoietic cell phenotypes, including seven
with tumor growth, cancer progression and patient outcome1,2. T-cell types, naïve and memory B cells, plasma cells, natural
Common methods for studying cell heterogeneity, such as killer (NK) cells and myeloid subsets (Supplementary Table 1,
immuno­histochemistry and flow cytometry, rely on a limited rep- Supplementary Fig. 2 and Online Methods). Cell subsets can
ertoire of phenotypic markers, and tissue disaggregation before be further grouped into 11 major leukocyte types on the basis
flow cytometry can lead to lost or damaged cells, altering results3. of shared lineage (Supplementary Table 1). To validate the gene
Recently, computational methods were reported for predicting signatures in LM22, we applied it to deconvolve external data
fractions of multiple cell types in gene expression profiles (GEPs) sets of variably purified leukocyte subsets. CIBERSORT results
of admixtures3–9. Although such methods perform accurately on matched ground-truth phenotypes in 93% of these data sets
distinct cell subsets in mixtures with well-defined composition (Fig. 1b, Supplementary Fig. 3a and Supplementary Table 2).
npg

(for example, blood), they are considerably less effective for mix- CIBERSORT also produced results consistent with highly puri-
tures with unknown content and noise (for example, solid tumors) fied T and B cells that we flow sorted from five human tonsils
and for discriminating closely related cell types (for example, (Supplementary Fig. 3b).
naïve vs. memory B cells). We present cell-type identification by We next evaluated the CIBERSORT P value metric for
estimating relative subsets of RNA transcripts (CIBERSORT), sensitivity and specificity by using LM22 to deconvolve 3,061
a computational approach that accurately resolves relative human transcriptomes11. We first scored expression profiles
fractions of diverse cell subsets in GEPs from complex tissues as ‘positive’ or ‘negative’ depending on the presence or absence
(http://cibersort.stanford.edu/). of at least one cell type in LM22, respectively. This distinc-
tion was considered separately for primary tissue specimens
RESULTS (n = 1,425 positive, 376 negative) and transformed cell lines
CIBERSORT requires an input matrix of reference gene expres- (n = 118 positive, 1,142 negative). At a P value threshold of ~0.01,
sion signatures, collectively used to estimate the relative propor- CIBERSORT achieved ≥94% sensitivity and ≥95% specificity
tions of each cell type of interest. To deconvolve the mixture, we for distinguishing positive from negative samples (area under
employ a novel application of linear support vector regression the curve (AUC) ≥ 0.98; Fig. 1c). Results were similar using an

1Institute for Stem Cell Biology and Regenerative Medicine, Stanford University, Stanford, California, USA. 2Department of Medicine, Division of Oncology,

Stanford Cancer Institute, Stanford University, Stanford, California, USA. 3Center for Cancer Systems Biology, Stanford University, Stanford, California, USA.
4Department of Radiology, Stanford University, Stanford, California, USA. 5Department of Radiation Oncology, Stanford University, Stanford, California, USA.
6Department of Cardiothoracic Surgery, Division of Thoracic Surgery, Stanford University, Stanford, California, USA. 7Stanford Cancer Institute, Stanford University,

Stanford, California, USA. 8Department of Medicine, Division of Hematology, Stanford Cancer Institute, Stanford University, Stanford, California, USA. 9Present address:
Eppley Institute for Research in Cancer and Allied Diseases, University of Nebraska Medical Center, Omaha, Nebraska, USA. 10These authors contributed equally to this
work. Correspondence should be addressed to A.A.A. ([email protected]).
Received 23 MAY 2014; accepted 2 FEBRUARY 2015; published online 30 March 2015; doi:10.1038/nmeth.3337

nature methods | VOL.12 NO.5 | MAY 2015 | 453

 Articles

a Bulk
tumor Relative fractions and
Transcriptome global P value estimate
Purify profile Gene expression
signature matrix Transcriptome
profile

Permutation analysis

Significance
CIBERSORT
threshold

v-support vector regression

(linear kernel)

b Validation of leukocyte subsets in LM22

(n = 22 studies, 208 arrays)
c CIBERSORT P value evaluation
(n = 3,061 diverse human arrays) d Whole blood
(n = 24)
Naïve B cells 100 100

Sensitivity for cell types in LM22 (%)

Memory B cells R = 0.91
Plasma cells P < 0.0001
CD8 T cells

CIBERSORT fraction (%)

CD4 naïve T cells 80 80
CD4 memory T cells–
CD4 memory T cells+ Sn Sp AUC
Follicular helper T cells 60
γδ T cells 60 Non-cell lines 97% 99% 0.99
NK cells–
NK cells+ (n = 1,801)
Monocytes 40
M0-Macrophages 40 Cell lines 94% 95% 0.98
M2-Macrophages (n = 1,260)
Dendritic cells–
Dendritic cells+ 20 Lymphocytes
Mast cells– 20
Monocytes
© 2015 Nature America, Inc. All rights reserved.

Mast cells+
Eosinophils PMNs
Neutrophils
0 0
CIBERSORT fraction 0 100 0 20 40 60 80 100 0 20 40 60 80 100
0% 90% % correct 100 – specificity (%) Coulter counter fraction (%)

Figure 1 | Overview of CIBERSORT and application to leukocyte deconvolution. (a) Schematic of the approach. (b,c) Application of a leukocyte signature
matrix (LM22) to deconvolution of 208 arrays of distinct purified or enriched leukocyte subsets (b; Supplementary Table 2) and 3,061 diverse human
transcriptomes (c). Sensitivity (Sn) and specificity (Sp) in c are defined in relation to positive and negative groups (Online Methods). AUC, area under
the curve. (d) CIBERSORT analysis of 24 whole blood samples for lymphocytes, monocytes and neutrophils (PMNs) compared to measurements by Coulter
counter12. Concordance was measured by Pearson correlation (R) and linear regression (dashed line), and statistical significance was assessed by an
F-test. “CIBERSORT fraction” in b denotes the relative fraction assigned to each leukocyte subset by CIBERSORT. Resting and activated subsets in b are
indicated by − and +, respectively.

independently derived leukocyte signature matrix4 instead of (Supplementary Figs. 5 and 6 and Online Methods). By com-
LM22 (data not shown). bining these mixtures with a colon cancer cell line, we simulated
human solid tumors with varying leukocyte infiltration (1–100%).
Performance on well-defined mixtures We also tested the addition of non-log-linear noise to simulate
We then benchmarked CIBERSORT on idealized mixtures with sample handling, stochastic gene expression variation and
well-defined composition4,12,13 (Online Methods) and compared platform-to-platform differences. Although this simulation
it with six GEP deconvolution methods: linear least-squares framework does not fully reflect biological admixtures of solid
npg

regression (LLSR)4, quadratic programming (QP)5, perturba- tumors, it provided a reasonable model in which unknown
tion model for gene expression deconvolution (PERT)6, robust content and added noise could be finely tuned and tested.
linear regression (RLR), microarray microdissection with Nearly all methods degraded in performance as a function of
analysis of differences (MMAD)7 and digital sorting algorithm signal loss (Supplementary Fig. 5 and Supplementary Table 4),
(DSA)8 (Supplementary Table 3). CIBERSORT, like other meth- showing highly reduced accuracy below 50% immune content.
ods, achieved accurate results on idealized mixtures (Fig. 1d, Only CIBERSORT accurately resolved known mixture propor-
Supplementary Fig. 4a,b and Supplementary Table 4). Then, tions over nearly the entire ranges of tumor content (up to ~95%)
to investigate whether CIBERSORT might be useful for immune and noise (up to ~70%) (Fig. 2a), exhibiting strong performance
monitoring, we profiled peripheral blood in patients immediately on mixtures that diverged considerably from their original com-
before and after they received rituximab monotherapy for non- positions (Pearson’s R as low as ~0.05; Fig. 2b). As many solid
Hodgkin’s lymphoma. CIBERSORT analysis of post-treatment tumor types are composed of <50% infiltrating immune cells14,
peripheral blood mononuclear cells (PBMCs) with LM22 revealed the parameter range in which CIBERSORT outperformed other
a selective depletion of B cells targeted by rituximab in all four methods is highly relevant for bulk tumor analysis.
patients tested (Supplementary Fig. 4c), suggesting utility for To assess the detection limit of each method for rare cell types
leukocyte monitoring during immunotherapy, especially when in bulk tissues, we created a second synthetic data set of the same
specimens cannot be immediately processed. cell lines, but with one blood cell line spiked into random mix-
tures of the other three blood subsets. CIBERSORT detected
Evaluation by simulation of bulk tissues cellular fractions down to 0.5% in mixtures containing ≤50%
To compare CIBERSORT’s technical performance with that of tumor content, and down to 1% in mixtures with >50% tumor
other methods on mixtures with unknown content, we employed content (Fig. 2c). Although all methods overestimated spike-ins
commonly used benchmark data sets consisting of four admixed with higher tumor content, the effect was least pronounced for
blood cancer cell lines4, each with distinct reference profiles CIBERSORT (Supplementary Fig. 6). Overestimation was less

454 | VOL.12 NO.5 | MAY 2015 | nature methods

 Articles
a R
1.0 b 35 RMSE
35 c
10

Performance error (RMSE)

0.8 0.8 30 29 Known
Added noise (×1 s.d.)

spike (%)

Estimated spike (%)

25
0.6 0.6 23 10.0
20 7.5
0.4 0.4 1 5.0
17
15 2.5
1.0
0.2 0.2 10 11 0.5
5
0 0 5 0.1
0 20 40 60 80 0 0.2 0.4 0.6 0.8 1.0 0 10 20 30 40 50 60 70 80 90
Tumor content (%) Deviation from original mixture (1 – R) Tumor content (%)

d Benchmark data set e Blood mixed with breast tissue

f Lung adeno- Ovarian Follicular
CIBERSORT Add tumor/noise carcinoma carcinoma lymphoma
Detection limit 100 100

Known blood
PMNs

content (%)
RLR 100R2 = 0.96 Eos
LM22-normalized immune

Estimated fraction of
P < 0.0001 50 MCs

leukocyte RNA (%)

PERT 80 75
n = 15 DCs
0
Method

Mono/MΦs
DSA
index (%)

60 1.0 50 NKs
γδ T cells

consistency (R)
Deconvolution
LLSR 0.9
40 CD4
0.8 CIBERS. 25 CD8
QP PERT
20 0.7 PCs
RLR
MMAD 0.6 QP 0 B cells
0 LLSR

ca00182
GSE31210
GSE3141
GSE10245
GSE19188

TCGA
GSE9899
GSE3149
GSE14764
GSE18521

GSE16131
GSE53820
GSE37088
GSE18736
This study
0.5
0 10 20 30 40 50 0 50 100
© 2015 Nature America, Inc. All rights reserved.

Sample (n = 12)
Performance error (RMSE) Known blood
content (%)

g DLBCL
(n = 18 tumors) h Normal lung tissue
i Follicular lymphoma
(n = 14 tumors)
60 50 100
R = 0.92 R = 0.97
R = 0.98
CIBERSORT fraction (%) (n = 77)

P < 0.0001 P = 0.002

40 P < 0.0001
80

CIBERSORT fraction (%)

40 *
30 60
FFPE (%)

Mono/MΦs Mono/MΦs (CD14)

CD4 T cells 20 CD4 T cells (CD4) 40
20 CD8 T cells CD8 T cells (CD8)
B cells B cells (CD19) B cells (CD20)
NK cells 10 NK cells (CD56) 20 * CD4 T cells (CD4)
Other CD45 Other CD45 CD8 T cells (CD8)
0 0 0
0 20 40 60 0 10 20 30 40 50 0 20 40 60 80 100
Frozen (%) Flow cytometry fraction (%) (n = 11) Flow cytometry fraction (%)

Figure 2 | Performance assessment on RNA mixtures from complex tissues. (a–c) CIBERSORT accuracy for leukocyte subset resolution in simulated tissues,
showing performance across added tumor content (x axis) and noise (y axis) (a), deviation of mixtures in a from original values (b), and detection limits
of a given cell type as a function of increasing tumor content (c) (n = 5 random mixtures for each data point). (d) Comparison of six GEP deconvolution
methods with CIBERSORT for the analyses shown in a–c (Supplementary Figs. 5 and 6). RMSE, r.m.s. error. (e) Analysis of whole blood spiked into
npg

breast tissue. Left, reported blood proportions versus immune-related gene expression (LM22-normalized immune index; Online Methods). Right, stability
of leukocyte deconvolution across methods. (f) CIBERSORT consistency across independent studies within and across cancer types (for leukocyte
abbreviations, see Supplementary Table 1; for data set details, see Online Methods). (g–i) CIBERSORT performance compared between 18 paired frozen
and formalin-fixed, paraffin-embedded (FFPE) DLBCL samples (g) and compared to flow cytometry analysis of 11 normal lung tissues (h) and 14 follicular
lymphoma tumors (i). Asterisks in i indicate potential outliers from the same patient. Surface markers used for quantitation in h and i are indicated
in parentheses. Results in e–i were obtained using LM22 and then collapsed into 11 major leukocyte types before analysis (Supplementary Table 1).
Concordance was determined by Pearson correlation (R) and linear regression (dashed lines), and statistical significance was evaluated by an F-test in e
(left) and g–i. Values in c and h are presented as medians ± 95% confidence intervals.

common in a separate analysis, in which each cell type in LM22 to diverse cell phenotypes (Supplementary Fig. 8). In addi-
was spiked into random combinations of the remaining 21 sub- tion, despite performing a feature selection on signature matrix
sets over a range of unknown content (Supplementary Fig. 7). genes (Supplementary Fig. 1), we did not observe any effect of
Overall, CIBERSORT consistently outperformed other methods, the choice of genes on deconvolution of closely related cell types
substantially in some cases (Fig. 2d, Supplementary Figs. 5–7 (Supplementary Fig. 9 and Supplementary Results). When
and Supplementary Table 4). compared to other methods on synthetic mixtures of increas-
ingly correlated cell types, CIBERSORT performed most accu-
Deconvolution of closely related cell types rately (Supplementary Fig. 10), demonstrating potential for deep
We next investigated CIBERSORT’s discriminatory ability on cell deconvolution of many cell subsets3.
types with correlated GEPs, which can be difficult to resolve in
mixtures owing to multicollinearity15. Previous approaches avoid Consistency on mixtures with unknown content or noise
this issue by using cell type–specific marker genes7,8,13 or highly Having benchmarked CIBERSORT on simulated mixtures, we next
distinct GEPs4,5. In contrast, CIBERSORT does not require cell focused on in vitro and in vivo mixtures of solid tissues, includ-
type–specific expression for every gene, suggesting applicability ing bulk tumors. We used LM22 for all remaining analyses and

nature methods | VOL.12 NO.5 | MAY 2015 | 455

 Articles
restricted our comparative assessments to expression-based meth- tissues or (ii) GEPs from 14 paired bulk FL samples, results were
ods (RLR, PERT, QP and LLSR). First, we tested deconvolution significantly correlated with corresponding flow cytometry mea­
stability in a spike series of whole blood added to breast tissue5. surements (P ≤ 0.005, Fig. 2h,i) and, in both tissue types, more
After verifying relative spike-in proportions by comparison with closely reflected experimental values than did previous methods
immune-related gene expression, we found that CIBERSORT was (Supplementary Table 4).
significantly more consistent than other methods (P < 0.02; n = 9 To assess performance on individual cell subsets, we used flow
samples with <100% blood; paired two-sided Wilcoxon signed rank cytometry to enumerate nearly 50% of the phenotypic repertoire
test; Fig. 2e and Supplementary Table 4). Separately, across inde- of LM22 (10 of 22 cell subsets) and evaluated CIBERSORT’s
pendent studies, leukocyte fractions enumerated by CIBERSORT capability for deep deconvolution in PBMCs and tumor biopsies.
were more similar within a cancer type than across cancers (Fig. 2f). Blood samples from 27 adult subjects were profiled for ten pheno­
These results indicate that unknown content and lab-specific types captured in LM22 (20 subjects were profiled for nine cell
factors only marginally affect CIBERSORT performance. types, and seven profiled for FOXP3+ T regulatory cells (Tregs);
We next applied CIBERSORT to ­ formalin-fixed, paraffin- Supplementary Note). Of these ten phenotypes, half are highly
embedded (FFPE) samples. Using publicly available GEPs of collinear in LM22 (Supplementary Fig. 2c) and half have low
matching FFPE and frozen DLBCL (diffuse large B-cell lym- median frequencies (<5%) in PBMCs (Supplementary Note).
phoma) tumors16, we found that leukocyte fractions estimated Despite the diversity of phenotypes analyzed, 90% of distinct leuko­
by CIBERSORT were significantly correlated across all tumors cyte subsets were significantly correlated between CIBERSORT
(Fig. 2g) and were more concordant than fractions determined and flow cytometry (P ≤ 0.02; Fig. 3a), including four of five sub-
by other methods (Supplementary Table 4). Indeed, CIBERSORT sets with median fractions below 5% (for example, Tregs; Fig. 3b).
results were also significantly correlated in 16 of 18 individual Only γδ T cells were not significant (albeit positively correlated;
© 2015 Nature America, Inc. All rights reserved.

tumors (P < 0.05; Supplementary Fig. 11a) and in specific cell R = 0.29), possibly owing to technical issues with flow cyto­metry
subsets (Supplementary Fig. 11b), implying potential utility for or the use of a suboptimal reference profile (Supplementary
large-scale analysis of cellular composition in FFPE specimens. Fig. 3a). Separately, in FL tumor biopsies, CD4 and CD8 T cells
and malignant B cells were each significantly correlated between
Comparison to flow cytometry CIBERSORT and flow cytometry (P ≤ 0.02; Fig. 3c).
To evaluate CIBERSORT against ground-
truth measurements of leukocyte content
in solid tissues, we used flow cytometry to a Deep deconvolution of PBMCs (n = 20 subjects)
Naïve B cells Memory B cells CD8 T cells Naïve CD4 T cells
enumerate immune subsets in two tissue 20
R = 0.58
8
R = 0.63
60
R = 0.85
30
R = 0.65
types: lung specimens obtained during sur- P = 0.008 P = 0.003 P < 0.0001 P = 0.002

gical resection of early stage non-small-cell 10 4 30 15

lung carcinomas and disaggregated lymph
CIBERSORT fraction (%)

node biopsies from follicular lymphoma 0 0 0 0

(FL) patients. Whether applied to (i) inde- 0 10 20 0 3 6 0 20 40 0 25 50
Resting memory Activated memory NK cells
pendent microarray studies of normal lung CD4 T cells CD4 T cells
Monocytes

40 20 40 80
R = 0.50 R = 0.61 R = 0.76 R = 0.74
P = 0.02 P = 0.004 P = 0.0001 P = 0.0002
Figure 3 | Deep deconvolution and enumeration
npg

of individual cell subsets in 41 human subjects. 20 10 20 40

(a–c) Direct comparison between CIBERSORT and

flow cytometry for the indicated eight immune 0 0 0 0
cell subsets in peripheral blood mononuclear 0 20 40 0 4 8 0 20 40 0 25 50
Flow cytometry fraction (%)
cells (PBMCs) from 20 subjects (a), FOXP3+
T regulatory cells (Tregs) in PBMCs from seven b Tregs in PBMCs c Deconvolution of FL lymph node biopsies (n = 14 subjects)
(n = 7 subjects) B cells CD8 T cells CD4 T cells
subjects (b), and the indicated three immune
4 90 20 40
CIBERSORT fraction
CIBERSORT fraction

cell subsets in lymph node biopsies from 14 R = 0.86 R = 0.65 R = 0.68 R = 0.71
P = 0.01 P = 0.01 P = 0.007 P = 0.005
subjects with follicular lymphoma (FL) (c).
Concordance was determined by Pearson
(%)
(%)

2 70 10 20
correlation (R) and linear regression (solid lines)
and statistical significance was assessed by an
0 50 0 0
F-test. (d) Comparison of five expression-based 0 2 4 50 75 100 0 4 8 0 25 50
Flow cytometry
deconvolution methods on the data sets analyzed fraction (%)
Flow cytometry fraction (%)
in a–c. The shaded area denotes deconvolved
cell types that significantly correlated with d 1.0 PBMCs Tumor biopsies (FL)
Cell type
Naïve B cells
flow cytometry (P < 0.05, Pearson correlation). Memory B cells
Performance (R)

P < 0.05
Scatter plots and r.m.s. error values for all 0.5
CD8 T cells
Naïve CD4 T cells
methods are provided in Supplementary Resting memory CD4 T cells
Activated memory CD4 T cells
Figures 12 and 13 and Supplementary Table 4. Tregs
0 NK cells
In three instances, correlation coefficients could Monocytes
not be determined; these were assigned a value B cells (FL)
CD8 T cells (FL)
of 0 for inclusion in this panel (Supplementary
P

SR

LR

LR

T
R

R
Q

CD4 T cells (FL)

R

R
PE

SO

PE

SO
LL

LL

Table 4 and Supplementary Fig. 12). Data are

ER

ER
IB

IB

presented as means ± s.d.

C

Method

456 | VOL.12 NO.5 | MAY 2015 | nature methods

 Articles
When applied to the same data sets using LM22, other ­expression- Acknowledgments
based methods were generally less accurate, and none yielded We are grateful to H. Maecker, M. Davis, R. Levy and the Stanford Human
Immune Monitoring Center for assistance with this study. This work was
significant correlations for >50% of analyzed phenotypes supported by grants from the Doris Duke Charitable Foundation (A.A.A.),
(Fig. 3d, Supplementary Figs. 12 and 13 and Supplementary the Damon Runyon Cancer Research Foundation (A.A.A.), the B&J Cardan
Table 4). Moreover, certain subsets were observed to ‘drop Oncology Research Fund (A.A.A.), the Ludwig Institute for Cancer Research
out’ when enumerated by other methods, likely owing to multi­ (A.A.A. and M.D.), US National Institutes of Health (NIH) grant U01 CA154969
(A.J.G., W.F., Y.X., C.D.H. and M.D.), NIH grant U19 AI090019, NIH grant PHS
collinearity (for example, naïve CD4 T cell levels estimated by NRSA 5T32 CA09302-35 (A.M.N.), US Department of Defense grant W81XWH-12-
QP and LLSR in PBMCs; Fig. 3d and Supplementary Figs. 12 1-0498 (A.M.N.) and a grant from the Siebel Stem Cell Institute and the Thomas
and 13). Furthermore, for FL tumor biopsies, significant correla- and Stacey Siebel Foundation (A.M.N.).
tions were achieved by other methods only when all leukocyte AUTHOR CONTRIBUTIONS
subsets were evaluated together, not separately (except for CD8 A.M.N. and A.A.A. conceived of CIBERSORT, developed strategies for related
T cells inferred by RLR; Supplementary Fig. 13). Potential experiments, analyzed the data and wrote the paper. A.M.N. developed and
reasons for these performance differences are discussed in the implemented CIBERSORT. C.L.L. implemented web infrastructure and wrote
the paper. M.R.G. performed flow cytometry and gene expression profiling of
Online Methods and Supplementary Discussion. Collectively, leukocytes from human tonsils and peripheral blood. A.J.G. assisted in the
these results further demonstrate the advantages of CIBERSORT conceptual development of CIBERSORT. W.F., Y.X., C.D.H. and M.D. assisted in
for deep deconvolution and enumeration of cell subsets in tissues the collection and analysis of lung tissue. All authors discussed the results and
implications and commented on the manuscript at all stages.
with complex compositions.
COMPETING FINANCIAL INTERESTS
DISCUSSION The authors declare no competing financial interests.
To summarize, we present CIBERSORT for characterizing
© 2015 Nature America, Inc. All rights reserved.

Reprints and permissions information is available online at http://www.nature.

cell heterogeneity using RNA mixtures from nearly any tissue. com/reprints/index.html.
CIBERSORT exhibits substantially improved accuracy for the
analysis of mixtures with (i) noise or unknown content and 1. Hanahan, D. & Weinberg, R.A. Hallmarks of cancer: the next generation.
Cell 144, 646–674 (2011).
(ii) closely related cell types (Supplementary Fig. 14). When
2. Coussens, L.M., Zitvogel, L. & Palucka, A.K. Neutralizing tumor-promoting
applied with statistical filtration, CIBERSORT coupled with chronic inflammation: a magic bullet? Science 339, 286–291 (2013).
LM22 allows for highly sensitive and specific discrimination of 3. Shen-Orr, S.S. & Gaujoux, R. Computational deconvolution: extracting cell
human leukocyte subsets. We note that our filtration approach type-specific information from heterogeneous samples.. Curr. Opin.
Immunol. 25, 571–578 (2013).
is likely applicable to other signature matrices and other GEP 4. Abbas, A.R., Wolslegel, K., Seshasayee, D., Modrusan, Z. & Clark, H.F.
deconvolution methods. Deconvolution of blood microarray data identifies cellular activation
The most significant current limitation of CIBERSORT, and patterns in systemic lupus erythematosus. PLoS ONE 4, e6098
indeed all signature gene–based methods, is the fidelity of refer- (2009).
5. Gong, T. et al. Optimal deconvolution of transcriptional profiling data
ence profiles, which could deviate in cells undergoing ­heterotypic using quadratic programming with application to complex clinical blood
interactions, phenotypic plasticity or disease-induced dysregula- samples. PLoS ONE 6, e27156 (2011).
tion. Sampling a larger expression space by sorting populations 6. Qiao, W. et al. PERT: a method for expression deconvolution of human
blood samples from varied microenvironmental and developmental
from diverse physiological conditions (for example, tumor- conditions. PLoS Comput. Biol. 8, e1002838 (2012).
infiltrating immune cells) may mitigate this issue. Second, 7. Liebner, D.A., Huang, K. & Parvin, J.D. MMAD: microarray microdissection
CIBERSORT currently does not provide P values for detection with analysis of differences is a computational tool for deconvoluting
npg

limits of individual cell types. Third, despite CIBERSORT show- cell type-specific contributions from tissue samples. Bioinformatics 30,
682–689 (2014).
ing a considerably lower estimation bias than other approaches, it 8. Zhong, Y., Wan, Y.-W., Pang, K., Chow, L. & Liu, Z. Digital sorting of
systematically over- or underestimated some cell types (see r.m.s. complex tissues for cell type-specific gene expression profiles. BMC
error values in Supplementary Table 4); efforts to address this Bioinformatics 14, 89 (2013).
9. Zuckerman, N.S., Noam, Y., Goldsmith, A.J. & Lee, P.P. A self-directed
are under way. Finally, although CIBERSORT was not explicitly
method for cell-type identification and separation of gene expression
tested on RNA-seq data, the linearity assumptions made by our microarrays. PLoS Comput. Biol. 9, e1003189 (2013).
method are likely to hold, as previously suggested17. 10. Schölkopf, B., Smola, A.J., Williamson, R.C. & Bartlett, P.L. New support
We anticipate that CIBERSORT will prove valuable for analysis vector algorithms. Neural Comput. 12, 1207–1245 (2000).
11. Lukk, M. et al. A global map of human gene expression.. Nat. Biotechnol.
of cellular heterogeneity in microarray or RNA-seq data derived 28, 322–324 (2010).
from fresh, frozen and fixed specimens, thereby complementing 12. Shen-Orr, S.S. et al. Cell type–specific gene expression differences in
methods that require living cells as input. complex tissues. Nat. Methods 7, 287–289 (2010).
13. Kuhn, A., Thu, D., Waldvogel, H.J., Faull, R.L.M. & Luthi-Carter, R.
Population-specific expression analysis (PSEA) reveals molecular changes
Methods in diseased brain. Nat. Methods 8, 945–947 (2011).
Methods and any associated references are available in the online 14. Yoshihara, K. et al. Inferring tumour purity and stromal and immune cell
version of the paper. admixture from expression data. Nat. Commun. 4, 2612 (2013).
15. Farrar, D.E. & Glauber, R.R. Multicollinearity in regression analysis: the
problem revisited. Rev. Econ. Stat. 49, 92–107 (1967).
Accession codes. NCBI Gene Expression Omnibus: expression 16. Burington, B. et al. CD40 pathway activation status predicts response to CD40
data have been deposited with accession code GSE65136. therapy in diffuse large B cell lymphoma. Sci. Transl. Med. 3, 74ra22 (2011).
17. Gong, T. & Szustakowski, J.D. DeconRNASeq: a statistical framework for
Note: Any Supplementary Information and Source Data files are available in the deconvolution of heterogeneous tissue samples based on mRNA-Seq data..
online version of the paper. Bioinformatics 29, 1083–1085 (2013).

nature methods | VOL.12 NO.5 | MAY 2015 | 457

 ONLINE METHODS Illumina RNA amplification kit (Life Technologies), and 750 ng
Patient samples. All patient samples in this study were collected of labeled cRNA were hybridized overnight to Human HT-12
with informed consent for research use and were approved by V4 BeadChip arrays (Illumina). The arrays were then washed,
the Stanford Institutional Review Board in accordance with the blocked, stained and scanned on an Illumina BeadStation 500
Declaration of Helsinki. Tonsils were collected as part of routine following the manufacturer’s protocols. GenomeStudio software
tonsillectomy procedures at Lucile Packard Children’s Hospital version 1.9.0 (Illumina) was used to generate signal intensity
at Stanford University and then mechanically disaggregated values from the scans. For the second cohort (Fig. 3b), PBMCs
before cell suspensions were cryopreserved (Supplementary (1.4 × 106 to 4.0 × 106 cells per mL) from six healthy male adults
Fig. 3b). Peripheral blood mononuclear cells (PBMCs) were were isolated and prepared as described in Supplementary Note
isolated from specimens taken before and immediately follow- and then frozen at −80 °C until use. Total cellular RNA (≥300 ng)
ing four weekly doses of infusional rituximab (375 mg m −2) was isolated from these six subjects along with viably preserved
monotherapy for extranodal marginal zone lymphoma (EMZL) PBMCs from patient 4 (Supplementary Fig. 4c) using RNeasy
in a subject without measurable circulating disease (patient 1 Mini Kit (Qiagen) and assessed for yield (NanoDrop 2000,
in Supplementary Fig. 4c). PBMCs were respectively isolated Thermo Scientific) and quality (2100 Bioanalyzer, Agilent). Total
from specimens taken immediately following four cycles and RNA was linearly amplified (3′ IVT Express, Affymetrix), and
six cycles of RCHOP immunochemotherapy for treatment of cRNA was hybridized to HGU133A microarrays (Affymetrix)
DLBCL (patients 2 and 3 in Supplementary Fig. 4c). PBMCs according to the manufacturer’s protocol.
were also isolated from a subject following four cycles of rituxi-
mab for treatment of FL (patient 4 in Supplementary Fig. 4c); Overview of CIBERSORT. Deconvolution model. A variety of
this subject had ~2% circulating lymphoma cells at diagnosis, GEP deconvolution methods have been proposed, most of which
© 2015 Nature America, Inc. All rights reserved.

which were undetectable by CIBERSORT and flow cytometry fol- model an mRNA mixture m by a system of linear equations, corre-
lowing four rituximab infusions. Specimens of adjacent normal sponding to a weighted sum of cell type–specific GEPs3–9,12,13,17,19.
lung tissue were obtained during surgical resection of early stage Let B denote a GEP signature matrix and let f denote a vector con-
non-small-cell lung tumors (Fig. 2h). Surgical tissue biopsies were sisting of the unknown fractions of each cell type in the mixture.
obtained from untreated FL patients enrolled in a phase III clini- Then the problem of GEP deconvolution can be represented by
cal trial (NCT00017290 (ref. 18)) (Figs. 2i and 3c). Last, PBMCs m = f × B, provided that B contains more marker genes than cell
were obtained from 20 adults of varying ages receiving influenza types (i.e., the system is overdetermined4). The preponderance of
immunization (NCT01827462) (Fig. 3a) and from seven adults genes in whole-transcriptome studies renders this requirement
consisting of patient 4 in Supplementary Fig. 4c and six healthy trivial in practice. If the linearity argument is biologically plausi-
subjects (Fig. 3b, which includes patient 4). ble, as previous studies imply4,12,13,20, then genes with expression
profiles enriched in each cell type can be leveraged to impute
Flow cytometry. Details are provided in the supplement unknown cell fractions from mixture profiles5.
(Supplementary Note and Supplementary Results). Signature matrix. Matrices of cell-specific expression signatures—
termed base or basis matrices in prior studies 4,5,19—can be
Gene expression profiling. Nucleic acids were extracted from obtained by differential expression analysis of purified or enriched
tonsil specimens (Supplementary Fig. 3b) and PBMCs (patients cell populations. Gene signature matrices can be made more
1–3 in Supplementary Fig. 4c) using AllPrep DNA/RNA Mini robust by minimizing an inherent matrix property called the con-
npg

kits (Qiagen). For FL specimens (Figs. 2i and 3c), total RNA and dition number, which measures the stability of the linear system
genomic DNA were prepared and stored using Trizol and RNeasy to input variation or noise4,5. In this work, we measure signature
Midi Kits (Qiagen). Sufficient nucleic acid was confirmed for 80% matrix stability via the 2-norm condition number, calculated with
of archival FL specimens after quality-control assessment of a the “kappa” function in R. The specific steps used to build LM22
subset of these patients. Total RNA from FL samples was linearly are described in “LM22 signature matrix” below.
amplified (3′ IVT Express, Affymetrix) before microarray hybrid- The process of building a signature matrix represents a type of
ization. For all above samples, total cellular RNA (at least 300 ng) ‘filter method’, a preprocessing step that removes irrelevant fea-
was assessed for yield (NanoDrop 2000, Thermo Scientific) and tures before application of a specific machine learning approach or
quality (2100 Bioanalyzer, Agilent), and cRNA was hybridized prediction algorithm21. Specifically, the use of a signature matrix
to HGU133 Plus 2.0 microarrays (Affymetrix) according to the facilitates (i) faster computational running time owing to the elimi-
manufacturer’s protocol. nation of genes with uniform expression levels across the cell types
Two additional cohorts of PBMCs were analyzed in this study of interest (for example, housekeeping genes and unexpressed
(Fig. 3a,b). For the first cohort (n = 20 subjects; Fig. 3a), PBMCs genes) and (ii) a greater signal-to-noise ratio by preselecting
(~1 × 106 viable cells per mL) were collected in 1 mL Trizol reference profiles that have maximal discriminatory power
(Invitrogen) and stored at −80 °C until use. Total RNA was iso- (as measured by condition number). Whereas several previous
lated according to the Trizol protocol (Invitrogen). Total RNA expression-based methods rely on feature selection to build a sig-
yield was assessed using the Thermo Scientific NanoDrop 1000 nature matrix for deconvolution (for example, LLSR and QP), to
micro-volume spectrophotometer (absorbance at 260 nm and our knowledge, this is the first work to incorporate an additional
the ratio of 260/280 and 260/230). RNA integrity was assessed round of feature selection to adaptively select genes from an existing
using a Bioanalyzer NANO Lab-on-a-Chip instrument (Agilent). signature matrix (detailed below).
Biotinylated, amplified antisense complementary RNA (cRNA) Support vector regression (SVR). To address limitations of pre-
targets were prepared from 200 to 250 ng of total RNA using the vious methods (Supplementary Discussion), we propose a new

nature methods doi:10.1038/nmeth.3337

 approach for cell-type identification by estimating relative sub- As previously suggested for other linear deconvolution meth-
sets of RNA transcripts: CIBERSORT. Our strategy is based on ods, CIBERSORT works best on expression values in non-log-
a novel application of nu–support vector regression (ν-SVR)10, linear space20.
a machine learning method that outperformed other approaches Taken together, linear ν-SVR as implemented by CIBERSORT
in benchmarking experiments (Supplementary Fig. 14 and uniquely addresses key outstanding issues of gene expression
Supplementary Table 4). ν-SVR is an instance of support vector deconvolution (see Supplementary Discussion), including
machine (SVM), a class of optimization methods for binary clas- (i) robustness to noise and overfitting owing to both a linear loss
sification problems, in which a hyperplane is discovered that function22 and feature selection of genes from the signature
maximally separates both classes. The support vectors are a subset matrix and (ii) tolerance to multicollinearity via utilization of
of the input data that determine hyperplane boundaries. Unlike the L2-norm penalty function25. Moreover, CIBERSORT does
standard SVM, SVR discovers a hyperplane that fits as many data not require cell type–specific expression patterns for every gene,
points as possible (given its objective function 10) within a con- thereby allowing the construction of signature matrices with
stant distance, ε, thus performing a regression (Supplementary more cell types and phenotypic states than other methods do
Fig. 1). All data points within ε (termed the ‘ε-tube’) are ignored (Supplementary Fig. 8).
(open circles in Supplementary Fig. 1, left panel), whereas all P value estimation. In contrast to previous methods,
data points lying outside of the ε-tube are evaluated according to CIBERSORT produces an empirical P value for the deconvolution
a linear ε-insensitive loss function10. These outlier data points, using Monte Carlo sampling. This approach allows CIBERSORT
referred to as ‘support vectors’ (red circles in Supplementary to test the null hypothesis that no cell types in the signature matrix
Fig. 1), define the boundaries of the ε-tube and are sufficient (for example, LM22) are present in a given GEP mixture, m. For
to completely specify the linear regression function. In this way, this purpose, we use the Pearson product-moment correlation R
© 2015 Nature America, Inc. All rights reserved.

support vectors can provide a sparse solution to the regression calculated between m and f × B as the test statistic, though other
in which overfitting is minimized (a type of feature selection). distance metrics could be used. In order to derive an empirical
Notably, support vectors represent genes selected from the sig- P value, CIBERSORT must first derive a null distribution R*.
nature matrix in this work. Because the signature matrix B will contain only a small subset
The primary objective of SVR is to minimize both a loss func- of genes g compared to the whole transcriptome, g expression
tion and penalty function given a defined set of constraints. The values are randomly drawn from the parent GEP of m to create a
former measures the error associated with fitting the data, whereas random mixture m*i such that |m| = |m*i|. CIBERSORT is then
the latter determines model complexity. More specifically, SVR run on m*i to produce a vector of estimated cellular fractions, f*i.
solves an optimization problem that minimizes the following CIBERSORT determines the correlation coefficient R*i between
two quantities10: (i) a linear ε-insensitive loss function, which the random mixture m*i and the reconstituted mixture, f*i × B.
outperforms other common loss functions (for example, squared This process is repeated for I iterations (I = 500 in this work) to
error used in LLSR) in noisy samples22 and (ii) an L2-norm pen- produce R*.
alty function (the same as that used in ridge regression23), which Running time. Using three threads to simultaneously process
penalizes model complexity while minimizing the variance in the three values of ν (0.25, 0.5 and 0.75), and a 2.3-GHz Intel Core
weights assigned to highly correlated predictors24,25 (for example, i7 CPU with 8 GB RAM, we clocked CIBERSORT run time with
closely related cell types), thereby combating multicollinearity. LM22 at approximately 1.7 s per mixture sample after an empirical
For further details, see ref. 10. P value was calculated. The latter depends on the number of
npg

Two major types of SVR have been described, ε-SVR26 and permutations selected; for 100×, it would take ~170 s, or an
ν-SVR10; however, we applied ν-SVR in CIBERSORT because the additional 2.75 min.
ν parameter conveniently imparts both an upper bound on training Implementation and website. CIBERSORT was developed in
errors and a lower bound on support vectors10. Higher values of Java and R with a simple command-line interface for processing
ν yield narrower ε-tubes and, consequently, more support vectors10 gene expression data representing a mixture of different cell types,
(Supplementary Fig. 1). For CIBERSORT, ν-SVR is applied with along with a signature genes file that enumerates the genes that
a linear kernel to solve for f, and the best result from three values define the signature expression profile for each cell type. Given
of ν = {0.25, 0.5, 0.75} is saved, where ‘best’ is defined as the lowest these data, the tool generates the fractional representations of
r.m.s. error between m and the deconvolution result, f × B. Our each cell type present in the mixture and returns it to the website
current implementation of CIBERSORT executes ν-SVR using the to be rendered as a heat-map table and stacked bar plot represen-
“svm” function in the R package e1071. Regression coefficients tations. The application can also produce custom signature gene
are extracted with the following R command: files when provided with gene expression profiles of reference cell
populations and a class comparison table for those populations.
coef <- t(model$coefs) %*% model$SV The back-end website for CIBERSORT was built in PHP. The
interactive user interface is powered by the jQuery JavaScript
Negative SVR regression coefficients are subsequently set to 0 library and various open-source libraries (including phpMailer,
(as is done for LLSR), and the remaining regression coefficients idiorm, blueimp jQuery-File-Upload, DataTables, phpExcel and
are normalized to sum to 1, yielding a final vector of estimated mPDF), with the graphical user interface of the website powered
cell type fractions, f (notably, f denotes relative, not absolute, by Twitter Bootstrap 2.3.2. The site runs on an Apache server
fractions of each cell type from B in m). To decrease running time on a virtual machine and stores user and job data in a MySQL
and promote better overall performance, we normalize both B database. However, users have complete control over their data
and m to zero mean and unit variance before running CIBERSORT. and can delete them at will. The CIBERSORT website is hosted at

doi:10.1038/nmeth.3337 nature methods

 http://cibersort.stanford.edu/ and includes data sets used for LM22 signature matrix. We obtained GEP data from the pub-
benchmarking, tutorials for the use of CIBERSORT and prepara- lic domain for 22 leukocyte subsets profiled on the HGU133A
tion of input data, downloadable software and source code, and platform (Supplementary Table 1). Probe sets were preprocessed
example files. as described above. Significantly differentially expressed genes
between each population and all other populations were identified
Other GEP deconvolution methods. We compared CIBERSORT using a two-sided unequal variance t-test. Genes with a q value
results with six GEP deconvolution methods: four that take refer- <0.3 (false discovery rate29) were considered significant.
ence expression profiles as input—linear least-squares regression For each leukocyte subset, significant genes were ordered by
(LLSR)4, quadratic programming (QP)5, PERT6 and robust linear decreasing fold change compared to other cell subsets, and the
regression (RLR)—plus two that take genes uniquely expressed top G marker genes from each cell subset were combined into a
in a given cell type as input (i.e., marker genes)—MMAD7 and signature matrix BG. We iterated G from 50 to 200 across all sub-
DSA8 (Supplementary Table 3). To the best of our knowledge, sets and retained the signature matrix with the lowest condition
RLR was first applied to GEP deconvolution in this work. LLSR, number (condition number = 11.4, G = 102, n = 547 distinct
QP, RLR and DSA were run in R using “stats” (“lm” function), genes; Supplementary Table 1).
“quadprog,” “MASS” (“rlm” function, 100 maximum iterations) To prevent genes expressed on nonhematopoietic cell types
and “DSA”8 packages, respectively. Negative coefficients from from confounding deconvolution results, we also used two gene-
LLSR were set to 0 to approximate the approach used by Abbas filtration strategies. First, we identified genes with enriched
et al.4, and QP was run with non-negativity and sum-to-1 expression in nonhematopoietic cells or tissues using the Gene
constraints used by Gong et al.5,17. MMAD and PERT were run in Enrichment Profiler, an online compendium of diverse cells and
Matlab using author-supplied code6,7 (PERT was converted from tissues profiled on HGU133A (http://xavierlab2.mgh.harvard.
© 2015 Nature America, Inc. All rights reserved.

Octave using the Matlab converter “oct2ml”). PERT was assessed edu/EnrichmentProfiler/)30. Gene Enrichment Profiler calculates
using the same signature-gene matrices used for the other expression- an enrichment score (ES) for a given gene in a given cell or tissue
based methods. MMAD was evaluated using marker genes type based on the sum of linear model coefficients from all pair-
only, as this approach yielded superior results when compared wise comparisons of that gene with other samples. For each gene
to expression-based deconvolution (Fig. 3C vs. Fig. 3A in Liebner and cell or tissue type with ES >0, we determined the fraction of
et al.7). However, cell-specific marker genes could not be deter- nonhematopoietic cell or tissue samples in the Gene Enrichment
mined for all cell types in LM22; therefore, MMAD and DSA Profiler database and excluded genes from the signature
were not run on data sets where LM22 was applied. All methods matrix with a nonhematopoietic fraction >0.05. As a second fil-
were run in non-log-linear space. tration step, we omitted all genes from further analysis with a
mean log2 expression level ≥7 in all nonhematopoietic cancer
Microarray data sets and preprocessing. Samples profiled on cell lines profiled in the Cancer Cell Line Encyclopedia (CCLE)
Illumina or Agilent platforms in Figure 1b (and Supplementary (prenormalized gene expression data were extracted from CCLE_
Table 2) were downloaded as normalized matrices from pub- Expression_Entrez_2012-09-29.txt, downloaded from the Broad
lic repositories (either NCBI, EBI or literature; referenced in Institute). We termed the final signature matrix “LM22.”
Supplementary Table 2), and probes were converted to HUGO To validate the gene signatures used to distinguish each leu-
gene symbols using chipset definition files available from the kocyte subset in LM22, we applied CIBERSORT to a variety of
NCBI Gene Expression Omnibus (GEO). Human transcriptome external data sets, each containing one purified population also
npg

data11 from Figure 1c were downloaded as RMA-normalized present in the signature matrix. We tested GEPs from three micro-
arrays (E-MTAB-62, EBI ArrayExpress). Other Affymetrix array platforms: Affymetrix HGU133A and HGU133 Plus 2.0 and
arrays were obtained as CEL files, MAS5 normalized using the the Illumina Human-6 v2 Expression BeadChip. Affymetrix plat-
“affy” package in Bioconductor, mapped to NCBI Entrez gene forms were normalized and processed the same as described for
identifiers using a custom chip definition file (Brainarray ver- signature-matrix GEPs. The BeadChip data set was downloaded as
sion 12.1.0; http://brainarray.mbni.med.umich.edu/Brainarray/; a processed normalized matrix from ArrayExpress (E-TABM-633
also available at http://cibersort.stanford.edu/) and converted to (ref. 31)), and for genes mapped to more than one probe, the
HUGO gene symbols. The Illumina BeadChip arrays analyzed in probe with highest mean expression across all samples was further
Figure 3a were normalized with limma v3.20.8 (Bioconductor) analyzed. For each sample, the population with the highest
using “normexp” background correction with negative controls CIBERSORT-inferred fraction was compared to the known cell
(“neqc” function). For non-Affymetrix platforms, genes mapping type to assess CIBERSORT accuracy (Supplementary Table 2).
to >1 probe were collapsed at the gene level according to the probe For the analysis presented in Figure 1c, arrays were grouped
with highest mean expression across all samples. All microarray into 1,801 primary human specimens, consisting of 1,425 ‘positive’
studies were quantile normalized before analysis. Data in Figure 2f samples containing at least one mature hematopoietic subset in
were analyzed as described above for Affymetrix chipsets, and LM22 and 376 ‘negative’ samples containing incompletely dif-
were obtained from the following sources: GEO (identifiers that ferentiated nonhematopoietic specimens, normal brain tissue
begin with ‘GSE’), The Cancer Genome Atlas (TCGA), caArray (which typically contains microglia, but generally not cell types
(ca00182, which has identifier jacob-00182) and this study. For in LM22), and hematopoietic stem cells and progenitors (not in
normal lung tissues in Figure 2h, we analyzed GEO data sets LM22). Arrays were separately grouped into 1,260 transformed
GSE7670 (ref. 27) and GSE10072 (ref. 28), and for paired frozen cell lines, divided into 118 ‘positive’ hematopoietic samples and
and FFPE samples of DLBCL tumors in Figure 2g, we analyzed 1,142 ‘negative’ samples, with the latter consisting of both non-
GSE18377 (ref. 16). hematopoietic samples and K562 erythromyeloblastoid cell lines,

nature methods doi:10.1038/nmeth.3337

 which are hematopoietic in origin but highly distinct from subsets we combined the cell line mixtures with defined inputs of a GEP
present in LM22. Poorly annotated arrays were excluded from this from a colon cancer cell line (HCT116), calculated as the mean of
analysis. Although significance filtering was not applied in com- two replicate arrays (GSM269529 and GSM269530; GSE10650).
paring CIBERSORT to other methods, we imposed a P value cutoff Both GSE11003 and GSE10650 data sets were MAS5 and quan-
(≤0.01; see Fig. 1c) for deconvolution of bulk tumors (Fig. 2f). tile normalized together before analysis. To introduce noise, we
added values randomly sampled from the distribution 2N(0, f × σ),
Other signature matrices. In addition to LM22, custom signa- with f in the range [0,1) (i.e., y axis in Fig. 2a and Supplementary
ture matrices were designed for mixtures of hematopoietic cell Fig. 5a) and σ set to the global s.d. across the original mixtures
lines and neural populations (Supplementary Fig. 4a,b). In both represented in log2 space (σ = 11.6). As GSE11103 consists of
cases, previously normalized series matrix data sets (GSE11103 four distinct mixtures with three replicates each, we measured
(ref. 4) and GSE19380 (ref. 13)) were downloaded from GEO the performance of each algorithm over the entire set of 12 mix-
and quantile normalized. Signature matrices were subsequently tures (R and RMSE; Supplementary Fig. 5 and Supplementary
constructed using the same condition number minimization algo- Table 4). Moreover, we independently iterated over tumor content
rithm described for LM22 (above), omitting nonhematopoietic (0% to <100%) and noise (f, [0,1)) in 30 regularly spaced intervals
gene-filtration and validation steps. The final signature matrices such that, together, 900 sets of mixtures were analyzed.
for GSE11103 and GSE19380 comprised 584 probe sets (condi-
tion number = 1.86) and 280 probe sets (condition number = 1.8), Analysis of cell subset detection limit. We performed two in silico
respectively. To compare CIBERSORT performance with marker experiments to assess the detection limits of different deconvolu-
gene–based methods (as in Supplementary Table 4), we defined tion algorithms. In the first experiment (Supplementary Fig. 6),
marker genes from each signature matrix by selecting all genes we used the same cell line GEPs described above to compare
© 2015 Nature America, Inc. All rights reserved.

with at least fivefold higher expression in one cell type compared CIBERSORT and RLR with five other GEP deconvolution meth-
to the others (as in ref. 7). ods4–8. We evaluated detection limit using Jurkat cells (spike-in
concentrations of 0.5%, 1%, 2.5%, 5%, 7.5% and 10%), whose ref-
Statistical analysis. Unless stated otherwise, concordance erence GEP (median of three replicates in GSE11103) was added
between known and predicted cell-type proportions was deter- into randomly created background mixtures of the other three
mined by Pearson correlation coefficient (R) and r.m.s. error blood cell lines. Five mixtures were created for each spike-in con-
(RMSE) to measure linear fit and estimation bias, respectively. centration. Predicted Jurkat fractions were assessed in the pres-
Of note, the latter was calculated on cell-type proportions repre- ence of differential tumor content, which we simulated by adding
sented as percentages. Group comparisons were determined using HCT116 (described above) in 10 even increments, from 0% to
a two-sided Wilcoxon test, unpaired or paired, as appropriate. 90%. Of note, we also used the same marker or signature genes
All results with P < 0.05 were considered significant. Statistical described for simulated tumors (above). In a second experiment
analyses were performed with R and Prism v6.0d (GraphPad (Supplementary Fig. 7a), we compared CIBERSORT with QP5,
Software, Inc.). The investigators were not blinded to allocation LLSR4, PERT6 and RLR. We spiked naïve-B-cell GEPs from the
during experiments and outcome assessment. No sample-size leukocyte signature matrix into four random background mixtures
estimates were performed to ensure adequate power to detect a of the remaining 21 leukocyte subsets in the signature matrix. The
prespecified effect size. same background mixtures were used for each spike-in. We also
tested the addition of unknown content by adding defined pro-
npg

Analysis of idealized mixtures. Unlike complex mixtures, ideal- portions (0–90%) of randomly permuted expression values from
ized mixtures are defined in this work as having well-defined com- a naïve-B-cell reference transcriptome (median expression profile
position, in which the majority of the mixture can be accounted for from samples used to build LM22, Supplementary Table 1). We
by highly distinct (uncorrelated) reference profiles of purified cell then repeated this analysis for each of the remaining leukocyte
types and in which the contribution from unknown cell content subsets in LM22 (Supplementary Fig. 7b).
and noise is minimal. CIBERSORT performed comparably to other
methods on idealized mixtures such as in vitro mixtures of blood Analysis of cell type–specific marker genes. Cell type–specific
cancer cell lines4 and neural cell types13 (Supplementary Fig. 4a,b) marker genes may be difficult if not impossible to ascertain
and whole blood12 (Fig. 1d and Supplementary Table 4). between closely related cell types. As such, we tested whether
marker genes expressed by >1 cell type in the signature matrix
Analysis of simulated tumors with added noise. We bench- could still be useful to CIBERSORT, provided that each reference
marked CIBERSORT against six GEP deconvolution methods profile in the signature matrix remains unique. We created two
(RLR and five others4–8) by comparing their results on mixtures artificial signature matrices (containing ten genes and five cell
with different levels of unknown content (i.e., tumor) and noise. types each) representing opposite extremes: one containing only
To facilitate a fair comparison, we used previously defined in vitro cell type–specific genes (called SM1; Supplementary Fig. 8a)
mixtures (n = 12) of four blood cell lines (GSE11103), each of and the other without any cell type–specific genes (called SM2;
which is highly distinct and readily deconvolved (Supplementary Supplementary Fig. 8b). Of note, unlike signature matrices
Fig. 4a). To evaluate expression-based methods, we used a signa- derived from real expression data, SM1 and SM2 are fully defined
ture matrix with nearly 600 distinguishing genes (described above and therefore ideally suited for this analysis. Moreover, reference
and applied in Supplementary Fig. 4a), whereas for marker-based profiles in SM2 are highly intercorrelated, as might be expected
deconvolution, we selected marker genes as described above for subsets without unique marker genes. We generated random
(n = 500 genes). To simulate tumors with infiltrating leukocytes, mixing proportions according to a uniform distribution and

doi:10.1038/nmeth.3337 nature methods

 combined the cell types in each signature matrix to create ten of deconvolution results over defined levels of blood admixed with
mixtures. We then added low-level noise by randomly shuffling breast tissue. To confirm reported fractions of blood admixed
genes in one of the mixtures and combining 5% of the resulting with breast tissue, we compared these proportions with an ‘LM22-
vector with 95% of each of the ten mixtures. CIBERSORT and normalized immune index’, defined for each sample as the median
DSA were compared using SM1 (Supplementary Fig. 8c), and gene expression value of all genes in LM22 (Supplementary Table 1)
CIBERSORT, RLR, QP, LLSR and PERT were compared using divided by the median expression level of the transcriptome and
SM2 (Supplementary Fig. 8d,e). Although CIBERSORT per- normalized into the range of known leukocyte content across the
formed identically to DSA on SM1, it was substantially more data sets (Fig. 2e). As a consistency metric, we compared decon-
accurate than other methods on SM2, closely approximating its volution results for each sample with results from the sample with
performance on SM1 (Supplementary Fig. 8d,e). This analysis highest immune purity (Fig. 2e).
demonstrates CIBERSORT’s softer dependency on cell type–
specific signature-matrix genes, an important requirement for
18. Levy, R. et al. Active idiotypic vaccination versus control immunotherapy
deep deconvolution3. for follicular lymphoma. J. Clin. Oncol. 32, 1797–1803 (2014).
19. Lu, P., Nakorchevskiy, A. & Marcotte, E.M. Expression deconvolution: a
Analysis of multicollinearity. We compared CIBERSORT with reinterpretation of DNA microarray data reveals dynamic changes in cell
three signature-gene-expression–based deconvolution methods, populations. Proc. Natl. Acad. Sci. USA 100, 10370–10375 (2003).
20. Zhong, Y. & Liu, Z. Gene expression deconvolution in linear space.
QP5, LLSR4 and RLR (this work), for the impact of multicolline- Nat. Methods 9, 8–9 (2012).
arity (i.e., the degree of intersample correlation in the signature 21. Guyon, I. & Elisseeff, A. An introduction to variable and feature selection.
matrix) on mixtures with unknown content (i.e., parts of the mix- J. Mach. Learn. Res. 3, 1157–1182 (2003).
22. Cherkassky, V. & Ma, Y. Practical selection of SVM parameters and noise
ture unaccounted for in the signature matrix), and noise added to
© 2015 Nature America, Inc. All rights reserved.

estimation for SVM regression. Neural Netw. 17, 113–126 (2004).

either B or m. Random signature matrices were created from 41 23. Hoerl, A.E. & Kennard, R.W. Ridge regression: biased estimation for
naïve-B-cell signature genes (derived from GSE22886 (ref. 32)) nonorthogonal problems. Technometrics 12, 55–67 (1970).
by randomly selecting and permuting N gene expression values 24. Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical
Learning 2nd edn. (Springer, 2009).
from the original nonrandom set of 41 genes, thus maintaining 25. Wang, L., Zhu, J. & Zou, H. The doubly regularized support vector
realistic gene expression distributions (n = 10 populations). The machine. Statist. Sinica 16, 589–615 (2006).
number of genes N was used to control multicollinearity within 26. Drucker, H., Burges, C.J.C., Kaufman, L., Smola, A. & Vapnik, V. in Adv.
the signature matrix (higher N = less collinear, and vice versa), Neural Inf. Process. Syst. (eds. Mozer, M.C., Jordan, M.I. & Petsche, T.) 9,
155–161 (MIT Press, 1997).
and for each N, ten random signature matrices were generated. 27. Su, L.J. et al. Selection of DDX5 as a novel internal control for Q-RT-PCR
Simulated mixtures were created by randomly apportioning pop- from microarray data using a block bootstrap re-sampling scheme.
ulations from the signature matrix. To simulate unknown content BMC Genomics 8, 140 (2007).
28. Landi, M.T. et al. Gene expression signature of cigarette smoking and its
(Supplementary Fig. 10a–c), we randomly combined and added role in lung adenocarcinoma development and survival. PLoS One 3, e1651
three concentrations (5%, 25% and 50%) of ten additional cell (2008).
populations to each mixture. Non-log-linear noise was additively 29. Storey, J.D. & Tibshirani, R. Statistical significance for genomewide
introduced into simulated mixtures (Supplementary Fig. 10d) by studies. Proc. Natl. Acad. Sci. USA 100, 9440–9445 (2003).
30. Benita, Y. et al. Gene enrichment profiles reveal T-cell development,
randomly sampling from 2N(0, j) (the exponent denotes a normal differentiation, and lineage-specific transcription factors including ZBTB25
distribution with mean of 0 and s.d. of j). Under all conditions as a novel NF-AT repressor. Blood 115, 5376–5384 (2010).
tested, CIBERSORT outperformed the other three methods. 31. Watkins, N.A. et al. A HaemAtlas: characterizing gene expression in
npg

differentiated human blood cells. Blood 113, e1–e9 (2009).

32. Abbas, A.R. et al. Immune response in silico (IRIS): immune-specific genes
Analysis of deconvolution consistency. We applied LM22 to a identified from a compendium of microarray expression data.
publicly available data set (GSE29832 (ref. 5)) to measure stability Genes Immun. 6, 319–331 (2005).

nature methods doi:10.1038/nmeth.3337