See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/281141495

An Introduction to Automated Flow Cytometry Gating Tools and Their

Implementation

Article in Frontiers in Immunology · August 2015

DOI: 10.3389/fimmu.2015.00380 · Source: PubMed

CITATIONS READS

51 2,761

4 authors, including:

Chris P Verschoor Alina Lelic

Health Sciences North McMaster University
91 PUBLICATIONS 2,443 CITATIONS 28 PUBLICATIONS 612 CITATIONS

SEE PROFILE SEE PROFILE

Dawn Bowdish
McMaster University
156 PUBLICATIONS 8,298 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

CASPER (Calgary Streptococcus pneumoniae Epidemiology Research) View project

All content following this page was uploaded by Dawn Bowdish on 04 September 2015.

The user has requested enhancement of the downloaded file.

 Hypothesis and Theory
published: 27 July 2015
doi: 10.3389/fimmu.2015.00380

Chris P. Verschoor, Alina Lelic, Jonathan L. Bramson and Dawn M. E. Bowdish*

Department of Pathology and Molecular Medicine, McMaster Immunology Research Centre (MIRC), McMaster University,
Hamilton, ON, Canada

Current flow cytometry (FCM) reagents and instrumentation allow for the measurement
of an unprecedented number of parameters for any given cell within a homogenous
or heterogeneous population. While this provides a great deal of power for hypothesis
testing, it also generates a vast amount of data, which is typically analyzed manually
through a processing called “gating.” For large experiments, such as high-content
screens, in which many parameters are measured, the time required for manual analysis
Edited by: as well as the technical variability inherent to manual gating can increase dramatically,
Anja Fuchs, even becoming prohibitive depending on the clinical or research goal. In the following
Washington University, USA
article, we aim to provide the reader an overview of automated FCM analysis as well as
Reviewed by:
Alfred Hyoungju Kim,
an example of the implementation of FLOw Clustering without K, a tool that we con-
Washington University School of sider accessible to researchers of all levels of computational expertise. In most cases,
Medicine, USA
computational assistance methods are more reproducible and much faster than manual
Holden Maecker,
Stanford University, USA gating, and for some, also allow for the discovery of cellular populations that might not
*Correspondence: be expected or evident to the researcher. We urge any researcher who is planning or has
Dawn M. E. Bowdish, previously performed large FCM experiments to consider implementing computational
Department of Pathology and
Molecular Medicine, McMaster
assistance into their analysis pipeline.
Immunology Research Centre
Keywords: flow cytometry, automated analysis, gating, high-throughput, software, immunology
(MIRC), McMaster University, MDCL
4020, 1280 Main Street West,
Hamilton, ON L8S 4K1, Canada
[email protected]
Introduction
Recent advances in flow cytometry (FCM) have provided researchers in the fields of cellular and
Specialty section:
clinical immunology an incredible amount of leverage toward testing new hypotheses. These include
This article was submitted to
Molecular Innate Immunity, a section
improvements to reagents and instrumentation employed in traditional fluorescence-based FCM,
of the journal Frontiers in Immunology allowing for the measurement of up to 20 parameters for any given cell (1), as well as the introduction
of mass cytometry [CyTOF, reviewed in Ref. (2)], which can measure up to 34 parameters for any
Received: 01 June 2015
Accepted: 12 July 2015
given cell. These technologies, while allowing for the discrimination of new and sometimes rare
Published: 27 July 2015 populations within a heterogeneous mixture of cells (akin to a needle in a haystack), also produce a
Citation:
tremendous amount of data, which is most commonly analyzed manually using proprietary software
Verschoor CP, Lelic A, Bramson JL (for example, FlowJo)1. For many immunologists, the manual analysis of FCM data does not hinder
and Bowdish DME (2015) An productivity, given that most FCM experiments are performed on a small number (<40) of experi-
introduction to automated mental units and involve less than a half dozen parameters. However, in the case of experiments
flow cytometry gating tools and that require the interrogation of more than 10 parameters in hundreds of experimental units, the
their implementation.
Front. Immunol. 6:380.
doi: 10.3389/fimmu.2015.00380 1
www.flowjo.com

Frontiers in Immunology | www.frontiersin.org 1 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

manual analysis of FCM can become a significant expenditure of surface or intracellularly expressed molecules (also known as
time. For example, our laboratory group has performed a number antigens) for a single cell within a larger population, doing so in a
of studies on the frequency and phenotype of peripheral blood high-throughput fashion, providing measurements for hundreds
­leukocytes (white blood cells) in elderly individuals, discrimi- to thousands of cells per second. FCM is particularly useful in sit-
nating up to 16 cell surface and intracellular molecules (across uations in which the researcher wishes to discriminate between
multiple stains) in 130 to more than a 1000 individuals (3–7). multiple different cell types within a heterogeneous population
For any one of these studies, the time required for FCM analysis and measure their individual frequencies or the expression of
alone was >15 h. a specific molecule of interest (Figure 1). For example, FCM is
In the following review, we will provide an overview of the often used to measure the frequency of two major T-lymphocyte
methods to automate FCM analysis computationally and describe populations in the peripheral blood, T helper cells and cytotoxic
one of the more accessible solutions available to researchers T-cells, which is easily accomplished by distinguishing leukocytes
with little or no experience in computer programing. We urge according to the expression of the cell-surface molecules CD4
any researcher who has conducted or is considering conducting (T helper cell) and CD8 (cytotoxic T-cell) (Figure 1). In many
a large-scale FCM study to evaluate these tools as well as many clinical studies, FCM has proved to be indispensable, allowing for
of the other exquisite techniques that are freely available to the the measurement of distinct, sometimes rare populations of cells
scientific community. that are indicative of the progression of disease (9) or the success
of therapeutic intervention (10).
An Overview of Flow Cytometry
Gating
Invented in the 1960s, and first described in 1972 (8), FCM or One of the most basic principles of FCM analysis is “gating,”
fluorescence-activated cell sorting (FACS), as it was first called, which is the sequential identification and refinement of a cellular
has transformed a number of fields, most of which being cellular population of interest using a panel of molecules (also known as
and clinical immunology. It allows for the quantification of either markers) that are visualized by fluorescence in a unique emission

FIGURE 1 | A brief overview of a flow cytometry experiment identifying the laser beam one at a time. The fluorescent dyes are then excited by the
the proportions of T helper and cytotoxic T-lymphocytes in human laser, and their emitted spectrum is detected by sensors, which digitize the
peripheral blood. Peripheral blood leukocytes are first stained with three information and visualizes it as two-dimensional dot plots, or one-dimensional
antibodies conjugated to unique fluorescent dyes (not represented here), histograms. The levels of CD3, CD4, and CD8 are recorded for every
which will specifically bind the T-lymphocyte markers (antigens) CD3, CD4, or leukocyte that passes through the flow cell, therefore, allowing for the
CD8. This sample of leukocytes is then transferred to the flow cytometer’s flow quantification of frequency of CD3+CD4+ T helper or CD3+CD8+ cytotoxic
cell, which focuses the stream of leukocytes, allowing them to pass through T-lymphocytes.

Frontiers in Immunology | www.frontiersin.org 2 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

spectrum. For example, if a researcher is interested in quantify- The inherent subjectivity of manual analysis should be
ing the proportions of CD4 expressing T helper cells and CD8 considered another major reason for the use of computational
expressing cytotoxic T-cells in peripheral blood, he/she may use assistance for FCM analysis. Since manual analysis requires
a combination of antibodies (which specifically recognize the the user to specify which cells to gate on (often called a “gating
marker of interest) conjugated to unique fluorescent dyes that strategy”), additional technical variation will naturally arise
will accurately identify these cells, while discriminating other due to the inability of humans to accurately reproduce this
cell types that are not of interest. In the example of Figure 2, strategy within and across FCM experiments. The amount of
the cells of interest that are being selected (gated) express CD45 technical variability related to human subjectivity, not surpris-
(a pan-leukocyte marker), CD3 (a marker specific to mainly ingly, is increased if more than one user is tasked to perform
T-lymphocytes), and CD4, or CD8. Cells that are not of interest the FCM analysis, and has been estimated to be as high as 78%
and will be “gated out” also express CD45, but uniquely express (12, 13). Computational assistance not only removes the neces-
CD14 (commonly expressed on monocytes), CD15 (commonly sity for multiple individuals to take part in FCM analysis but
expressed on neutrophils), or CD19 (commonly expressed on also compared to manual gating, many software packages have
B-lymphocytes). Thus, according to the fluorescence of dyes been shown to be vastly superior with regards to the variability
conjugated to antibodies recognizing each marker, the researcher observed due to the gating procedure, even when applied to
will be able to identify T helper cells and cytotoxic T-cells, which experiments performed with heterogeneous protocols and
may also be labeled CD45+CD14−CD15−CD19−CD3+CD4+CD8− reagents (13–15).
and CD45+CD14−CD15−CD19−CD3+CD4−CD8+, respectively. Finally, another great benefit to the use of computational
Additionally, based on the level of fluorescence for a given assistance for FCM analysis is the potential to discover new,
marker, the researcher can also measure the degree to which that biologically relevant cellular populations that were not initially
molecule is being expressed by the cell of interest. considered by the researcher. As previously discussed, the basis
of manual analysis is a gating strategy that allows one to capture
data for particular cellular populations of interest. Computational
Computational Assistance for methods are efficient “discovery” tools because they do not rely
FCM Analysis on any particular gating strategy or guidance from the end-user
to identify cellular populations in FCM experiments. Instead,
Why You Should Consider Using they apply mathematical algorithms that are able to detect trends
Computational Assistance Methods within an entire FCM dataset, which are inferred to be bona fide
One of the three primary reasons that researchers should consider cellular populations. Hence, populations that were not specified
implementing computational assistance for their FCM analysis is in a given gating strategy may be identified by computational
speed. As previously mentioned, the time it takes to manually analysis. That being said, upon discovering a new cellular
analyze an FCM experiment is dependent on the number of population, it is still necessary for researchers to validate their
experimental units to process as well as the number of markers finding biologically. Most computational methods are unable to
(also known as parameters) needed to gate. This can increase the distinguish between true fluorescence due to marker expression
analysis time dramatically if the gating strategy is particularly and artifactual fluorescence due to cellular autofluorescence or
complex. Depending on the particular software package used, the fluorescent spillover (16), and some will simply overestimate
use of computational assistance can reduce FCM analysis time the number of cellular populations without guidance from the
from hours to minutes (11). end-user (17).

FIGURE 2 | An example of a gating strategy used to identify CD4+ T AmCyan, Pacific Blue, Allophycocyanin (APC), APC Cy7, Alexa Fluor 700,
helper and CD8+ cytotoxic T-lymphocytes in human peripheral blood. PerCp Cy5.5, and PE Cy7, respectively, were used to sequentially identify
Antibodies that specifically recognize the cell-surface markers CD45, CD14, T-lymphocytes expressing CD45, CD3, and CD4 or CD8, while excluding cells
CD15, CD3, CD19, CD4, and CD8, and conjugated to the fluorescent dyes expressing CD14 (monocytes), CD15 (neutrophils), or CD19 (B-lymphocytes).

Frontiers in Immunology | www.frontiersin.org 3 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

An Overview of Some of the Procedures Used by over other automated gating algorithms. Some software pack-
Computational Assistance Software Packages ages [for example, flowMeans (20)] attempt to circumvent the
Dozens of freely available software packages have been reported drawback of a priori cluster specification by first computationally
[see Ref. (11, 18)], and can be categorized anywhere from being identifying the maximal number of clusters in a given FCM
completely automated without the need for guidance from the dataset, then iteratively collapsing that number by merging those
end-user, to being partially automated and requiring a great clusters that significantly overlap.
deal of guidance and adjustment (tuning) in order to complete As an alternative to clustering approaches, such as k-means,
its task. The basis of these packages, which allow them to adapt model-based approaches are attractive given that they are robust
to experimental environments that often include a great deal of to the shape of cellular populations and do not require a priori
technical and biological variability (Figure 3), is the algorithms input as does k-means clustering. However, these benefits come
and mathematical procedures upon which they are built. Below is at a computational cost, and therefore, model-based approaches
a brief summary of these approaches, including inherent advan- can be very time consuming (21). The most common approach
tages and disadvantages. is Gaussian, which requires FCM fluorescence data to follow a
According to Bashashati and Brinkman (18), there are five normal distribution, while others, such as t, skew-t, and uniform,
distinct requirements for an automated gating procedure: (1) offer more flexibility in this regard (18, 20). Some software pack-
computational efficiency, (2) the ability to identify a cellular ages use a combinational approach including k-means clustering
population regardless of shape, (3) robustness toward different and model-based algorithms to maximize efficiency and accuracy
antigen/marker densities and expression patterns, (4) the abil- [for example, flowPeaks (22)], while others include hierarchical
ity to determine the true population number accurately, and clustering in their combined approach [for example, Citrus (23)
(5) the ability to detect and account for outliers. All of these and Spade (24)].
requirements essentially surround the capacity of the software One of the most recently published software packages,
to correctly identify clusters or groups of data points, which are flowDensity (25), offers a different approach. Instead of using
assumed to be bona fide cellular populations. While there are clustering or model-based approaches to identify cellular
many approaches to do this, the most common use clustering populations, the software employs a manual gating strategy.
algorithms, with k-means clustering being the most popular, and This approach sacrifices the discovery aspect of automated FCM
model-based algorithms. analysis, but benefits greatly in computational efficiency and the
k-Means clustering is an iterative process in which “k” number ability to accurately measure rare cellular populations, since they
of clusters are defined (often by the user) and the center of each are effectively pre-defined by the end-user. Unlike many other
cluster (initially assigned randomly) is refined until each cluster packages, this software, as well as that of Feher and colleagues
encompasses a unique set of data points; in other words, each (16), can also make use of fluorescent-minus one (FMO) controls.
FCM data point ends up being closest to only one cluster center. An FMO is an important control for manual gating analysis in
Naturally, the clusters tend to center in areas of density, which which a replicate FCM experiment is processed that includes
are assumed to be cellular populations (19). Major limitations all but one of the conjugated antibodies in the staining cocktail.
of conventional k-means clustering with regards to automated Hence, the FMO provides the end-user an empirically derived
FCM analysis is that the user must specify “k,” the number of fluorescence cut-off for the missing antibody’s respective dye.
clusters needed to be identified, and it is restricted to identifying Software packages that incorporate FMO controls are expected
spherically shaped populations (19). However, k-means is also a to offer enhanced accuracy with regards to the measurement of
relatively fast procedure, which provides an important advantage rare cellular populations or those that are defined by markers that

FIGURE 3 | Biological and technical variability between flow cytometry demonstrates the variability in the fluorescent staining patterns that is common in
(FCM) experiments. Four separate experiments, performed on different days FCM analysis. FCM 1–3 represent experiments performed on different donors over
with peripheral blood mononuclear cells from at least two different donors different days. FCM 3 and 4 represent the same donor analyzed on different days.

Frontiers in Immunology | www.frontiersin.org 4 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

present as a continuous expression pattern (i.e., a smear, as in laboratory that relies on FCM for the diagnosis of patient health.
CD28 in Figure 4A, right panels). However, the authors stress that the predictive capacity of these
As alluded to above, the approach of a given software pack- software packages is highly dependent on the nature of classifica-
age can have a major effect on its overall performance. To shed tion. In other words, for diseases in which FCM markers are not
light on this issue in a standardized and unbiased manner, the useful for diagnosis, computational assistance will provide little
flow cytometry: critical assessment of population identification to no benefit.
methods (FlowCap) consortium was formed (11).
Implementation of Automated FCM
Flow Cytometry: Critical Assessment of Analysis for Immunologists
Population Identification Methods
The FlowCap consortium represents leaders in the areas of math- A major roadblock to the widespread implementation of auto-
ematics, statistics, biology, and software development with the mated FCM gating approaches is the perception by the scientific
primary goal to objectively test and compare FCM computational community that a great deal of technical expertise is required
assistance software and provide guidance on the best use of the to operate them (31). While this is true for some software pack-
available algorithms. To this date, the consortium has performed ages, many exist that can be easily and quickly employed by any
two competitions: FlowCap I, in which software packages were researcher with access to the internet.
compared against manual gating under various levels of adjust- Flow cytometry analysis software can be broadly grouped into
ment and tuning by the end-user, and FlowCap II, in which the two areas: those driven by graphical-user interfaces (GUIs), which
ability of the software to define cellular populations that could can be controlled by mouse and simple keyboard commands, and
then be used to stratify biological samples was tested. The results command-line driven modules, which require at least some com-
of these competitions can be found in Aghaeepour et al. (11). puter programing language expertise. The former are commonly
For the FlowCap I competition, 14 software packages were run from online servers or as stand-alone software, while the latter
employed to analyze 5 different FCM datasets submitted by most commonly require proficiency in the programing language
experts in the fields of medicine and immunology, and their results R2, an open-source and free-to-use software environment that
compared against those derived from manual gating. Scores were has applications for a myriad of analyses in physical, biological,
presented using an f-score, where a score of 1 indicates that a and social sciences. For those who are interested in learning R,
particular software package perfectly mimics manual gating. In there are a number of free-to-use, interactive online courses that
the first challenge, software packages were tested without any user can provide most individuals with enough proficiency to operate
guidance. From this, seven packages were observed to have an FCM analysis packages programed in this language; for example,
f-score of 0.80–0.90, the top four of which being ADICyt (com- Datacamp3 or Code School4.
mercially available from Adinis Ltd., Slovakia), flowMeans (20), For those who have not employed computation assistance
FLAME (27), and FLOw Clustering without K (FLOCK) (28). for their FCM analyses, we will briefly introduce FLOCK (28),
Interestingly, manual tuning or adjustment of software analysis a software package that is easily accessible by most researchers.
settings by the end-user only marginally improved the f-score Using a small FCM dataset, we will compare the results from
of each package tested. However, if the number of identified FLOCK using guidance free and tuned settings to manual gating
populations desired by the end-user were used as guidance, the as a frame of reference.
f-scores improved dramatically, with five algorithms ending up
with scores of 0.90 or greater. As before, this not only included FLOw Clustering Without K
ADIcyt, FlowMeans, and FLOCK but also SamSpectral (29) and FLOw Clustering without K was chosen because of its excellent
TCLUST (30). The overall conclusion from this challenge was performance in the FlowCap challenges (11) and represents an
that although some packages performed better than others, most automated FCM analysis package that does not require any com-
performed well and this was largely dependent on the dataset puter programing expertise, instead running from the publically
analyzed. Furthermore, it was also observed that combining the available Immunology Database and Analysis Portal (Immport)
results of multiple software packages (referred to as ensembl (32) from the National Institute of Allergy and Infectious
clustering) consistently performed best for all challenges, as high Diseases (NIAID). Furthermore, FLOCK does not require any
as 0.97, in fact. manual adjustment to operate, although user adjustments are
For the second FlowCap competition, a series of software possible. FLOCK’s approach to identify cellular populations in
packages were tested for their ability to stratify a cohort of indi- an FCM experiment is grid-based partitioning of the data fol-
viduals into different classification groups based on their FCM lowed by density distribution analysis. In density distribution
data. This differs from the first challenge in that the particular analysis, dense clusters of cells are identified and assigned a label
cellular populations identified were not of interest, rather the at their center (known as a centroid), which can then be used to
software’s ability to identify populations that are highly corre- identify those same cellular clusters in other FCM experiments.
lated to a classification group (i.e., diseased or not diseased). The Consequently, it is these two steps that users are able to manually
results from this challenge indicate that most packages are very
good predictors of sample classification, and, in some cases, the 2
http://cran.r-project.org/
software was perfect in its prediction. The results imply that using 3
https://www.datacamp.com
computational assistance will likely be an ideal fit for a clinical 4
tryr.codeschool.com

Frontiers in Immunology | www.frontiersin.org 5 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

FIGURE 4 | A graphical comparison of cellular populations identified by gate. FLOCK analysis was performed in parallel under (B) completely
manual gating and FLOCK. (A) The gating strategy for manual analysis to automated (no guidance) conditions as well as with (C) manual adjustment to
sequentially identify the proportion of CD4 and CD8 expressing CD3+ the analysis parameters (bin = 30, density = 25). Each color represents a
lymphocytes, as well as the frequency of those that also express CD28. Note: different cellular population as determined by FLOCK. Data employed have
the percentages shown represent the frequency of cells relative to the parent been previously published using a different analysis by Lelic et al. (26).

adjust in order to refine their results; first, the number of bins A major advantage of using FLOCK is that even though the
(6–30), which specifies the degree to which the FCM environ- user is able to manually tune the results, no guidance by way
ment is partitioned, and second, the density threshold cut-off of a gating strategy is required. Hence, FLOCK can be very
(6–100), which modulates how the software determines whether useful for identifying cellular populations that the researcher
two adjacent cellular clusters are one in the same or separate enti- was not initially intending to measure. Another major
ties. There is also an automated mode where these two parameters advantage is its ease of implantation. A user can register with
are determined by the algorithm empirically. Many FCM results ImmPort, upload their FCM dataset (as FCS or text format),
can be compared to each other after a canonical set of centroids and perform FLOCK analysis in less than thirty minutes.
is chosen by the user. Furthermore, the time to analyze a single FCM experiment

Frontiers in Immunology | www.frontiersin.org 6 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

file is trivial (minutes, for most experiments). A disadvantage TABLE 1 | Cellular populations identified in our sample FCM dataset by
manual gating and analysis by FLOCK.
of FLOCK, as shown in our example below, is that without
manual tuning, FLOCK has a tendency to overestimate the Manual gating FLOCK (completely FLOCK (manual tuning)
number of cellular populations in a given FCM dataset. This, automated)
of course, is dependent on the particular dataset, which can
CD3+ a CD3−CD4loCD8−CD28lo CD3−CD4loCD8−CD28−
vary greatly with regards to its heterogeneity and the number CD3+CD4+ b CD3−CD4−CD8−CD28lo CD3loCD4loCD8−CD28lo a,b
of parameters measured. CD3+CD4+CD28+ c CD3+CD4+CD8loCD28+ CD3+CD4loCD8−CD28+ a,b,c
CD3+CD8+ d CD3−CD4loCD8−CD28lo CD3−CD4loCD8−CD28lo
Implementing FLOCK for Automated FCM CD3+CD8+CD28+ e CD3loCD4−CD8−CD28lo CD3loCD4−CD8+CD28lo a,d
CD3−CD4+CD8−CD28− CD3+CD4−CD8loCD28+ a,d
Dataset Analysis CD3−CD4+CD8−CD28lo CD3−CD4+CD8−CD28lo
To provide an example of FLOCK analysis, we analyzed a subset CD3−CD4+CD8−CD28lo CD3−CD4+CD8−CD28−
of FCM experiments from a previously published study (26). In CD3+CD4loCD8−CD28+ CD3−CD4+CD8−CD28lo
this study, peripheral blood mononuclear cells from individuals CD3+CD4−CD8loCD28+ CD3+CD4+CD8loCD28+ a,b,c
aged 19–85 years old and acutely infected with West Nile virus CD3+CD4loCD8loCD28+ CD3+CD4loCD8loCD28+ a
CD3−CD4loCD8−CD28lo CD3+CD4−CD8+CD28lo a,d,e
(WNV) and chronically infected with Epstein–Barr virus (EBV) CD3−CD4−CD8−CD28−
were tested for their ability to respond to peptides derived from CD3loCD4loCD8−CD28lo
these viruses. For the current analysis, a subset of 42 individuals CD3+CD4+CD8−CD28+
were randomly chosen and the frequency of CD3+, CD3+CD4+, CD3+CD4−CD8loCD28+
CD3−CD4+CD8−CD28−
CD3+CD8+, CD3+CD4+CD28+, and CD3+CD8+CD28+ lym-
CD3+CD4loCD8loCD28lo
phocytes were measured by manual gating (Figure 4A) and CD3loCD4loCD8−CD28lo
FLOCK, under completely automated (Figure 4B) as well as CD3−CD4+CD8−CD28lo
manually tuned (Figure 4C) settings. Traditionally, the pres- CD3−CD4+CD8−CD28−
ence of these four markers on lymphocytes are discriminated as CD3+CD4loCD8loCD28+
CD3+CD4loCD8loCD28+
either expressed (+) or not expressed (−). Manual gating was
CD3−CD4+CD8−CD28lo
performed using FlowJo, and simple comparison by Spearman’s CD3−CD4loCD8loCD28lo
rank correlation was performed to judge the performance of CD3+CD4loCD8+CD28lo
FLOCK. CD3+CD4loCD8+CD28+
Under completely automated settings FLOCK performed very CD3+CD4−CD8+CD28lo
CD3loCD4−CD8+CD28−
quickly as compared to manual gating (23 min as compared to
CD3loCD4−CD8+CD28lo
~1 h), and as expected, the software also estimated more cellular
populations than expected, 30 (Table 1). It should be noted that Populations in the manual gating and FLOCK (manual tuning) groups with matching
superscripts were combined within group and used to compare analysis methods.
this includes a number of populations that would not be consid- Data employed have been previously published using a different analysis by Lelic
ered in the manual gating strategy (which is designed to measure et al. (26).
only five populations), and includes cells that do not express CD3
or CD28, and those that express neither CD4 nor CD8. However,
in addition to the conventional expression patterns for these cells included those labeled as CD8loCD4− and CD8+CD4−;
five markers, either expressed (+) or not expressed (−), FLOCK and CD28+ cells were either CD28lo or CD28+, and no CD28−
also specifies “low” expressing populations (i.e., CD3lo) for each cells were identified (Table 1). When the frequencies of those
marker. This dramatically increases the number of populations populations were combined into groups matching that in our
identified. Although it is possible that a “low” expression pattern manual gating strategy, and compared to our manual analysis
for CD3, CD4, CD8, or CD28 expressing lymphocytes is biologi- using Spearman’s rank correlation, we found that the results
cally relevant as opposed to an artifact of the FLOCK analysis, it from FLOCK are very similar. The ρ correlation for the CD3+,
is less likely that all 30 cellular populations are biologically and CD3+CD4+, CD3+CD4+CD28+, and CD3+CD8+ populations
functionally distinct. were 0.97, 0.96, 0.93, and 0.92, respectively, and significant at
To reduce the resulting cellular populations identified p < 0.0001 (Table 2). FLOCK estimated a higher frequency of
to a number that is more likely to be biologically relevant, cells in the CD3+CD8+CD28+ population, but was still compa-
we adjusted the analysis parameters for FLOCK. This was rable and significant (ρ = 0.62, p < 0.0001, Table 2).
determined on a fairly arbitrary basis, increasing the “bin” This brief comparison demonstrates the power of using
and “density” values until the number of cellular populations FLOCK, namely the ability to identify cellular populations that
identified were no more than three times what is specified by were not specified in the manual gating strategy, the reduced time
our manual gating strategy. Using a bin value of 30 and a density expenditure, and the high level of comparability to the results
value of 15, FLOCK identified 12 populations, which included determined by manual analysis. The relatively lower correlation
“+,” “−,” and “lo” designations (Table 1). Further inspection of for the CD3+CD8+CD28+ population indicates that a software
these populations indicated that as compared to the design of package may not be equally comparable to manual gating for all
our manual gating strategy, FLOCK perceived the following: populations measured, and suggests that significant tuning of
CD3+ cells included those labeled as CD3+ or CD3lo; CD4+ cells the analysis parameters may be required before a researcher is
included those labeled as CD4loCD8− and CD4+CD8lo; CD8+ confident in the final analysis performed.

Frontiers in Immunology | www.frontiersin.org 7 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

TABLE 2 | Summary statistics and correlations of a paralleled analysis by manual gating and FLOCK (manually tuned).

CD3+ CD4+ CD4+CD28+ CD8+ CD8+CD28+

Manual Flock Manual Flock Manual Flock Manual Flock Manual Flock

Average 84.2 78.1 52.9 50.3 49.6 48.7 21.9 24.0 9.0 16.2
SE 1.4 1.7 2.7 2.2 2.7 2.4 1.6 1.5 0.7 1.1
25th Percentile 79.6 74.7 45.3 42.6 41.8 41.4 14.6 18.1 6.0 10.1
75th Percentile 91.3 85.7 63.5 60.9 59.7 58.5 26.4 27.0 10.8 20.3
Spearman’s ρ* 0.97 0.96 0.93 0.92 0.62

An FCM dataset of peripheral blood mononuclear cells from 42 individuals was analyzed to identify the proportion (relative to the total lymphoid population) of CD4 and CD8
expressing CD3+ lymphocytes, as well as the frequency of those that also express CD28. Data employed have been previously published using a different analysis by Lelic et al. (26).
*Significance of all Spearman’s rank correlations was p < 0.0001.

Conclusion considered cellular populations in a given FCM dataset. We urge

any researcher who is planning or has previously performed
In conclusion, we have provided an overview of automated FCM large FCM experiments to consider implementing computational
analysis as well as its advantages and disadvantages as compared assistance into their analysis pipeline.
to manual gating. There are numerous software packages to
choose from, all of which differing slightly in the benefits they Acknowledgments
can provide to a given FCM dataset, as well as the technical
expertise required to operate them. In our example we describe The authors would like to thank Dr. Patrick Dunn, National
the results from FLOCK, a software package requiring little to no Institute of Allergy and Infectious Diseases, for his technical sup-
computing programing experience, and demonstrate the power port in the implementation of FLOCK. This work is funded by a
of automated gating to improve the time required FCM analysis. CIHR grant to DB. Work in the Bowdish lab is supported in part
Furthermore, our results from FLOCK demonstrate the potential by the M. G. DeGroote Institute for Infectious Disease Research
for computational assistance to discover new, not previously and the McMaster Immunology Research Centre.

References and is associated with worse response to treatment. Br J Haematol (2009)

144(2):223–9. doi:10.1111/j.1365-2141.2008.07457.x
1. Chattopadhyay PK, Roederer M. Cytometry: today’s technology and tomor- 11. Aghaeepour N, Finak G; FlowCAP Consortium, DREAM Consortium, Hoos
row’s horizons. Methods (2012) 57(3):251–8. doi:10.1016/j.ymeth.2012.02.009 H, Mosmann TR, et al. Critical assessment of automated flow cytometry
2. Tanner SD, Baranov VI, Ornatsky OI, Bandura DR, George TC. An intro- data analysis techniques. Nat Methods (2013) 10(3):228–38. doi:10.1038/
duction to mass cytometry: fundamentals and applications. Cancer Immunol nmeth.2365
Immunother (2013) 62(5):955–65. doi:10.1007/s00262-013-1416-8 12. Nomura L, Maino VC, Maecker HT. Standardization and optimization
3. Johnstone J, Millar J, Lelic A, Verschoor CP, Walter SD, Devereaux PJ, et al. of multiparameter intracellular cytokine staining. Cytometry A (2008)
Immunosenescence in the nursing home elderly. BMC Geriatr (2014) 14(1):50. 73(11):984–91. doi:10.1002/cyto.a.20602
doi:10.1186/1471-2318-14-50 13. Levin E, Serrano K, Devine DV; Biomedical Excellence for Safer Transfusion
4. Verschoor CP, Johnstone J, Millar J, Dorrington MG, Habibagahi M, Lelic (BEST) Collaborative. Standardization of CD62P measurement: results of an
A, et al. Blood CD33(+)HLA-DR(-) myeloid-derived suppressor cells are international comparative study. Vox Sang (2013) 105(1):38–46. doi:10.1111/
increased with age and a history of cancer. J Leukoc Biol (2013) 93(4):633–7. vox.12023
doi:10.1189/jlb.0912461 14. Gouttefangeas C, Chan C, Attig S, Køllgaard TT, Rammensee H-G, Stevanović
5. Verschoor CP, Johnstone J, Millar J, Parsons R, Lelic A, Loeb M, et al. S, et al. Data analysis as a source of variability of the HLA-peptide multimer
Alterations to the frequency and function of peripheral blood monocytes and assay: from manual gating to automated recognition of cell clusters. Cancer
associations with chronic disease in the advanced-age, frail elderly. PLoS One Immunol Immunother (2015) 64(5):585–98. doi:10.1007/s00262-014-1649-1
(2014) 9(8):e104522. doi:10.1371/journal.pone.0104522 15. Chen X, Hasan M, Libri V, Urrutia A, Beitz B, Rouilly V, et al. Automated
6. Verschoor CP, Loukov D, Naidoo A, Puchta A, Johnstone J, Millar J, et al. flow cytometric analysis across large numbers of samples and cell types. Clin
Circulating TNF and mitochondrial DNA are major determinants of neu- Immunol (2015) 157(2):249–60. doi:10.1016/j.clim.2014.12.009
trophil phenotype in the advanced-age, frail elderly. Mol Immunol (2015) 16. Feher K, Kirsch J, Radbruch A, Chang H-D, Kaiser T. Cell population iden-
65(1):148–56. doi:10.1016/j.molimm.2015.01.015 tification using fluorescence-minus-one controls with a one-class classifying
7. Johnstone J, Parsons R, Botelho F, Millar J, McNeil S, Fulop T, et al. Immune algorithm. Bioinformatics (2014) 30(23):3372–8. doi:10.1093/bioinformatics/
biomarkers predictive of respiratory viral infection in elderly nursing btu575
home residents. PLoS One (2014) 9(9):e108481. doi:10.1371/journal.pone. 17. Finak G, Bashashati A, Brinkman R, Gottardo R. Merging mixture compo-
0108481 nents for cell population identification in flow cytometry. Adv Bioinformatics
8. Bonner WA, Hulett HR, Sweet RG, Herzenberg LA. Fluorescence activated cell (2009). doi:10.1155/2009/247646 Available from: http://www.ncbi.nlm.nih.
sorting. Rev Sci Instrum (1972) 43(3):404–9. doi:10.1063/1.1685647 gov/pmc/articles/PMC2798116/
9. Nunes C, Wong R, Mason M, Fegan C, Man S, Pepper C. Expansion of a 18. Bashashati A, Brinkman RR. A survey of flow cytometry data analysis meth-
CD8(+)PD-1(+) replicative senescence phenotype in early stage CLL patients ods. Adv Bioinformatics (2009):584603. doi:10.1155/2009/584603
is associated with inverted CD4:CD8 ratios and disease progression. Clin 19. Luta G. On extensions of k-means clustering for automated gating of flow
Cancer Res (2012) 18(3):678–87. doi:10.1158/1078-0432.CCR-11-2630 cytometry data. Cytometry A (2011) 79(1):3–5. doi:10.1002/cyto.a.20988
10. Dalmazzo LFF, Jácomo RH, Marinato AF, Figueiredo-Pontes LL, Cunha 20. Aghaeepour N, Nikolic R, Hoos HH, Brinkman RR. Rapid cell population
RLG, Garcia AB, et al. The presence of CD56/CD16 in T-cell acute lym- identification in flow cytometry data. Cytometry A (2011) 79(1):6–13.
phoblastic leukaemia correlates with the expression of cytotoxic molecules doi:10.1002/cyto.a.21007

Frontiers in Immunology | www.frontiersin.org 8 July 2015 | Volume 6 | Article 380

Verschoor et al. Automated FCM analysis

21. O’Neill K, Aghaeepour N, Spidlen J, Brinkman R. Flow cytometry bioin- identification of cell populations in multidimensional flow cytometry data.
formatics. PLoS Comput Biol (2013) 9(12):e1003365. doi:10.1371/journal. Cytometry B Clin Cytom (2010) 78(Suppl 1):S69–82. doi:10.1002/cyto.b.20554
pcbi.1003365 29. Zare H, Shooshtari P, Gupta A, Brinkman RR. Data reduction for spectral clus-
22. Ge Y, Sealfon SC. flowPeaks: a fast unsupervised clustering for flow cytom- tering to analyze high throughput flow cytometry data. BMC Bioinformatics
etry data via K-means and density peak finding. Bioinformatics (2012) (2010) 11:403. doi:10.1186/1471-2105-11-403
28(15):2052–8. doi:10.1093/bioinformatics/bts300 30. Dost B, Wu C, Su A, Bafna V. TCLUST: a fast method for clustering genome-
23. Bruggner RV, Bodenmiller B, Dill DL, Tibshirani RJ, Nolan GP. Automated scale expression data. IEEE/ACM Trans Comput Biol Bioinform (2011)
identification of stratifying signatures in cellular subpopulations. Proc Natl 8(3):808–18. doi:10.1109/TCBB.2010.34
Acad Sci U S A (2014) 111(26):E2770–7. doi:10.1073/pnas.1408792111 31. Finak G, Frelinger J, Jiang W, Newell EW, Ramey J, Davis MM, et al. OpenCyto:
24. Qiu P, Simonds EF, Bendall SC, Gibbs KD, Bruggner RV, Linderman MD, et al. an open source infrastructure for scalable, robust, reproducible, and auto-
Extracting a cellular hierarchy from high-dimensional cytometry data with mated, end-to-end flow cytometry data analysis. PLoS Comput Biol (2014)
SPADE. Nat Biotechnol (2011) 29(10):886–91. doi:10.1038/nbt.1991 10(8):e1003806. doi:10.1371/journal.pcbi.1003806
25. Malek M, Taghiyar MJ, Chong L, Finak G, Gottardo R, Brinkman RR. 32. Bhattacharya S, Andorf S, Gomes L, Dunn P, Schaefer H, Pontius J, et al.
flowDensity: reproducing manual gating of flow cytometry data by auto- ImmPort: disseminating data to the public for the future of immunology.
mated density-based cell population identification. Bioinformatics (2015) Immunol Res (2014) 58(2–3):234–9. doi:10.1007/s12026-014-8516-1
31(4):606–7. doi:10.1093/bioinformatics/btu677
26. Lelic A, Verschoor CP, Ventresca M, Parsons R, Evelegh C, Bowdish D, et al. Conflict of Interest Statement: The authors declare that the research was con-
The polyfunctionality of human memory CD8+ T cells elicited by acute ducted in the absence of any commercial or financial relationships that could be
and chronic virus infections is not influenced by age. PLoS Pathog (2012) construed as a potential conflict of interest.
8(12):e1003076. doi:10.1371/journal.ppat.1003076
27. Pyne S, Hu X, Wang K, Rossin E, Lin T-I, Maier LM, et al. Automated high-­ Copyright © 2015 Verschoor, Lelic, Bramson and Bowdish. This is an open-access
dimensional flow cytometric data analysis. Proc Natl Acad Sci U S A (2009) article distributed under the terms of the Creative Commons Attribution License (CC
106(21):8519–24. doi:10.1073/pnas.0903028106 BY). The use, distribution or reproduction in other forums is permitted, provided the
28. Qian Y, Wei C, Eun-Hyung Lee F, Campbell J, Halliley J, Lee JA, et al. original author(s) or licensor are credited and that the original publication in this
Elucidation of seventeen human peripheral blood B-cell subsets and quantifi- journal is cited, in accordance with accepted academic practice. No use, distribution
cation of the tetanus response using a density-based method for the automated or reproduction is permitted which does not comply with these terms.

Frontiers in Immunology | www.frontiersin.org 9 July 2015 | Volume 6 | Article 380

View publication stats