FlowCAP. Comparison of Automated and Manual Gating of Standardized Lyoplate Flow Cytometry Data

Why Standardize Assays?

Lyoplates and the FOCIS / HIPC effort.

Brief history of FlowCAP.

FlowCAP III - Automated gating of standardized Lyoplate-based flow cytometry data.

OpenCyto Framework to make the FlowCAP algorithms avaialble and easy to use.

Flow is a complex assay - many potential sources of variability.
Sample collection, preparation, staining, acquisition, instrumentation, data analysis.
Need to control as many aspects of the assay as possible.

Proposed in 2010 at FITMan meeting.
Implemented as standardized Lyoplates (96 well plates, lyophilized reagents).

Figure Courtesy of H. Maecker

The panels leave room for custom markers to identify additional populations of interest.

• Different people performing gating produce different results.
  
• also compensation, transformation, tools and so forth.

FLowCAP: Critical Assessment of Cell Population Identification Methods

Three-year old series of workshops for benchmarking automated gating methods.
Aiming for objective comparison of automated gating algorithms.

Identify Gating Methods with low variability and bias relative to centralized manual gating

• FlowCAP III Lyoplate challenge focused on the T-cell and B-cell panels.
• 9 sites, 4 replicates of cryopreserved cells per site.
• Centralized gating of data based on a consensus "best" approach amongst the 9 centers.
• Automated algorithms were compared against the centralized gating.

In clinical trials, the things we want to measure are well defined a-priori.

• Flow assays are well defined.
• Cell populations of interest are well defined.
• No immediate need to apply high-dimensional gating for "discovery".
  • Such approaches are suitable for analysis of secondary endpoints.

Large data sets.

• Gating is tedious and subject to human error (this has been shown over and over).
• We want to automate the repetitive tasks. Make them:
  • Robust and Reproducible

DENSE ( A. Brandes, Broad Institute )

flowDensity ( J. Taghiyar, BC Cancer Agency )

OpenCyto ( J. Ramey, FHCRC )

emcytom ( K. Wang, University of Queensland )

FLOCK ( R. Stanton, JCVI )

Centralized Gating ( Current best practice )

Assess automated methods relative to central manual gating.

• Variability
  • Coefficients of variation across centers
• Bias: \(RMSD_{gpc} = \sqrt{\frac{\sum(y_{gcpr}-\mu_{mpc})}{R}}\)
• Mixed Effects Model: \(y_{gpcr}=\mu+\phi_{p}+\color{red}{\gamma_{g}}+\color{red}{\phi\gamma_{pg}}+(\phi\chi)_{pc}+\epsilon_{gpcr}\)
  • Fixed gating and cell population effects.
  • Random center \(\times\) cell population effects.
  • Interested in interaction and contrasts of fixed effects. \((\gamma_g + \phi\gamma_pg -\gamma_0 - \phi\gamma_{p0} = 0)\)
    An ideal automated gating method will have low bias and low variability for each population.

flowWorkspace: Import Manually Gated Data from flowJo

http://www.github.com/RGLab/flowWorkspace (Bioconductor)
Reproduce FlowJo gating in R from an exported workspace.

ws<-openWorkspace("./Data/Centralized T-cell.xml");
G<-parseWorkspace(ws);
plotGate(G[[1]]); #Plot all B-cell manual gates

Cross center variability of automated gating methods is comparable to centralized gating.

At least one method per panel matches the variability of centralized gating for all populations.

Three methods provided unbiased cell population estimates for the B-cell panel.

Three methods were mostly unbiased, having difficulty with some rare or poorly resolved cell populations.

Goal: Make the best algorithms available for use.

http://www.github.com/RGLab/openCyto

Integrates core R flow infrastructure with automated gating algorithms
(Bayesian flowClust, flowCore, flowDensity, DENSE)

• Modular architecture: plug-in any gating algorithms.
• High-level description of gating schemes
  • User defines hierarchy of cell populations and relevant markers
  • Higher-dimensional gating (e.g. >2D) is available.
    

Framework abstracts away most of the R-coding.

Example CSV Gating Template Definition (Lyoplate B-cell Panel)

R Code to Run the Gating

template<-gatingTemplate("bcellTemplate.csv")
fs<-readFlowSet(file="Data/Bcells/")
gs<-GatingSet(fs)
G<-gating(template,gs)

Makes complex algorithms easy to use.

There are automated gating algorithms that are sufficiently robust to be useful for data analysis today.

• DENSE (Broad Institute), flowDensity (BCCA), OpenCyto (FHCRC)

The tools are available to make them practical to use for your data.

Get these slides online: http://www.github.com/gfinak/Talks/LyoplateFlowCAP3