To evaluate statistical procedures for TAG microarray data, we tailored defined spike-in pools to resemble expected results in a typical synthetic lethality analyzed by microarray (SLAM) experiment (6
). Synthetic lethality is defined as inviability of cells containing two mutations which are individually not required for growth. In a SLAM experiment, viable YKOs in pooled form are compared under two conditions: absence versus presence of a specific second mutation (the ‘query’). The average number of genetic interactions expected in a genome-wide screen has been estimated to be ~35, although several query mutations with interactions exceeding 100 have been analyzed (10
). Therefore, we designed a pair of pools (‘A’ and ‘B’) with 5758 YKOs at equivalent representation, and a set of 94 YKOs with known differential representation ranging from 1:2⅓
and 1:infinity ( and Supplementary Table 1). Additionally, certain YKOs grow slowly, and representation of these in the control SLAM sample is expected to be lower than YKOs with wild-type growth rate. To examine our ability to address these mutants, we designed three representation levels in the control (B) pool: high (about equal to all other strains), medium (8-fold dilute) and low (64-fold dilute). TAGs from pools A and B were amplified with Cy3- and Cy5-labeled primers, respectively. These samples were mixed at equal ratio, such that most TAGs should exhibit equal hybridization, while Cy5:Cy3 ratios that deviate from one are expected for the few differentially represented TAGs. This design allows discovery of the best method to produce a measure of differential representation from hybridization results.
Before addressing differential representation, we document the utility of two filtering steps in data pre-processing. First, we noted TAG-specific hybridization artifacts evident in self-self hybridizations performed to examine the noise distribution. Pool A DNA served as template for preparation of labeled TAGs with both Cy5 and Cy3. Thus, all TAGs were present at equal amounts between channels. shows a scatter-plot of normalized log2 Cy5:Cy3 ratios for corresponding UPTAG and DNTAG probes. Because all these values should be zero plus measurement error, we expect these to be uncorrelated and centered at zero. confirms this except for a few YKOs with extreme values for one TAG. These outliers may have a negative impact on specificity. They are likely to be due to individual tag templates that enter the labeling PCR as contaminants, which are detectable even at very low levels (17
Figure 2 Development of TAG filters. (a) Self-self hybridizations. Pool A gDNA was used as template for TAG labeling reactions with each primer set (UP/DNTAG Cy5/Cy3). Median values across three experiments were displayed. Each point represents a single YKO. Red (more ...)
We determined that these artifacts are consistent across experiments performed with a single batch of labeled primer, but not between different primer batches (Supplementary Figure 1). To create a useful filter, we assumed that the data follow a bivariate normal distribution and defined outliers as TAGs with log ratios three SDs away from zero, using a robust estimate of the SD. If the log ratio data follow a normal distribution, excluding outliers, we expect to inappropriately remove only ~0.5% of the data (32 TAGs). We applied this filter (, red lines) independently to UP- and DNTAGs. Fortunately, the YKOs were designed with two TAGs per gene (except for 192 strains lacking DNTAGs), greatly improving chances that at least one TAG performs adequately. Because non-outlier UP/DNTAGs appear to provide independent measurements, the chance of inappropriately removing both TAGs for the same YKO is less than 0.00001. Using this procedure we defined 193 DNTAGs (purple circles) and 244 UPTAGs (blue circles) as primer-batch specific outliers. Six YKOs had both UP and DN ratios filtered (orange circles).
Next, we considered the effect of TAG-specific hybridization behavior resulting from the presence of nucleotide mutations found in some of the TAGs and universal priming sites (20
). This is important because sensitivity will be markedly affected when TAGs fail to provide a meaningful measure of strain representation. Histograms of log2 signal intensity display a bimodal distribution ( and data not shown) for UP- and DNTAGs whether Cy3 or Cy5 labeling is used. The lower peak is close to background intensities and contains non-functional TAGs with absent or inefficient hybridization. While TAG sequence discrepancies have been characterized (20
), knowledge of the presence and nature of mutations was insufficient to fully predict hybridization behavior (17
The naïve approach to summarizing UP- and DNTAG information is to average their observed log ratios. This solution will yield suboptimal measures when one of the TAGs is non-functional. We propose a procedure exploiting the bimodal distribution of TAG intensities to improve on simple averaging. To determine if a TAG is non-functional we fit a mixture model, as in Irizarry et al
), to the log intensity data for the control sample. The model fits two normal distributions to the Cy5 data, one for the lower mode and one for the upper mode. The ‘blank’ (YQL) features (17
) define the location and width (mean and SD) of the lower distribution. With this fitted model in place, we can predict the probability that each TAG is ‘present’ (). We consider a DN/UPTAG non-functional when it is predicted absent while the complementary UP/DNTAG is present. We define a weighted average = w
UP + (1 − w
DN, where w
= 0.5 + [P
(UP present) − P
(DN present)]/2. Thus, when UP is present (PUP
= 1) and DN is absent (PDN
= 0), w
= 1 and only UPTAG is used (). We describe a less complex procedure in Supplementary Note 1 that uses binary absent or present values (P
= 0 or 1) and performs similarly (data not shown). Researchers using unsophisticated analysis software such as spreadsheet applications may prefer the simple procedure.
We compared the performance of these two strategies and use of UP- or DNTAGs alone with Receiver Operating Characteristic (ROC) curves based on the spike-in experiment. For this analysis, nominal ratios below 2-fold were excluded from the list of True Positives. This choice is appropriate because 2-fold representation difference corresponds to a subtle growth defect at the margin of detection in colony measurement (1.25-fold colony diameter difference is predicted by hemispherical colony volume = 2 πr3/3). Supplementary Figures 2–3 present ROC curves with varying stringencies for inclusion as ‘True Positive’, including every spiked-in YKO (1.26-fold or higher). ROC curves in the range of false positives likely to be acceptable ( and Supplementary Figure 2) demonstrate that the artifact filtering process has a significant effect on specificity (Supplementary Figure 4 shows the full ROC curves). Additional filtering of non-functional TAGs by the weighted average improves results further ( and Supplementary Figure 3).
Figure 3 Measures of sensitivity/specificity. (a and b) ROC for several methods of calculating relative YKO representation. True Positive is defined as any YKO with known pool B:A ratio ≥ 2. False Positive is defined as any YKO with known B:A ratio of (more ...)
The effect of two filters, one which removes the systematic artifacts and a second which removes non-hybridizing TAGs, is demonstrated by ratio-intensity plots. A naïve approach to analysis would average UP and DN log ratio to produce a measure M for relative strain representation. By filtering systematic artifacts, noise is significantly reduced (, open circles and Supplementary Figure 5). Additionally, combining UP and DN selectively provides increased sensitivity for a number of spiked-in strains (filled shapes).