Study design and data generation
We designed and conducted a comparative study to answer the question whether gene expression data generated with Affymetrix U133A microarrays, four years past their expiration date, are useful and reliable. Figure depicts the overview of the study, the experimental procedures, data used, and comparative analyses applied. Two sources of data were used in this study: gene expression data newly generated in 2009 specifically for this study and data generated by the MAQC project in 2005.
Figure 1 The workflow of study design and data analysis New gene expression data were generated in 2009 with expired U133A microarrays and unexpired U133Plus2 microarrays using the same MAQC reference RNA samples and compared to the microarray and TaqMan® (more ...)
The new gene expression data were generated with 12 microarrays (2 types of microarrays × 2 samples × 3 replicates) (Table ). Three replicates for each of the two MAQC samples (A and B) were profiled in 2009 by using both the expired U133A microarrays (expired in 2004) and the unexpired U133Plus2 microarrays. In addition, gene expression data generated with unexpired U133Plus2 microarrays (AFX), other microarray platforms, and TaqMan® assays by the MAQC project in 2005 were used as references to assess the stability of the MAQC samples stored at -80°C for four years by comparing new microarray data with those obtained four years ago. The MAQC reference data also allowed for further evaluation of the usefulness of the data generated with expired U133A microarrays.
New data generated for this study with Affymetrix GeneChip® microarrays
It should be pointed out that in the current study the observed differences between the expired and unexpired Affymetrix microarrays were confounded with the use of two different types of GeneChip®
microarrays, U133A (expired four years ago) and U133Plus2 (unexpired). However, the probe design (probe length and sequence identity) for these two types of GeneChip®
microarrays is identical and the consistency of data between the two types of microarrays was demonstrated by the manufacturer (http://media.affymetrix.com/support/technical/technotes/hgu133_p2_technote.pdf
). Therefore, the observed difference between expired U133A and unexpired U133Plus2 microarrays can be attributed mainly to the expiration of the former.
Stability of the two MAQC samples over a period of four years
The two MAQC human reference RNA samples, from the same batches as used in the MAQC project in 2005 but stored at -80°C for over four years, were labeled and hybridized in 2009 on expired U133A and unexpired U133Plus2 microarrays. Before comparing data from the expired and unexpired microarrays, it is necessary to verify the stability of the MAQC samples through assessing the consistency between gene expression data generated in 2005 and in 2009 using the same type of unexpired Affymetrix U133Plus2 microarrays.
Figure shows the correlation of log2 fold changes observed in 2009 and in 2005 with unexpired U133Plus2 microarrays. The correlations are shown in Figures , , and for the 8,550 genes commonly probed by multiple microarray platforms, the 7,069 genes with a p < 0.05 in either of the two data sets (2009 and 2005), and the 5,880 genes with a p < 0.05 in both data sets, respectively. It can be seen from Figure that most genes showed similar log2 fold changes. Under the three gene selection scenarios (a, b, and c) with increasing stringency, the overlap of DEGs between 2009 and 2005 was 91.37%, 97.44%, and 99.78%, respectively, suggesting a reasonably high degree of stability of the two MAQC samples over a period of four years. Thirteen genes showed opposite regulation directionalities between the data sets generated in 2005 and in 2009 (Figure ). It should be pointed out that DEG concordance is only a surrogate of sample stability and more direct measurement of sample stability is warranted in future studies.
Figure 2 Comparisons of the log2 fold changes detected in 2009 and in 2005 using the same type of U133Plus2 microarrays (a) all 8,550 common genes; (b) the 7,069 genes with p < 0.05 in either 2009 or 2005; and (c) the 5,880 genes with p < 0.05 (more ...)
Repeatability of intensity data among sample replicates on expired microarrays
The probe-level raw intensity data were first summarized and normalized with the RMA algorithm [17
] and then transformed to log2
scale. Figures and show the correlation of the log2
gene expression intensities for the 8,550 common genes between replicates on the expired U133A and unexpired U133Plus2 microarrays, respectively. For the expired U133A microarrays (Figure ), the replicates of the same sample showed a high level of correlation similar to what was observed for the unexpired U133Plus2 microarrays (Figure ). More quantitatively, the average correlation coefficients of the log2
intensities among the replicates of sample A were 0.994 and 0.998 for the expired and unexpired microarrays, respectively. For sample B, the corresponding average correlation coefficients were 0.995 and 0.997 from the expired and unexpired microarrays, respectively (Table ). These results demonstrated a high level of intra-sample repeatability of absolute gene expression data from expired U133A microarrays.
Figure 3 Correlations of log2 intensities generated in 2009 among the replicates of samples A and B (a) expired U133A microarrays and (b) unexpired U133Plus2 microarrays. Each scatterplot represents the comparison of log2 intensities of 8,550 common genes from (more ...)
Comparison of log2 fold changes detected with expired U133A and unexpired U133Plus2 microarrays
To address the issue of using expired microarrays, the log2
fold changes of data obtained with the expired U133A microarrays and the unexpired U133Plus2 microarrays were compared (Figure ). The log2
fold changes for most genes were consistent in the direction of regulation (down or up). However, there was a slight fold change compression [18
] for the data obtained with the expired U133A microarrays. That is, the magnitude of differential expression (fold change) for the same gene measured from the expired U133A microarrays (the X axis) was lower than that from the unexpired U133Plus2 microarrays (the Y axis).
Figure 4 Comparisons of log2 fold changes detected with expired U133A and unexpired U133Plus2 microarrays (a) all 8,550 common genes; (b) the 6,835 genes with p < 0.05 in either expired U133A microarrays (2009) or unexpired U133Plus2 microarrays (2009); (more ...)
The percentage of overlapping genes with the same directional change in expression was 89.94% (7,690 out of 8,550 common genes) when no p-value cutoff was applied to either experiment (Figure ). The overlap increased to 96.99% (6,629 out of 6,835 genes) when genes were retained for comparison as long as a p < 0.05 criterion was met in either one of the two experiments (Figure ), a fair scenario for comparing two data sets. When the comparison was restricted to the subset of 5,120 genes that were simultaneously detected to be significantly differentially expressed (p < 0.05) in both experiments, the overlap increased to 99.98%. That is, only one gene showed opposite regulation directionalities between expired U133A microarrays and unexpired U133Plus2 microarrays. These results suggested that the DEGs detected by the expired U133A microarrays were highly consistent with those by the unexpired U133Plus2 microarrays.
Comparison between expired U133A microarrays with multiple microarray platforms used in the MAQC project
Microarray data generated in 2005 with the same MAQC samples from five platforms, Applied Biosystems (ABI), Affymetrix U133Plus2 (AFX), Agilent (AG1), Illumina (ILM), and GE Healthcare (GEH) by the MAQC project [15
] were used as references to further assess the usefulness of the expired U133A microarrays. To assess the concordance between two experiments, we first used a p
< 0.05 cutoff to eliminate genes separately in each experiment and then rank-ordered the remaining genes by log2
FCs. In one experiment, genes ranked higher in either the up- or down-regulation direction are selected as DEGs first. Figure shows the relationship between the POG and the number of selected DEGs, wherein genes with smaller FCs are selected when more genes are chosen as DEGs. The red line represented the average POG between DEG lists of each of the four reference microarray data sets in the MAQC project (ABI, AG1, ILM, and GEH) and the reference Affymetrix U133Plus2 (AFX) data, all generated in 2005. This reference line and its confidence limits can be used to judge the statistical significance between any two experiments, e.g
., between expired U133A microarrays and unexpired U133Plus2 microarrays.
Figure 5 Concordances of different microarrays in terms of percentage of overlapping genes (POG) The x-axis represents the number of genes (2L) selected as differentially expressed, and the y-axis represents the overlap (%) of two gene lists selected from the (more ...)
The concordance between DEG lists selected from expired U133A microarrays and unexpired U133Plus2 microarrays (2009 or 2005) were about 70% when the number of selected DEGs was as few as 10. These POG numbers increased to near 90% when more genes were selected as differentially expressed. For the same number of DEGs, the POGs between expired U133A (2009) and unexpired U133Plus2 (2009) microarrays, and between expired U133A (2009) and the reference U133Plus2 (AFX in 2005) microarrays were on the upper confidence limit of the reference line. This means that compared to data from the reference AFX microarrays (2005), the DEGs generated with expired U133A microarrays (2009) are more consistent than those from other unexpired, but different microarray platforms (ABI, AG1, ILM, and GEH) in 2005. In addition, the POG between the unexpired U133Plus2 (2009) and the reference AFX microarrays (2005) was the highest, again, suggesting that the MAQC samples were relatively stable.
Comparison between expired U133A microarrays and TaqMan® assays
assays have been widely used in gene expression profiling and are generally considered as a good reference for evaluating other gene expression techniques. In this study, the TaqMan®
gene expression data generated by the MAQC project in 2005 using the same MAQC samples [15
] were used to further assess the usefulness of the expired U133A microarrays based on data from the 813 genes that were probed by both the microarrays and TaqMan®
assays. The DEG lists obtained from expired U133A microarrays (2009), unexpired U133Plus2 microarrays (2009), and microarrays from ABI, AFX, AG1, ILM, and GEH (2005) were separately compared with those from the TaqMan®
assays performed in 2005 [19
]. The mean POG of the DEG lists between the TaqMan®
assays and the microarray platforms (ABI, AFX, AG1, ILM, and GEH) was shown as a reference line (the red crosses in Figure ). The POG between the expired U133A microarrays (2009) and the TaqMan®
assays (pink diamonds) was close to the reference line when the number of DEGs was as low as 40 but was about 10% lower than the reference line when all genes meeting the p
< 0.05 criterion were selected (a drop from ~77% to ~70%). In addition, the POG between unexpired U133Plus2 microarrays (2009) and TaqMan®
assays (black squares) and the POG between the AFX (2005) and TaqMan®
assays (sky blue triangles) were higher than, or at the same level as, the reference line. That is, compared to unexpired microarrays, expired U133A microarrays exhibited a slightly decreased ability to detect differential gene expression, mainly due to the fold change compression that made the rankings of genes with smaller fold changes more variable.
Figure 6 Concordance between TaqMan® assays and microarray data in terms of percentage of overlapping genes (POG) The x-axis represents the number of genes (2L) selected as differentially expressed, and the y-axis represents the overlap (%) of two gene (more ...)
Characteristics of gene expression data measured on unexpired U133Plus2 and expired U133A microarrays
The lower sensitivity of the expired U133A microarrays in detecting differential gene expression could be explained by a decrease in microarray signal or an increase in the scaling factor (Table ). The average scaling factor was 4.61 for the expired U133A microarrays and 2.83 for the unexpired U133Plus2 microarrays. The higher scaling factor for the expired microarrays indicates that compared to the unexpired microarrays, the quality of hybridization signals from the expired microarrays was somewhat compromised, leading to a decrease in sensitivity.
Figure shows the distributions of log2 intensities for 8,550 common genes when the MAQC samples were analyzed with the expired U133A and the unexpired U133Plus2 microarrays. Compared to the U133Plus2 microarrays, which showed a mean log2 intensity of 7.79 for sample A (Figure ) and 7.67 for sample B (Figure ), the expired U133A microarrays exhibited a significant decrease in mean log2 intensity, i.e., 6.27 for sample A (Figure ) and 6.29 for sample B (Figure ). The standard deviations of log2 intensity for unexpired U133Plus2 microarrays (2.32 for sample A and 2.30 for sample B) are also higher than those for expired U133A microarrays (1.93 for sample A and 1.95 for sample B), indicating a decrease in the ability of expired microarrays to distinguish differences of expression levels among genes in the same sample. In addition, as shown in Figure , there is a significant difference in the percentage of Present calls between unexpired U133Plus2 (73.6%) and expired U133A (64.5%) microarrays, also suggesting a significant loss of detection signals for the expired U133A microarrays.
Figure 7 Distribution of the log2 intensities of 8,550 common genes Sample A hybridized on expired U133A (a) and unexpired U133Plus2 (b) microarrays; Sample B hybridized on expired U133A (c) and unexpired U133Plus2 (d) microarrays. For each gene, the log2 intensities (more ...)
Figure 8 Percentage of Present calls based on 8,550 common genes There is a significant difference in the percentage of Present calls between unexpired U133Plus2 microarrays (73.6%) and expired U133A microarrays (64.5%). Based on microarray data generated in 2009. (more ...)