This study was carried out to compare cDNA (Incyte), long oligonucleotide (Operon 70-mer), and short 25-mer (Affymetrix) array platforms, with the goal of qualifying the 70-mer arrays for general use at the National Cancer Institute. More specifically, we compared the Incyte Unigem2 set of human cDNAs (~9900 genes), the Operon human Version 2.0 set of long oligonucleotides (~21,329 genes), and Affymetrix HG-U133A arrays (~22,200 genes). RNA preparations from cell lines MCF10A, LNCaP, Jurkat, L428, SUDHL6, and OCI-Ly3 were used as probe templates, and the expression of each gene was compared directly with that of the same gene in the Human Universal Reference (HUR) RNA from Stratagene.
Genes in common across platforms
Only genes common to all three platforms were used in the comparison. Genes were matched by UniGene Cluster (UniGene Build #161), and unique cluster memberships were determined for each array type, as listed in Table and enumerated in the Venn diagram in Figure . The intersection for the three platforms consisted of 6430 UniGene clusters, and all analyses were carried out with all of these genes or a subset of them.
Overlapping gene sets represented in 3 microarray platforms
Venn diagram with number of genes present in each platform, genes in common between platforms, and genes in common among all three platforms.
Comparison of expression ratios
An estimate of the concordance of the platforms was provided by the percentage of genes that showed expression ratios "in the same direction." All matched gene sets that had expression ratios (relative to the Universal Standard) greater than 1.0, irrespective of magnitude, were considered to be in the same direction, and the same for expression ratios less than 1.0. As seen in Table , the agreement was reasonably good, ranging from 74% to 82%. For comparison, ~50% concordance would be expected by chance. Many of the discordant values were at what might be considered "trivial" levels because a large proportion of ratios are near unity. For example, approximately 48%, 28% and 13% of the genes from Incyte, Operon and Affymetrix respectively, exhibit expression ratios within the range of 1.1 to 0.9. Thus, a ratio difference of only 0.95 versus 1.05 is designated a mismatch, which normally would be considered very concordant in the biological sense.
Proportion of genes expressed in same direction between platforms ignoring expression level. The numbers of genes in matched and mismatched directions are given for each cell line and each platform pair, and agreement is given in percentages.
Statistical tests and thresholds on fold change are commonly used for the identification of altered gene expression by microarray assays. Hence, we estimated the statistical significance of differential expression of each RNA from the HUR by one sample Student-t test for a change from ratio of 1. Comparison between platforms using expression values significant at p < 0.05 shows concordance in direction of 93%–98% (Table – no threshold). At the 1.5 and 2-fold threshold level, the agreement between platforms approaches 100% indicating that for reproducible data all three platforms are highly concordant in direction.
Concordance between platforms using statistically significant expression ratios at p-value < 0.05 and at 1.5 and 2-fold threshold levels.1
Correlation between platforms
The scatter plots shown in Figures indicate graphically the correlation between the platforms for Jurkat RNA. Values of r for all platforms and all cell lines are given in Table . The values ranged from 0.7 to 0.8. Operon and Affymetrix were better correlated than were Operon and Incyte. The values were intermediate for the relationship between Incyte and Affymetrix. Figures and Table indicate that the correlation coefficients between replicates within the same platform are closer, as expected, when compared to the correlations between platforms. In general, however, there was reasonably good concordance among the platforms.
Figure 2 a-f. Scatter plot analysis to determine correlation coefficients between and within platforms using Jurkat RNA as an example. Correlations for all cell lines are given in Table . (a) Operon versus Incyte (b) Affymetrix versus Incyte (c) (more ...)
Cross platform correlation coefficients. Pearson correlation coefficients are given for each platform pair and cell line, calculated over the genes common to all platforms. Values were obtained from scatter plots as shown in Figure 2a-c.
Median correlation coefficients of replicates within same platform. Calculations and values obtained as in Table 4.
Principal Component Analysis (PCA)
A projection on the first three principal components, which together explain 48.8 % (21%, 14%, and 13%) of the total variance, is shown in Figure . Close clustering of the cell samples is observed in this projection, indicating appreciable agreement among array platforms.
Principal Component Analysis (PCA) of the three microarray platforms and six cell lines using expression of the 3186 genes with signals above background.
"Correlation of Correlations" analysis
The global concordance of the three platforms across all of the cell lines was estimated by calculating the "correlation of correlations" coefficient [10
]. As seen in Figure , the correlations for the three platforms across all cells lines were quite good. The Pearson correlation of correlation coefficients was 0.965 between Operon and Incyte, 0.995 between Affymetrix and Incyte, and 0.956 between Operon and Affymetrix.
a-c. Correlation of correlations of platforms for all cell lines. Correlation values R for each pair of platforms are given in the figures. (a) Operon versus Incyte (b) Affymetrix versus Incyte (c) Affymetrix versus Operon.
Clustered Image Map (CIM) visualization and analysis [12
We used hierarchical clustering to demonstrate graphically the relationships among platforms, among cell lines, and among genes. 909 genes expressed at two times background or more in at least two of the six cell lines were included in the analysis. The resulting CIM ("heat map") is shown in Figure . All three platforms cluster together for all six cell lines, as one would wish to find, and almost all of the gene expression values show reasonable concordance. Only a few exceptions can be seen in the cluster shown in Figure . Some of the mismatches may be due to simple errors in gene identification, rather than to the technologies of the platforms. The Incyte library is guaranteed by the manufacturer to be only 90% correct, and an unknown percentage of the Operon and Affymetrix oligonucleotides may have been designed on the basis of incorrect sequences in the public databases. Indeed, we found one oligonucleotide in the Operon set that was apparently designed from an EST sequence that has since been withdrawn from the UniGene database (see RT-PCR studies below). In any case, the concordance is quite high across all platforms with this method of analysis as well as with the others.
Figure 5 a-b. Clustered image maps showing patterns of expression relationship among genes, platforms, and cell lines. The axes were ordered by hierarchical clustering using an uncentered correlation and the average linkage algorithm for 909 genes expressed at (more ...)
Quantitative real-time RT-PCR
In a pilot study with the three platforms, we compared and contrasted gene expression values for only the cell lines MCF10A and LNCaP. RT-PCR data for twelve genes are shown in Figure . Most of the values are in reasonable agreement except that there are differences in the magnitudes of the expression ratios. As found in other studies, the RT-PCR values are generally higher, probably because ratios are "flattened" with the microarray platforms. Affymetrix ratios are sometimes higher, but that may simply reflect the method of quantitation used in their analysis. The cDNA array ratios are generally lower than those from other platforms. Because the cDNA fragments are longer and double-stranded, they are more prone to non-specific hybridization and can cross-hybridize more easily to related sequences. These characteristics of the probes may result in higher background signal and concomitant reduction in dynamic range of the ratios. In general, we have found that the long oligonucleotide arrays have a larger dynamic range than the do the cDNA arrays.
Quantitative real-time RT-PCR analysis of 12 genes matched for direction of expression relative to the reference RNA for all three platforms. Log2 ratios are given in table below the graph. This example is a comparison between LnCaP and MCF-10A.
For further RT-PCR analysis, we chose a set of ten genes to test the accuracy of the three array platforms for all six cell lines. Those genes were chosen because there appeared to be a discrepancy among platforms in the direction of their ratios (i.e., whether they ratios were greater or less than unity). The results are shown in Figures . Of special interest was gene ETR101, in which the Operon array was in disagreement across all cell lines. Further inquiry revealed that the sequence had been found to be incorrect and had been removed from the UniGene database. Since the oligonucleotide had been designed from the incorrect sequence, it is not surprising that it gave a different value. Other discrepancies may be due to similar sequence errors, as even the most up-to-date databases are still being corrected and modified. In the case of AGL, the RT-PCR assay is in disagreement in several cases with two out of three of the array platforms; it appears to demonstrate an upregulation of the mRNA, whereas the arrays, with the exception of Incyte, point to a downregulation. Although RT-PCR is supposed to be the "gold standard" for measuring gene expression, this result shows that caution is indicated in interpreting results with even the PCR technology.
a-f. Quantitative RT-PCR analysis of 10 mismatched genes in the six cells lines for all three platforms. (a) MCF10A, (b) LnCaP, (c) OCI-Ly3, (d) Jurkat, (e) SUDHL-6 and (f) L428.