Even though fine-root turnover is a highly studied topic, it is often poorly understood as a result of uncertainties inherent in its sampling, e.g., quantifying spatial and temporal variability. While many methods exist to quantify fine-root turnover, use of minirhizotrons has increased over the last two decades, making sensor errors another source of uncertainty. Currently, no standardized methodology exists to test and compare minirhizotron camera capability, imagery, and performance. This paper presents a reproducible, laboratory-based method by which minirhizotron cameras can be tested and validated in a traceable manner. The performance of camera characteristics was identified and test criteria were developed: we quantified the precision of camera location for successive images, estimated the trueness and precision of each camera's ability to quantify root diameter and root color, and also assessed the influence of heat dissipation introduced by the minirhizotron cameras and electrical components. We report detailed and defensible metrology analyses that examine the performance of two commercially available minirhizotron cameras. These cameras performed differently with regard to the various test criteria and uncertainty analyses. We recommend a defensible metrology approach to quantify the performance of minirhizotron camera characteristics and determine sensor-related measurement uncertainties prior to field use. This approach is also extensible to other digital imagery technologies. In turn, these approaches facilitate a greater understanding of measurement uncertainties (signal-to-noise ratio) inherent in the camera performance and allow such uncertainties to be quantified and mitigated so that estimates of fine-root turnover can be more confidently quantified.
DNA microarrays are rapidly becoming a fundamental tool in discovery-based genomic and biomedical research. However, the reliability of the microarray results is being challenged due to the existence of different technologies and non-standard methods of data analysis and interpretation. In the absence of a "gold standard"/"reference method" for the gene expression measurements, studies evaluating and comparing the performance of various microarray platforms have often yielded subjective and conflicting conclusions. To address this issue we have conducted a large scale TaqMan® Gene Expression Assay based real-time PCR experiment and used this data set as the reference to evaluate the performance of two representative commercial microarray platforms.
In this study, we analyzed the gene expression profiles of three human tissues: brain, lung, liver and one universal human reference sample (UHR) using two representative commercial long-oligonucleotide microarray platforms: (1) Applied Biosystems Human Genome Survey Microarrays (based on single-color detection); (2) Agilent Whole Human Genome Oligo Microarrays (based on two-color detection). 1,375 genes represented by both microarray platforms and spanning a wide dynamic range in gene expression levels, were selected for TaqMan® Gene Expression Assay based real-time PCR validation. For each platform, four technical replicates were performed on the same total RNA samples according to each manufacturer's standard protocols. For Agilent arrays, comparative hybridization was performed using incorporation of Cy5 for brain/lung/liver RNA and Cy3 for UHR RNA (common reference). Using the TaqMan® Gene Expression Assay based real-time PCR data set as the reference set, the performance of the two microarray platforms was evaluated focusing on the following criteria: (1) Sensitivity and accuracy in detection of expression; (2) Fold change correlation with real-time PCR data in pair-wise tissues as well as in gene expression profiles determined across all tissues; (3) Sensitivity and accuracy in detection of differential expression.
Our study provides one of the largest "reference" data set of gene expression measurements using TaqMan® Gene Expression Assay based real-time PCR technology. This data set allowed us to use an alternative gene expression technology to evaluate the performance of different microarray platforms. We conclude that microarrays are indeed invaluable discovery tools with acceptable reliability for genome-wide gene expression screening, though validation of putative changes in gene expression remains advisable. Our study also characterizes the limitations of microarrays; understanding these limitations will enable researchers to more effectively evaluate microarray results in a more cautious and appropriate manner.
Several studies, including the MicroArray Quality Control (MAQC) project, have reported that concordant results in relative expression measures can be obtained in different microarray laboratories using the same RNA samples. These encouraging results have led to increased interest in and utilization of microarray technology in clinical and diagnostic assays, which require stringent regulatory approval. Many laboratories monitor microarray quality metrics calculated from each hybridization image, such as background or percent present. While these internal metrics provide essential information for individual hybridizations, they are not sufficient for validating overall competency of the facility, including precision and accuracy of the results, which may vary between laboratories and over time. Comprehensive assessments of laboratory performance must include inter-laboratory comparisons, such as a proficiency testing program, where identical samples are tested in multiple facilities.
We have initiated a proficiency testing program for microarray laboratories using the Affymetrix platform. In previous testing rounds, we compared data from 18 laboratories over a nine-month period using rat samples. Now we are expanding the program to additional laboratories using human samples. Three replicates of the two human MAQC reference RNA samples were repeatedly tested in multiple facilities. The poster will show the results of several analyses to assess the repeatability and comparability of each laboratory’s results with a variety of microbiomes to understand complex metabolic interactions and their impact on the host health and nutritional status.
Molecular biomarkers that are based on mRNA transcripts are being developed for the diagnosis and treatment of a number of diseases. DNA microarrays are one of the primary technologies being used to develop classifiers from gene expression data for clinically relevant outcomes. Microarray assays are highly multiplexed measures of comparative gene expression but have a limited dynamic range of measurement and show compression in fold change detection. To increase the clinical utility of microarrays, assay controls are needed that benchmark performance using metrics that are relevant to the analysis of genomic data generated with biological samples.
Ratiometric controls were prepared from commercial sources of high quality RNA from human tissues with distinctly different expression profiles and mixed in defined ratios. The samples were processed using six different target labeling protocols and replicate datasets were generated on high density gene expression microarrays. The area under the curve from receiver operating characteristic plots was calculated to measure diagnostic performance. The reliable region of the dynamic range was derived from log2 ratio deviation plots made for each dataset. Small but statistically significant differences in diagnostic performance were observed between standardized assays available from the array manufacturer and alternative methods for target generation. Assay performance using the reliable range of comparative measurement as a metric was improved by adjusting sample hybridization conditions for one commercial kit.
Process improvement in microarray assay performance was demonstrated using samples prepared from commercially available materials and two metrics - diagnostic performance and the reliable range of measurement. These methods have advantages over approaches that use a limited set of external controls or correlations to reference sets, because they provide benchmark values that can be used by clinical laboratories to help optimize protocol conditions and laboratory proficiency with microarray assays.
Concerns are often raised about the accuracy of microarray technologies and the degree of cross-platform agreement, but there are yet no methods which can unambiguously evaluate precision and sensitivity for these technologies on a whole-array basis.
A methodology is described for evaluating the precision and sensitivity of whole-genome gene expression technologies such as microarrays. The method consists of an easy-to-construct titration series of RNA samples and an associated statistical analysis using non-linear regression. The method evaluates the precision and responsiveness of each microarray platform on a whole-array basis, i.e., using all the probes, without the need to match probes across platforms. An experiment is conducted to assess and compare four widely used microarray platforms. All four platforms are shown to have satisfactory precision but the commercial platforms are superior for resolving differential expression for genes at lower expression levels. The effective precision of the two-color platforms is improved by allowing for probe-specific dye-effects in the statistical model. The methodology is used to compare three data extraction algorithms for the Affymetrix platforms, demonstrating poor performance for the commonly used proprietary algorithm relative to the other algorithms. For probes which can be matched across platforms, the cross-platform variability is decomposed into within-platform and between-platform components, showing that platform disagreement is almost entirely systematic rather than due to measurement variability.
The results demonstrate good precision and sensitivity for all the platforms, but highlight the need for improved probe annotation. They quantify the extent to which cross-platform measures can be expected to be less accurate than within-platform comparisons for predicting disease progression or outcome.
The increasing use of DNA microarrays in biomedical research, toxicogenomics, pharmaceutical development, and diagnostics has focused attention on the reproducibility and reliability of microarray measurements. While the reproducibility of microarray gene expression measurements has been the subject of several recent reports, there is still a need for systematic investigation into what factors most contribute to variability of measured expression levels observed among different laboratories and different experimenters.
We report the results of an interlaboratory comparison of gene expression array measurements on the same microarray platform, in which the RNA amplification and labeling, hybridization and wash, and slide scanning were each individually varied. Identical input RNA was used for all experiments. While some sources of variation have measurable influence on individual microarray signals, they showed very low influence on sample-to-reference ratios based on averaged triplicate measurements in the two-color experiments. RNA labeling was the largest contributor to interlaboratory variation.
Despite this variation, measurement of one particular breast cancer gene expression signature in three different laboratories was found to be highly robust, showing a high intralaboratory and interlaboratory reproducibility when using strictly controlled standard operating procedures.
The maturing of gene expression microarray technology and interest in the use of microarray-based applications for clinical and diagnostic applications calls for quantitative measures of quality. This manuscript presents a retrospective study characterizing several approaches to assess technical performance of microarray data measured on the Affymetrix GeneChip platform, including whole-array metrics and information from a standard mixture of external spike-in and endogenous internal controls. Spike-in controls were found to carry the same information about technical performance as whole-array metrics and endogenous "housekeeping" genes. These results support the use of spike-in controls as general tools for performance assessment across time, experimenters and array batches, suggesting that they have potential for comparison of microarray data generated across species using different technologies.
A layered PCA modeling methodology that uses data from a number of classes of controls (spike-in hybridization, spike-in polyA+, internal RNA degradation, endogenous or "housekeeping genes") was used for the assessment of microarray data quality. The controls provide information on multiple stages of the experimental protocol (e.g., hybridization, RNA amplification). External spike-in, hybridization and RNA labeling controls provide information related to both assay and hybridization performance whereas internal endogenous controls provide quality information on the biological sample. We find that the variance of the data generated from the external and internal controls carries critical information about technical performance; the PCA dissection of this variance is consistent with whole-array quality assessment based on a number of quality assurance/quality control (QA/QC) metrics.
These results provide support for the use of both external and internal RNA control data to assess the technical quality of microarray experiments. The observed consistency amongst the information carried by internal and external controls and whole-array quality measures offers promise for rationally-designed control standards for routine performance monitoring of multiplexed measurement platforms.
Quantum precision enhancement is of fundamental importance for the development of advanced metrological optical experiments, such as gravitational wave detection and frequency calibration with atomic clocks. Precision in these experiments is strongly limited by the 1/√N shot noise factor with N being the number of probes (photons, atoms) employed in the experiment. Quantum theory provides tools to overcome the bound by using entangled probes. In an idealized scenario this gives rise to the Heisenberg scaling of precision 1/N. Here we show that when decoherence is taken into account, the maximal possible quantum enhancement in the asymptotic limit of infinite N amounts generically to a constant factor rather than quadratic improvement. We provide efficient and intuitive tools for deriving the bounds based on the geometry of quantum channels and semi-definite programming. We apply these tools to derive bounds for models of decoherence relevant for metrological applications including: depolarization, dephasing, spontaneous emission and photon loss.
Quantum metrology employs the properties of quantum states to further enhance the accuracy of some of the most precise measurement schemes to date. Here, a method for estimating the upper bounds to achievable precision in quantum-enhanced metrology protocols in the presence of decoherence is presented.
High-throughput sequencing is gradually replacing microarrays as the preferred method for studying mRNA expression levels, providing nucleotide resolution and accurately measuring absolute expression levels of almost any transcript, known or novel. However, existing microarray data from clinical, pharmaceutical, and academic settings represent valuable and often underappreciated resources, and methods for assessing and improving the quality of these data are lacking.
To quantitatively assess the quality of microarray probes, we directly compare RNA-Seq to Agilent microarrays by processing 231 unique samples from the Allen Human Brain Atlas using RNA-Seq. Both techniques provide highly consistent, highly reproducible gene expression measurements in adult human brain, with RNA-Seq slightly outperforming microarray results overall. We show that RNA-Seq can be used as ground truth to assess the reliability of most microarray probes, remove probes with off-target effects, and scale probe intensities to match the expression levels identified by RNA-Seq. These sequencing scaled microarray intensities (SSMIs) provide more reliable, quantitative estimates of absolute expression levels for many genes when compared with unscaled intensities. Finally, we validate this result in two human cell lines, showing that linear scaling factors can be applied across experiments using the same microarray platform.
Microarrays provide consistent, reproducible gene expression measurements, which are improved using RNA-Seq as ground truth. We expect that our strategy could be used to improve probe quality for many data sets from major existing repositories.
Allen Brain Atlas; Microarray; RNA-Seq; High-throughput sequencing; Transcriptome profiling; Reliability; Gene expression; Brain
Several high-density oligonucleotide microarray platforms are available for genome-wide single nucleotide polymorphism (SNP) detection and microarray-based comparative genomic hybridisation (array CGH), which may be used to detect copy number aberrations in human tumours. As part of the EuroBoNeT network of excellence for research on bone tumours (eurobonet.eu), we have evaluated four different commercial high-resolution microarray platforms in order to identify the most appropriate technology for mapping DNA copy number aberrations in such tumours.
DNA from two different cytogenetically well-characterized bone sarcoma cell lines, representing a simple and a complex karyotype, respectively, was tested in duplicate on four high-resolution microarray platforms; Affymetrix Genome-Wide Human SNP Array 6.0, Agilent Human Genome CGH 244A, Illumina HumanExon510s-duo and Nimblegen HG18 CGH 385 k WG tiling v1.0. The data was analysed using the platform-specific analysis software, as well as a platform-independent analysis algorithm. DNA copy number was measured at six specific chromosomes or chromosomal regions, and compared with the expected ratio based on available cytogenetic information. All platforms performed well in terms of reproducibility and were able to delimit and score small amplifications and deletions at similar resolution, but Agilent microarrays showed better linearity and dynamic range. The platform-specific analysis software provided with each platform identified in general correct copy numbers, whereas using a platform-independent analysis algorithm, correct copy numbers were determined mainly for Agilent and Affymetrix microarrays.
All platforms performed reasonably well, but Agilent microarrays showed better dynamic range, and like Affymetrix microarrays performed well with the platform-independent analysis software, implying more robust data. Bone tumours like osteosarcomas are heterogeneous tumours with complex karyotypes that may be difficult to interpret, and it is of importance to be able to well separate the copy number levels and detect copy number changes in subpopulations. Taking all this into consideration, the Agilent and Affymetrix microarray platforms were found to be a better choice for mapping DNA copy numbers in bone tumours, the latter having the advantage of also providing heterozygosity information.
Identifying sources of variation in expression microarray data and the effect of variance in gene expression measurements on complex predictive and diagnostic models is essential when translating microarray-based experimental approaches into clinical assays. The technical reproducibility of microarray platforms is well established. Here, we investigate the additional impact of intratumor heterogeneity, a largely unstudied component of variance, on the performance of several microarray-based assays in breast cancer.
Patients and Methods
Genome-wide expression profiling was performed on 50 core needle biopsies from 18 breast cancer patients using Affymetrix GeneChip Human Genome U133 Plus 2.0 arrays. Global profiles of expression were characterized using unsupervised clustering methods and variance components models. Array-based measures of estrogen receptor (ER) and progesterone receptor (PR) status were compared with immunohistochemistry. The precision of genomic predictors of ER pathway status, recurrence risk, and sensitivity to chemotherapeutics was evaluated by interclass correlation.
Global patterns of gene expression demonstrated that intratumor variation was substantially less than the total variation observed across the patient population. Nevertheless, a fraction of genes exhibited significant intratumor heterogeneity in expression. A high degree of reproducibility was observed in single-gene predictors of ER (intraclass correlation coefficient [ICC] = 0.94) and PR expression (ICC = 0.90), and in a multigene predictor of ER pathway activation (ICC = 0.98) with high concordance with immunohistochemistry. Substantial agreement was also observed for multigene signatures of cancer recurrence (ICC = 0.71) and chemotherapeutic sensitivity (ICC = 0.72 and 0.64).
Intratumor heterogeneity, although present at the level of individual gene expression, does not preclude precise microarray-based predictions of tumor behavior or clinical outcome in breast cancer patients.
The MicroArray Quality Control (MAQC) project evaluated the inter- and intra-platform reproducibility of seven microarray platforms and three quantitative gene expression assays in profiling the expression of two commercially available Reference RNA samples (Nat Biotechnol 24:1115-22, 2006). The tested microarrays were the platforms from Affymetrix, Agilent Technologies, Applied Biosystems, GE Healthcare, Illumina, Eppendorf and the National Cancer Institute, and quantitative gene expression assays included TaqMan® Gene Expression PCR Assay, Standardized (Sta) RT-PCR™ and QuantiGene®. The data showed great consistency in gene expression measurements across different microarray platforms, different technologies and test sites. However, SYBR® Green real-time PCR, another common technique utilized by half of all real-time PCR users for gene expression measurement, was not addressed in the MAQC study. In the present study, we compared the performance of SYBR Green PCR with TaqMan PCR, microarrays and other quantitative technologies using the same two Reference RNA samples as the MAQC project. We assessed SYBR Green real-time PCR using commercially available RT2 Profiler™ PCR Arrays from SuperArray, containing primer pairs that have been experimentally validated to ensure gene-specificity and high amplification efficiency.
The SYBR Green PCR Arrays exhibit good reproducibility among different users, PCR instruments and test sites. In addition, the SYBR Green PCR Arrays have the highest concordance with TaqMan PCR, and a high level of concordance with other quantitative methods and microarrays that were evaluated in this study in terms of fold-change correlation and overlap of lists of differentially expressed genes.
These data demonstrate that SYBR Green real-time PCR delivers highly comparable results in gene expression measurement with TaqMan PCR and other high-density microarrays.
The measurement of gene expression using microarray technology is a complicated process in which a large number of factors can be varied. Due to the lack of standard calibration samples such as are used in traditional chemical analysis it may be a problem to evaluate whether changes done to the microarray procedure actually improve the identification of truly differentially expressed genes. The purpose of the present work is to report the optimization of several steps in the microarray process both in laboratory practices and in data processing using criteria that do not rely on external standards.
We performed a cDNA microarry experiment including RNA from samples with high expected differential gene expression termed "high contrasts" (rat cell lines AR42J and NRK52E) compared to self-self hybridization, and optimized a pipeline to maximize the number of genes found to be differentially expressed in the "high contrasts" RNA samples by estimating the false discovery rate (FDR) using a null distribution obtained from the self-self experiment. The proposed high-contrast versus self-self method (HCSSM) requires only four microarrays per evaluation. The effects of blocking reagent dose, filtering, and background corrections methodologies were investigated. In our experiments a dose of 250 ng LNA (locked nucleic acid) dT blocker, no background correction and weight based filtering gave the largest number of differentially expressed genes. The choice of background correction method had a stronger impact on the estimated number of differentially expressed genes than the choice of filtering method. Cross platform microarray (Illumina) analysis was used to validate that the increase in the number of differentially expressed genes found by HCSSM was real.
The results show that HCSSM can be a useful and simple approach to optimize microarray procedures without including external standards. Our optimizing method is highly applicable to both long oligo-probe microarrays which have become commonly used for well characterized organisms such as man, mouse and rat, as well as to cDNA microarrays which are still of importance for organisms with incomplete genome sequence information such as many bacteria, plants and fish.
Despite the widespread use of microarrays, much ambiguity regarding data analysis, interpretation and correlation of the different technologies exists. There is a considerable amount of interest in correlating results obtained between different microarray platforms. To date, only a few cross-platform evaluations have been published and unfortunately, no guidelines have been established on the best methods of making such correlations. To address this issue we conducted a thorough evaluation of two commercial microarray platforms to determine an appropriate methodology for making cross-platform correlations.
In this study, expression measurements for 10,763 genes uniquely represented on Affymetrix U133A/B GeneChips® and Amersham CodeLink™ UniSet Human 20 K microarrays were compared. For each microarray platform, five technical replicates, derived from the same total RNA samples, were labeled, hybridized, and quantified according to each manufacturers' standard protocols. The correlation coefficient (r) of differential expression ratios for the entire set of 10,763 overlapping genes was 0.62 between platforms. However, the correlation improved significantly (r = 0.79) when genes within noise were excluded. In addition to levels of inter-platform correlation, we evaluated precision, statistical-significance profiles, power, and noise levels for each microarray platform. Accuracy of differential expression was measured against real-time PCR for 25 genes and both platforms correlated well with r values of 0.92 and 0.79 for CodeLink and GeneChip, respectively.
As a result of this study, we recommend using only genes called 'present' in cross-platform correlations. However, as in this study, a large number of genes may be lost from the correlation due to differing levels of noise between platforms. This is an important consideration given the apparent difference in sensitivity of the two platforms. Data from microarray analysis need to be interpreted cautiously and therefore, we provide guidelines for making cross-platform correlations. In all, this study represents the most comprehensive and specifically designed comparison of short-oligonucleotide microarray platforms to date using the largest set of overlapping genes.
Quantum metrology bears a great promise in enhancing measurement precision, but is unlikely to become practical in the near future. Its concepts can nevertheless inspire classical or hybrid methods of immediate value. Here we demonstrate NOON-like photonic states of m quanta of angular momentum up to m=100, in a setup that acts as a ‘photonic gear’, converting, for each photon, a mechanical rotation of an angle θ into an amplified rotation of the optical polarization by mθ, corresponding to a ‘super-resolving’ Malus’ law. We show that this effect leads to single-photon angular measurements with the same precision of polarization-only quantum strategies with m photons, but robust to photon losses. Moreover, we combine the gear effect with the quantum enhancement due to entanglement, thus exploiting the advantages of both approaches. The high ‘gear ratio’ m boosts the current state of the art of optical non-contact angular measurements by almost two orders of magnitude.
Beating the standard measurement limits is a goal of metrology, as it would allow for more precise estimation of physical quantities. Borrowing concepts from NOON-state quantum metrology, this work presents a single-photon scheme to measure rotation angles of light with super-resolution precision.
Gene expression profiling is an important approach for detecting diagnostic and prognostic biomarkers, and predicting drug safety. The development of a wide range of technologies and platforms for measuring mRNA expression makes the evaluation and standardization of transcriptomic data problematic due to differences in protocols, data processing and analysis methods. Thus, universal RNA standards, such as those developed by the External RNA Controls Consortium (ERCC), are proposed to aid validation of research findings from diverse platforms such as microarrays and RT-qPCR, and play a role in quality control (QC) processes as transcriptomic profiling becomes more commonplace in the clinical setting.
Panels of ERCC RNA standards were constructed in order to test the utility of these reference materials (RMs) for performance characterization of two selected gene expression platforms, and for discrimination of biomarker profiles between groups. The linear range, limits of detection and reproducibility of microarray and RT-qPCR measurements were evaluated using panels of RNA standards. Transcripts of low abundance (≤ 10 copies/ng total RNA) showed more than double the technical variability compared to higher copy number transcripts on both platforms. Microarray profiling of two simulated 'normal' and 'disease' panels, each consisting of eight different RNA standards, yielded robust discrimination between the panels and between standards with varying fold change ratios, showing no systematic effects due to different labelling and hybridization runs. Also, comparison of microarray and RT-qPCR data for fold changes showed agreement for the two platforms.
ERCC RNA standards provide a generic means of evaluating different aspects of platform performance, and can provide information on the technical variation associated with quantification of biomarkers expressed at different levels of physiological abundance. Distinct panels of standards serve as an ideal quality control tool kit for determining the accuracy of fold change cut-off threshold and the impact of experimentally-derived noise on the discrimination of normal and disease profiles.
MicroRNAs (miRNAs) represent a growing class of small non-coding RNAs that are important regulators of gene expression in both plants and animals. Studies have shown that miRNAs play a critical role in human cancer and they can influence the level of cell proliferation and apoptosis by modulating gene expression. Currently, methods for the detection and measurement of miRNA expression include small and moderate-throughput technologies, such as standard quantitative PCR and microarray based analysis. However, these methods have several limitations when used in large clinical studies where a high-throughput and highly quantitative technology needed for the efficient characterization of a large number of miRNA transcripts in clinical samples. Furthermore, archival formalin fixed, paraffin embedded (FFPE) samples are increasingly becoming the primary resource for gene expression studies because fresh frozen (FF) samples are often difficult to obtain and requires special storage conditions. In this study, we evaluated the miRNA expression levels in FFPE and FF samples as well as several lung cancer cell lines employing a high throughput qPCR-based microfluidic technology. The results were compared to standard qPCR and hybridization-based microarray platforms using the same samples.
We demonstrated highly correlated Ct values between multiplex and singleplex RT reactions in standard qPCR assays for miRNA expression using total RNA from A549 (R = 0.98; p < 0.0001) and H1299 (R = 0.95; p < 0.0001) lung cancer cell lines. The Ct values generated by the microfluidic technology (Fluidigm 48.48 dynamic array systems) resulted in a left-shift toward lower Ct values compared to those observed by ABI 7900 HT (mean difference, 3.79), suggesting that the microfluidic technology exhibited a greater sensitivity. In addition, we show that as little as 10 ng total RNA can be used to reliably detect all 48 or 96 tested miRNAs using a 96-multiplexing RT reaction in both FFPE and FF samples. Finally, we compared miRNA expression measurements in both FFPE and FF samples by qPCR using the 96.96 dynamic array and Affymetrix microarrays. Fold change comparisons for comparable genes between the two platforms indicated that the overall correlation was R = 0.60. The maximum fold change detected by the Affymetrix microarray was 3.5 compared to 13 by the 96.96 dynamic array.
The qPCR-array based microfluidic dynamic array platform can be used in conjunction with multiplexed RT reactions for miRNA gene expression profiling. We showed that this approach is highly reproducible and the results correlate closely with the existing singleplex qPCR platform at a throughput that is 5 to 20 times higher and a sample and reagent usage that was approximately 50-100 times lower than conventional assays. We established optimal conditions for using the Fluidigm microfluidic technology for rapid, cost effective, and customizable arrays for miRNA expression profiling and validation.
Transcriptome sequencing using next-generation sequencing platforms will soon be competing with DNA microarray technologies for global gene expression analysis. As a preliminary evaluation of these promising technologies, we performed deep sequencing of cDNA synthesized from the Microarray Quality Control (MAQC) reference RNA samples using Roche's 454 Genome Sequencer FLX.
We generated more that 3.6 million sequence reads of average length 250 bp for the MAQC A and B samples and introduced a data analysis pipeline for translating cDNA read counts into gene expression levels. Using BLAST, 90% of the reads mapped to the human genome and 64% of the reads mapped to the RefSeq database of well annotated genes with e-values ≤ 10-20. We measured gene expression levels in the A and B samples by counting the numbers of reads that mapped to individual RefSeq genes in multiple sequencing runs to evaluate the MAQC quality metrics for reproducibility, sensitivity, specificity, and accuracy and compared the results with DNA microarrays and Quantitative RT-PCR (QRTPCR) from the MAQC studies. In addition, 88% of the reads were successfully aligned directly to the human genome using the AceView alignment programs with an average 90% sequence similarity to identify 137,899 unique exon junctions, including 22,193 new exon junctions not yet contained in the RefSeq database.
Using the MAQC metrics for evaluating the performance of gene expression platforms, the ExpressSeq results for gene expression levels showed excellent reproducibility, sensitivity, and specificity that improved systematically with increasing shotgun sequencing depth, and quantitative accuracy that was comparable to DNA microarrays and QRTPCR. In addition, a careful mapping of the reads to the genome using the AceView alignment programs shed new light on the complexity of the human transcriptome including the discovery of thousands of new splice variants.
In the field of molecular biology the ability to determine total RNA integrity is a valuable tool used to assess the suitability of biological samples for downstream analysis using techniques such as microarray profiling, RT-PCR, and genomic sequencing. Using the results of such analyses, RNA degradation status and contamination can be measured and the data used to predict sample performance in genomic assays. Historically, agarose gel electrophoresis was the quality assessment technique used in most molecular biology laboratories but this procedure requires microgram quantities of RNA and the sensitivity of the assay is often dependent on the interpretation of a photographed gel image. With the development of the Agilent 2100 Bioanalyzer, a microfluidics-based platform, samples are now being evaluated more quickly and precisely. This instrument allows researchers to use low amounts of RNA and the Agilent software provides quality assessment metrics such as ribosomal RNA ratios, concentration estimates, a pseudo -gel image, electropherogram traces and an Agilent algorithmic interpretation of the sample RNA integrity (“RIN”-RNA Integrity Number). With a 12 sample per chip limitation and a 30 minute per chip processing time, the Agilent 2100 Bioanalyzer system can be considered relatively low-throughput and may not meet the needs of a laboratory that requires processing of samples on a larger scale. With the development of the Caliper GX II, a high-throughput microfluidics-based platform, 96 samples can be processed in 2.5 hours. In this study, the Agilent 2100 Bioanalyzer and the Caliper GX II have been evaluated to compare total RNA scoring algorithms, reproducibility, and the ability to provide informative RNA quality metrics to help in predicting microarray performance. It was observed that the performance of the two instruments is comparable in functionality and reproducibility but there are notable differences between the RNA scoring algorithms.
Although numerous investigations have compared gene expression microarray platforms, preprocessing methods and batch correction algorithms using constructed spike-in or dilution datasets, there remains a paucity of studies examining the properties of microarray data using diverse biological samples. Most microarray experiments seek to identify subtle differences between samples with variable background noise, a scenario poorly represented by constructed datasets. Thus, microarray users lack important information regarding the complexities introduced in real-world experimental settings. The recent development of a multiplexed, digital technology for nucleic acid measurement enables counting of individual RNA molecules without amplification and, for the first time, permits such a study.
Using a set of human leukocyte subset RNA samples, we compared previously acquired microarray expression values with RNA molecule counts determined by the nCounter Analysis System (NanoString Technologies) in selected genes. We found that gene measurements across samples correlated well between the two platforms, particularly for high-variance genes, while genes deemed unexpressed by the nCounter generally had both low expression and low variance on the microarray. Confirming previous findings from spike-in and dilution datasets, this “gold-standard” comparison demonstrated signal compression that varied dramatically by expression level and, to a lesser extent, by dataset. Most importantly, examination of three different cell types revealed that noise levels differed across tissues.
Microarray measurements generally correlate with relative RNA molecule counts within optimal ranges but suffer from expression-dependent accuracy bias and precision that varies across datasets. We urge microarray users to consider expression-level effects in signal interpretation and to evaluate noise properties in each dataset independently.
Electronic supplementary material
The online version of this article (doi:10.1186/1471-2164-15-649) contains supplementary material, which is available to authorized users.
Microarray; NanoString; nCounter; Gene expression
Many researchers are concerned with the comparability and reliability of microarray gene expression data. Recent completion of the MicroArray Quality Control (MAQC) project provides a unique opportunity to assess reproducibility across multiple sites and the comparability across multiple platforms. The MAQC analysis presented for the conclusion of inter- and intra-platform comparability/reproducibility of microarray gene expression measurements is inadequate. We evaluate the reproducibility/comparability of the MAQC data for 12901 common genes in four titration samples generated from five high-density one-color microarray platforms and the TaqMan technology. We discuss some of the problems with the use of correlation coefficient as metric to evaluate the inter- and intra-platform reproducibility and the percent of overlapping genes (POG) as a measure for evaluation of a gene selection procedure by MAQC.
A total of 293 arrays were used in the intra- and inter-platform analysis. A hierarchical cluster analysis shows distinct differences in the measured intensities among the five platforms. A number of genes show a small fold-change in one platform and a large fold-change in another platform, even though the correlations between platforms are high. An analysis of variance shows thirty percent of gene expressions of the samples show inconsistent patterns across the five platforms. We illustrated that POG does not reflect the accuracy of a selected gene list. A non-overlapping gene can be truly differentially expressed with a stringent cut, and an overlapping gene can be non-differentially expressed with non-stringent cutoff. In addition, POG is an unusable selection criterion. POG can increase or decrease irregularly as cutoff changes; there is no criterion to determine a cutoff so that POG is optimized.
Using various statistical methods we demonstrate that there are differences in the intensities measured by different platforms and different sites within platform. Within each platform, the patterns of expression are generally consistent, but there is site-by-site variability. Evaluation of data analysis methods for use in regulatory decision should take no treatment effect into consideration, when there is no treatment effect, "a fold-change cutoff with a non-stringent p-value cutoff" could result in 100% false positive error selection.
Obtaining reliable and reproducible two-color microarray gene expression data is critically important for understanding the biological significance of perturbations made on a cellular system. Microarray design, RNA preparation and labeling, hybridization conditions and data acquisition and analysis are variables difficult to simultaneously control. A useful tool for monitoring and controlling intra- and inter-experimental variation is Universal Reference RNA (URR), developed with the goal of providing hybridization signal at each microarray probe location (spot). Measuring signal at each spot as the ratio of experimental RNA to reference RNA targets, rather than relying on absolute signal intensity, decreases variability by normalizing signal output in any two-color hybridization experiment.
Human, mouse and rat URR (UHRR, UMRR and URRR, respectively) were prepared from pools of RNA derived from individual cell lines representing different tissues. A variety of microarrays were used to determine percentage of spots hybridizing with URR and producing signal above a user defined threshold (microarray coverage). Microarray coverage was consistently greater than 80% for all arrays tested. We confirmed that individual cell lines contribute their own unique set of genes to URR, arguing for a pool of RNA from several cell lines as a better configuration for URR as opposed to a single cell line source for URR. Microarray coverage comparing two separately prepared batches each of UHRR, UMRR and URRR were highly correlated (Pearson's correlation coefficients of 0.97).
Results of this study demonstrate that large quantities of pooled RNA from individual cell lines are reproducibly prepared and possess diverse gene representation. This type of reference provides a standard for reducing variation in microarray experiments and allows more reliable comparison of gene expression data within and between experiments and laboratories.
microarray; universal reference RNA; standardization.
The incorporation of statistical models that account for experimental variability provides a necessary framework for the interpretation of microarray data. A robust experimental design coupled with an analysis of variance (ANOVA) incorporating a model that accounts for known sources of experimental variability can significantly improve the determination of differences in gene expression and estimations of their significance.
To realize the full benefits of performing analysis of variance on microarray data we have developed CARMA, a microarray analysis platform that reads data files generated by most microarray image processing software packages, performs ANOVA using a user-defined linear model, and produces easily interpretable graphical and numeric results. No pre-processing of the data is required and user-specified parameters control most aspects of the analysis including statistical significance criterion. The software also performs location and intensity dependent lowess normalization, automatic outlier detection and removal, and accommodates missing data.
CARMA provides a clear quantitative and statistical characterization of each measured gene that can be used to assess marginally acceptable measures and improve confidence in the interpretation of microarray results. Overall, applying CARMA to microarray datasets incorporating repeated measures effectively reduces the number of gene incorrectly identified as differentially expressed and results in a more robust and reliable analysis.
Biomarker discovery and development requires measurement reproducibility studies in addition to case-control studies. Parallel pursuit of reproducibility studies is especially important for emerging technologies such as protein biomarkers based on time-of-flight mass spectrometry, the case considered in this paper. For parallel studies, a way to improve reproducibility prior to identification of protein species is necessary. One approach is use of functional principal components analysis (PCA) as the basis for assessing measurement reproducibility. Reproducibility studies involve repeated measurement of a reference material such as a human serum standard. Measurement in our example is by SELDI-TOF (surface-enhanced laser desorption and ionization time-of-flight) mass spectrometry. Reproducibility is defined in reference to a source of variation, which in our example is associated with a type of commercially available protein biochip. We obtained spectra for 8 spots on each 11 chips. Two spectra are generally more alike when obtained from the same chip rather than different chips. Thus, our experiment indicates potential improvements from reducing variation in chip manufacture and chip handling during measurement. Our analysis involves careful registration of the spectra and characterization of the spectral differences. As shown by our example, a metrological analysis may enhance case-control studies by guiding optimization of the measurements underlying the biomarker.
Microarray technology has become a widely used tool in the biological sciences. Over the past decade, the number of users has grown exponentially, and with the number of applications and secondary data analyses rapidly increasing, we expect this rate to continue. Various initiatives such as the External RNA Control Consortium (ERCC) and the MicroArray Quality Control (MAQC) project have explored ways to provide standards for the technology. For microarrays to become generally accepted as a reliable technology, statistical methods for assessing quality will be an indispensable component; however, there remains a lack of consensus in both defining and measuring microarray quality.
We begin by providing a precise definition of microarray quality and reviewing existing Affymetrix GeneChip quality metrics in light of this definition. We show that the best-performing metrics require multiple arrays to be assessed simultaneously. While such multi-array quality metrics are adequate for bench science, as microarrays begin to be used in clinical settings, single-array quality metrics will be indispensable. To this end, we define a single-array version of one of the best multi-array quality metrics and show that this metric performs as well as the best multi-array metrics. We then use this new quality metric to assess the quality of microarry data available via the Gene Expression Omnibus (GEO) using more than 22,000 Affymetrix HGU133a and HGU133plus2 arrays from 809 studies.
We find that approximately 10 percent of these publicly available arrays are of poor quality. Moreover, the quality of microarray measurements varies greatly from hybridization to hybridization, study to study, and lab to lab, with some experiments producing unusable data. Many of the concepts described here are applicable to other high-throughput technologies.