PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of ijmsMDPIhomeThis articleThis journalInstructions for authorsSubscribeIJMS
 
Int J Mol Sci. 2010; 11(9): 3397–3412.
Published online 2010 September 20. doi:  10.3390/ijms11093397
PMCID: PMC2956103

Practical Application of Toxicogenomics for Profiling Toxicant-Induced Biological Perturbations

Abstract

A systems-level understanding of molecular perturbations is crucial for evaluating chemical-induced toxicity risks appropriately, and for this purpose comprehensive gene expression analysis or toxicogenomics investigation is highly advantageous. The recent accumulation of toxicity-associated gene sets (toxicogenomic biomarkers), enrichment in public or commercial large-scale microarray database and availability of open-source software resources facilitate our utilization of the toxicogenomic data. However, toxicologists, who are usually not experts in computational sciences, tend to be overwhelmed by the gigantic amount of data. In this paper we present practical applications of toxicogenomics by utilizing biomarker gene sets and a simple scoring method by which overall gene set-level expression changes can be evaluated efficiently. Results from the gene set-level analysis are not only an easy interpretation of toxicological significance compared with individual gene-level profiling, but also are thought to be suitable for cross-platform or cross-institutional toxicogenomics data analysis. Enrichment in toxicogenomics databases, refinements of biomarker gene sets and scoring algorithms and the development of user-friendly integrative software will lead to better evaluation of toxicant-elicited biological perturbations.

Keywords: toxicogenomics, microarray, biomarker, bioinformatics, systems biology

1. Introduction

The term ‘toxic agent’ can be defined as any substance that causes harmful effects on living organisms, but in general, such hazardous effects are substantially dependent on the chemical’s exposure level. For instance, dietary salt may cause nephrotoxicity if an extreme amount is ingested at any time; however, do we consider sodium chloride a toxic agent? This is not the case because ordinarily we only need a spoon of salt for cooking and that amount will not harm healthy human bodies. In addition, in the case of pharmaceutical drugs, any chemical can be toxic when overdosed, and what makes them beneficial or toxic depends on their dose levels. Various types of preclinical toxicity studies are required before starting a human clinical trial to collect information on the toxicological profile such as target organs of toxicities, dose-response profile, recovery, toxicokinetics, genotoxicity, teratogenicity and carcinogenicity, using a sufficient number of experimental animals. However, even after a long and costly preclinical toxicity evaluation, species difference in mechanism of action (MOA) between humans and animals sometimes brings about unpredictable toxicities in humans [1]. Furthermore, a chemical once regarded as non-toxic could cause exaggerated toxicity in case undesirable chemical-chemical interaction occurs through inhibition or induction of ABC transporters [2] or hepatic drug metabolizing enzymes [3]. To evaluate and manage the potential hazardous risks of the chemicals, it is desirable to gain insight into the chemical-induced MOA in the target organ of toxicity, by which we can appropriately evaluate whether the chemical should be regarded as toxic or not according to its overall risk/benefit property.

For living organisms, it is sometimes necessary to modulate biological homeostasis to overcome potential hazardous effects caused by chemical exposure. For example, administration of anticonvulsant phenobarbital to rats causes liver hypertrophy, which is associated with induction of hepatic drug metabolizing enzymes such as CYP2B and CYP3A through the activation of the nuclear receptor CAR [4]. Although an increase in liver weight may look like a deleterious perturbation of hepatic homeostasis, toxicologists usually regard it as a non-toxic but rather a desirable response or adaptive response for the body. This is because such hepatic enzyme induction facilitates efficient metabolism and disposition of the exposed chemicals. In a systems-biological point of view, such liver system is called ‘robust’, not ‘homeostatic’ against phenobarbital exposure [5], where biological systems are not static but exhibit dynamic molecular reconstructions to maintain their cellular functional integrity. From this perspective, drug-induced toxicity can be defined as a ‘collapse of the biological robustness by drug exposure’. The recent advancement in Systems Biology is supported by dramatic advances in functional genomics techniques, especially the microarray technique by which expression levels of tens of thousands of genes can be measured simultaneously. Application of the microarray technique to toxicology research is called toxicogenomics (TGx), and is now widely utilized by pharmaceutical scientists in drug development [6]. The major problem in utilizing the TGx technique lies in its huge data size, as well as the complexity of the systems-level molecular interactions. We cannot avoid performing multivariate analyses to handle huge data sets, but toxicologists usually struggle to implement complicated statistical analysis. To overcome such difficulties, an easy, simple and practical analytical flow is desired. In this paper, we present practical methods of TGx data analysis for profiling chemical-elicited molecular perturbations using an open source analytical software, which will lead to better and easier utilization of TGx data to understand the MOA of toxicities.

2. Advancement of Toxicogenomics

2.1. Toxicogenomic Biomarker Gene Sets

In 2001, it was reported that the hepatic gene expression profiles in rats following treatment with various chemicals showed clear chemical-specific patterns when measured with the microarray technique [7,8]. Such chemical-specific changes in the transcriptome profile leads to changes in the proteome profile, the metabolome profile and eventually the tissue-level phenotypes. Thus, it is natural that the transcriptome profile would contain a significant degree of information for biological conditions at the moment, which may lead to a profound understanding of chemical-induced molecular perturbations. However, such chemical-specific gene expression data contain mixed molecular events that reflect complicated interactions among biological pathways such as xenobiotic metabolism, stress response, energy metabolism, protein synthesis/degradation, mRNA transcription/degradation, DNA repair/replication, cell proliferation/cell death control, etc. Microarray analysis measures tens of thousands of gene expression levels simultaneously, and is usually too complicated to appropriately interpret the significance of the gene expression changes at a time. Instead, it would be more practical to focus on the data for certain gene sets whose expression levels are closely associated with certain biological functions like glycolysis and cell proliferation. Such gene sets can be prepared from public information such as PubMed literature search, Gene Ontology [9], KEGG [10] and GenMAPP [11] biological pathway information. In addition, a number of gene sets have been reported whose expression levels are closely associated with certain toxicological endpoints, or TGx biomarker gene sets [12] such as cell injury [13], carcinogenicity [14,15], phospholipidosis [16,17] and glutathione depletion [18,19]. These TGx biomarkers can then be utilized for evaluation, diagnosis or prediction of toxicity based on their expression changes. As shown in Figure 1, it is much more informative and easy to interpret the microarray data by focusing on certain gene sets rather than observing whole data sets (i.e., >30,000 gene probe sets in the case of Affymetrix GeneChip system). However, it is still too complicated when we need to handle a number of biomarker gene sets at a time. Because pharmaceutical toxicologists are usually not experts in handling vast amounts of data sets, it is crucial to develop user-friendly analytical software whose output is clear enough to interpret the toxicological significance.

Figure 1
Expression profiling for toxicogenomic biomarker gene sets. Gene sets whose expression levels are closely associated with cell proliferation, glutathione metabolism and inflammatory responses are presented. The heat map represents gene expression changes, ...

2.2. Public and Commercial Microarray Database

A high quality and large-scale reference microarray database is desired for an appropriate interpretation of TGx data. A number of public databases are currently available, such as Gene Expression Ominibus (GEO) [20], ArrayExpress [21], Chemical Effects in Biological Systems (CEBS) [22], Comparative Toxicogenomics Database (CTD) [23] or EDGE [24]. In addition, the Toxicogenomics Project in Japan (http://wwwtgp.nibio.go.jp/index.html) and the InnoMed PredTox Consortium (http://www.innomed-predtox.com/) developed large-scale toxicogenomic databases, both of which contain microarray datasets for well-studied toxicants, as well as proprietary drugs using both in vivo and in vitro systems.

2.3. Open Source Software

Development of open source software is another driving factor of the recent advances in functional genomics research [25], although in many cases they require users to possess a certain degree of computational skills. In this paper, Bioconductor [26] (http://www.bioconductor.org/), which is implemented on the statistical software R [27] (http://cran.r-project.org/), GraphViz (http://www.graphviz.org/) and Cytoscape [28] (http://www.cytoscape.org/) were actually utilized and the results are presented.

3. Practical Application of TGx Database and Biomarkers

3.1. Scoring the Gene Set-Level Expression Changes

As stated before, the process of data analysis and interpretation of the results becomes increasingly complex when we handle large-scale microarray data sets and multiple TGx biomarker gene sets simultaneously. To facilitate our understanding of the toxicological significance based on the TGx data, a simple scoring strategy was introduced to evaluate gene set-level expression changes. Figure 2 shows the general concept on how microarray data and multiple TGx biomarker gene sets can be processed to generate a simple score for each TGx biomarker gene set, by which toxicologists can identify which biological endpoints were affected by chemical exposure, and thereby can evaluate the toxicological significance efficiently.

Figure 2
Scoring multiple toxicological endpoints using toxicogenomics data. Multiple toxicological endpoint-associated gene sets or TGx biomarkers need to be prepared in advance, and the overall expression changing levels for each gene set are calculated by certain ...

In the Toxicogenomics Project in Japan, a large-scale TGx database called the Toxicogenomics Project-Genomics Assisted Toxicity Evaluation system (TG-GATEs) was developed that consisted of Affymetrix GeneChip data for approximately 150 prototypical toxicants on rat liver, kidney, hepatocytes and human hepatocytes [29]. In the TG-GATEs system, an expression ratio-based scoring method called ‘TGP1 score’ was utilized to facilitate understanding of the toxicological significance based on the TGx data set [30].

The TGP1 score is calculated by multiplying two elements: one element represents an index for the “overall direction of the expression change per probe set”, and the other represents the “overall magnitude of the expression change per gene” of the Biomarker X gene set. The sign of the first index will be either positive or negative when the overall expression changes of the genes in Biomarker X were up- or down-regulated, respectively, and will be expected to approach zero when the direction of expression changes is divergent. The second index is always positive and will be higher when the expression change levels of the genes show a higher value. Collectively, the TGP1 score will be higher when the genes included in Biomarker X show uniform up-regulation with higher expression changing levels.

3.2. Differentially-Regulated Gene Score (D-score)

The TGP1 score was found to be useful for efficient comprehension of large-scale microarray data sets. However, the TGP1 score calculation treats both low and high quality gene expression data equally, and therefore the calculated score would be flawed if low quality data was involved in the calculation. To overcome this shortcoming, we introduced a new scoring method called Differentially-expressed gene score (D-score) [31], where the data quality as well as the expression changing level for each gene is considered in the score calculation. Thus, the calculated score is much more reliable compared with the TGP1 score.

Figure 3 represents D-scores for microarray data on rat livers treated with one of the four hepatotoxicants: acetaminophen (APAP), phenobarbital (PB), clofibrate (CFB) or acetamidofluorene (AAF) using 10 biomarker gene sets for score calculation. Stimulation of glutathione depletion and inflammation by APAP, induction of Cyp2b and Cyp3a genes, induction of PPARα by CFB and induction of Cyp1a genes are evident by D-score analysis results (Figure 3(A)), where the dose-dependent stimulation of these endpoints or genes can also be evaluated (Figure 3(B)). These results demonstrate that TGx biomarker gene sets and the D-score calculation method dramatically facilitate the interpretation of the TGx data.

Figure 3
Detection of affected toxicological endpoints by D-score. (A) Rats were treated with prototypical hepatotoxicants acetaminophen (APAP), phenobarbital (PB), clofibrate (CFB) or acetamidofluorene (AAF), and the hepatic microarray data were obtained at 3, ...

3.3. Inference of Gene Set-Level Network Structure Using a TGx Database

Previous studies reported inconsistency of interlaboratory/inter-platform microarray results [32,33], while others have reported good concordance among laboratories [3436] or inconclusive results [37,38]. Because a gene set-level or biological pathway-level analysis has been reported to be more robust and comparable for microarray data sets obtained from different studies [39,40], we hypothesized that the biological pathway-level interactions could be better evaluated using D-scores for multiple gene sets as compared with generic molecular network analysis methods, which were conducted with individual gene-by-gene-level analysis [41]. We inferred the gene set-level network structure using a large-scale TGx database, TG-GATEs, and a total of 58 gene sets [42] with a Gaussian graphical model (GGM) algorithm to calculate partial correlation coefficients among the gene sets [43]. In addition to gene expression data, we also included changing levels of phenotypic data such as organ weight, blood chemistry and hematology parameters for calculation.

The inferred network presented in Figure 4 was found to contain a number of toxicologically-relevant gene set—gene set and gene set-phenotype relationships, such as blood glucose level and hepatic glycolysis-associated gene sets, or blood aminotransferase enzyme activity and inflammation-associated gene sets [42]. These results demonstrate that the retrospective network inference using a GGM algorithm successfully highlighted toxicologically significant gene set- and phenotype-level relationships from a large-scale TGx database. Furthermore, microarray data set obtained outside TG-GATEs was found to be well compatible with the network structure inferred based on TG-GATEs [42], suggesting that the gene set-level network structure was robust enough to be applicable for external microarray data sets.

Figure 4
Gene set- and phenotype-level network analysis. A large-scale TGx database, TG-GATE, was used for extracting statistically significant relationships among gene sets and phenotypes by utilizing a GGM algorithm. The network consists of D-scores for 58 gene ...

4. Case Study: Bromobenzene-Induced Molecular Perturbation

In this section, a case study is presented for evaluating hepatic molecular perturbations elicited by 300 mg/kg bromobenzene (BBz) treatment in rats. D-score was calculated for a total of 58 gene sets and the calculated score was presented as either a radar chart (Figure 5), heat map (Figure 6) or network structure (Figure 7).

Figure 5
Time course of D-score: Radar chart presentation. D-scores were calculated for rat livers harvested at 2, 6, 12 and 24 h after bromobenzene treatment, and are presented in a radar chart. The red line indicates the D-score for each gene set, and the blue ...
Figure 6
Time course of D-score: Heat map presentation. Red and blue indicate high and low D-scores, or up- and down-regulation for each gene set, respectively. The heat map indicates that the first trigger invoked by bromobenzene exposure was glutathione depletion ...
Figure 7
Network structure presentation of D-scores. Biological and toxicological relationships among gene sets were visualized as a supervised network using GraphViz software. The D-scores calculated for microarray data on rat livers at 24 h after bromobenzene ...

4.1. Radar Chart Presentation

Figure 5 shows the time course of absolute D-score values presented in radar charts, where a total of 58 gene sets, a detailed gene list which has been reported previously [42], were used for score calculation. Significant biological perturbations became evident at 6 h after BBz treatment, where DNA damage- and Glutathione depletion-associated gene sets exhibited high D-scores. BBz is reported to cause hepatic glutathione depletion through the generation of reactive metabolites [44], which is concordant with the high D-score for the glutathione depletion-associated genes. At 12 h, D-scores for Oxidative stress- and Inflammation-associated gene sets exhibited high scores, suggesting that oxidative stress, supposedly associated with glutathione depletion, induced liver injury, which was followed by an inflammatory response. Gluthatione homeostasis-associated genes, which include glutathione synthesis-related genes, were also activated at 12 h, which would contribute to feedback up-regulation of glutathione synthesis against acute glutathione depletion. The D-scores for Inflammation- and DNA damage-associated gene sets became considerably high at 24 h after BBz treatment, suggesting the level of liver injury progressed in spite of activation of Glutathione homeostasis-associated genes.

4.2. Heat Map Presentation

Time course transition of D-score profiles is presented in Figure 6, which clearly demonstrates the molecular mechanisms of BBz-elicited toxicity: Glutathione depletion and oxidative stress are the first triggers after BBz treatment, which caused cell death suggested by up-regulation of DNA damage-associated genes. This was followed by inflammation, tissue repair, as well as activation of antioxidant factors such as up-regulation of glutathione synthesis-related genes, glutathione S-transferase (Gst) genes or aldo-keto reductase (Akr) genes.

4.3. Supervised Network Structure Presentation

The BBz-elicited biological responses can also be visualized with the pre-defined biological network structure. Figure 7 represents a D-score profile at 24 h after BBz treatment, where gene sets associated with glutathione depletion, oxidative stress and inflammation were up-regulated (colored in red). On the other hand, gene sets associated with energy metabolism (i.e., cholesterol synthesis and glycolysis) were down-regulated (colored in blue). Thus, the systems-level biological responses can be intuitively characterized with the supervised gene set-level biological network. However, the network structure should be kept up-to-date in a timely manner when novel toxicological knowledge has been obtained.

5. Conclusion

Systems-level understanding of molecular perturbations is crucial for evaluating chemical-induced toxicity risks. Microarray data provides comprehensive gene expression responses against chemical exposure, and therefore the TGx approach is highly advantageous for understanding systems-level biological perturbations. To solve the difficulty in handling huge amounts of TGx data sets, preparation of TGx biomarker gene sets and implementation of gene set-level data analysis are effective. The general flow of TGx data analysis is shown in Figure 8. The first step is to identify biological pathways that were affected by the chemical exposure, for which a radar chart presentation will be useful (Figure 8(A)). Detailed expression analysis of the individual genes will be needed to focus on the affected biological pathways (Figure 8(B)). When available, large-scale TGx reference database will be utilized for comparative analysis to appropriately evaluate the toxicological significance (Figures 8(C) and 8(D)). To comprehend the systems-level molecular dynamics, relationships among pathways should be taken into consideration (Figure 8(F)). Such analytical flow can be automated if appropriate computational skills are available. Refining toxicogenomic biomarker gene sets, scoring algorithm and development of user-friendly integrative software will substantially help the utilization of the TGx data set to evaluate biological response by which hazardous effects of exposed chemicals could be appropriately managed.

Figure 8
Analytical flow for toxicity evaluation using TGx data. (A) Radar chart for D-scores using 58 gene sets; (B) Box plot for comparative analysis using a reference TGx database; (C) Heat map for individual genes; (D) TGx reference database and TGx biomarker ...

Acknowledgements

The authors thank Kazumi Ito and Yosuke Ando for their helpful discussions, and Kyoko Watanabe, Noriyo Niino, Miyuki Kanbori and Shusuke Yamauchi for their dedication to the Toxicogenomics team in Daiichi Sankyo.

References

1. Kaplowitz N. Idiosyncratic drug hepatotoxicity. Nat. Rev. Drug Discov. 2005;4:489–499. [PubMed]
2. Marchetti S, Mazzanti R, Beijnen JH, Schellens JH. Concise review: Clinical relevance of drug drug and herb drug interactions mediated by the ABC transporter ABCB1 (MDR1, P-glycoprotein) Oncologist. 2007;12:927–941. [PubMed]
3. Haddad A, Davis M, Lagman R. The pharmacological importance of cytochrome CYP3A4 in the palliation of symptoms: Review and recommendations for avoiding adverse drug interactions. Support. Care Cancer. 2007;15:251–257. [PubMed]
4. Kodama S, Negishi M. Phenobarbital confers its diverse effects by activating the orphan nuclear receptor car. Drug Metab. Rev. 2006;38:75–87. [PubMed]
5. Kitano H. Towards a theory of biological robustness. Mol. Syst. Biol. 2007;3:137. [PMC free article] [PubMed]
6. Boverhof DR, Zacharewski TR. Toxicogenomics in risk assessment: applications and needs. Toxicol. Sci. 2006;89:352–360. [PubMed]
7. Waring JF, Jolly RA, Ciurlionis R, Lum PY, Praestgaard JT, Morfitt DC, Buratto B, Roberts C, Schadt E, Ulrich RG. Clustering of hepatotoxins based on mechanism of toxicity using gene expression profiles. Toxicol. Appl. Pharmacol. 2001;175:28–42. [PubMed]
8. Waring JF, Ciurlionis R, Jolly RA, Heindel M, Ulrich RG. Microarray analysis of hepatotoxins in vitro reveals a correlation between gene expression profiles and mechanisms of toxicity. Toxicol. Lett. 2001;120:359–368. [PubMed]
9. Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, Davis AP, Dolinski K, Dwight SS, Eppig JT, Harris MA, Hill DP, Issel-Tarver L, Kasarskis A, Lewis S, Matese JC, Richardson JE, Ringwald M, Rubin GM, Sherlock G. Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nat. Genet. 2000;25:25–29. [PMC free article] [PubMed]
10. Goto S, Bono H, Ogata H, Fujibuchi W, Nishioka T, Sato K, Kanehisa M. Organizing and computing metabolic pathway data in terms of binary relations. Pac. Symp. Biocomput. 1997:175–186. [PubMed]
11. Dahlquist KD, Salomonis N, Vranizan K, Lawlor SC, Conklin BR. GenMAPP, a new tool for viewing and analyzing microarray data on biological pathways. Nat. Genet. 2002;31:19–20. [PubMed]
12. Kiyosawa N, Ando Y, Manabe S, Yamoto T. Toxicogenomic biomarkers for liver toxicity. J. Toxicol. Pathol. 2009;22:35–52. [PMC free article] [PubMed]
13. Kier LD, Neft R, Tang L, Suizu R, Cook T, Onsurez K, Tiegler K, Sakai Y, Ortiz M, Nolan T, Sankar U, Li AP. Applications of microarrays with toxicologically relevant genes (tox genes) for the evaluation of chemical toxicants in Sprague Dawley rats in vivo and human hepatocytes in vitro. Mutat. Res. 2004;549:101–113. [PubMed]
14. Ellinger-Ziegelbauer H, Gmuender H, Bandenburg A, Ahr HJ. Prediction of a carcinogenic potential of rat hepatocarcinogens using toxicogenomics analysis of short-term in vivo studies. Mutat. Res. 2008;637:23–39. [PubMed]
15. Uehara T, Hirode M, Ono A, Kiyosawa N, Omura K, Shimizu T, Mizukawa Y, Miyagishima T, Nagao T, Urushidani T. A toxicogenomics approach for early assessment of potential non-genotoxic hepatocarcinogenicity of chemicals in rats. Toxicology. 2008;250:15–26. [PubMed]
16. Sawada H, Takami K, Asahi SA. Toxicogenomic approach to drug-induced phospholipidosis: Analysis of its induction mechanism and establishment of a novel in vitro screening system. Toxicol. Sci. 2005;83:282–292. [PubMed]
17. Hirode M, Ono A, Miyagishima T, Nagao T, Ohno Y, Urushidani T. Gene expression profiling in rat liver treated with compounds inducing phospholipidosis. Toxicol. Appl. Pharmacol. 2008;229:290–299. [PubMed]
18. Kiyosawa N, Ito K, Sakuma K, Niino N, Kanbori M, Yamoto T, Manabe S, Matsunuma N. Evaluation of glutathione deficiency in rat livers by microarray analysis. Biochem. Pharmacol. 2004;68:1465–1475. [PubMed]
19. Kiyosawa N, Uehara T, Gao W, Omura K, Hirode M, Shimizu T, Mizukawa Y, Ono A, Miyagishima T, Nagao T, Urushidani T. Identification of glutathione depletion-responsive genes using phorone-treated rat liver. J. Toxicol. Sci. 2007;32:469–486. [PubMed]
20. Edgar R, Domrachev M, Lash AE. Gene Expression Omnibus: NCBI gene expression and hybridization array data repository. Nucleic. Acids Res. 2002;30:207–210. [PMC free article] [PubMed]
21. Brazma A, Parkinson H, Sarkans U, Shojatalab M, Vilo J, Abeygunawardena N, Holloway E, Kapushesky M, Kemmeren P, Lara GG, Oezcimen A, Rocca-Serra P, Sansone SA. Array Express—A public repository for microarray gene expression data at the EBI. Nucleic Acids Res. 2003;31:68–71. [PMC free article] [PubMed]
22. Waters M, Stasiewicz S, Merrick BA, Tomer K, Bushel P, Paules R, Stegman N, Nehls G, Yost KJ, Johnson CH, Gustafson SF, Xirasagar S, Xiao N, Huang CC, Boyer P, Chan DD, Pan Q, Gong H, Taylor J, Choi D, Rashid A, Ahmed A, Howle R, Selkirk J, Tennant R, Fostel J. CEBS—Chemical Effects in Biological Systems: A public data repository integrating study design and toxicity data with microarray and proteomics data. Nucleic Acids Res. 2008;36:D892–D900. [PMC free article] [PubMed]
23. Mattingly CJ, Rosenstein MC, Davis AP, Colby GT, Forrest JN, Jr, Boyer JL. The comparative toxicogenomics database: A cross-species resource for building chemical-gene interaction networks. Toxicol. Sci. 2006;92:587–595. [PMC free article] [PubMed]
24. Hayes KR, Vollrath AL, Zastrow GM, McMillan BJ, Craven M, Jovanovich S, Rank DR, Penn S, Walisser JA, Reddy JK, Thomas RS, Bradfield CA. EDGE: A centralized resource for the comparison, analysis, and distribution of toxicogenomic information. Mol. Pharmacol. 2005;67:1360–1368. [PubMed]
25. Gehlenborg N, O’Donoghue SI, Baliga NS, Goesmann A, Hibbs MA, Kitano H, Kohlbacher O, Neuweger H, Schneider R, Tenenbaum D, Gavin AC. Visualization of omics data for systems biology. Nat. Methods. 2010;7:S56–S68. [PubMed]
26. Gentleman RC, Carey VJ, Bates DM, Bolstad B, Dettling M, Dudoit S, Ellis B, Gautier L, Ge Y, Gentry J, Hornik K, Hothorn T, Huber W, Iacus S, Irizarry R, Leisch F, Li C, Maechler M, Rossini AJ, Sawitzki G, Smith C, Smyth G, Tierney L, Yang JY, Zhang J. Bioconductor: Open software development for computational biology and bioinformatics. Genome Biol. 2004;5:R80. [PMC free article] [PubMed]
27. R: A Language and Environment for Statistical Computing. R Development Core Team; Vienna, Australia: 2008.
28. Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, Ramage D, Amin N, Schwikowski B, Ideker T. Cytoscape: A software environment for integrated models of biomolecular interaction networks. Genome Res. 2003;13:2498–2504. [PubMed]
29. Urushidani T. Prediction of hepatotoxicity based on the toxicogenomics database. In: Sahu S, editor. Hepatotoxicity from Genomics to in Vitro and in Vivo Models. John Wiley & Sons; Hoboken, NJ, USA: 2008. pp. 507–529.
30. Kiyosawa N, Shiwaku K, Hirode M, Omura K, Uehara T, Shimizu T, Mizukawa Y, Miyagishima T, Ono A, Nagao T, Urushidani T. Utilization of a one-dimensional score for surveying chemical-induced changes in expression levels of multiple biomarker gene sets using a large-scale toxicogenomics database. J. Toxicol. Sci. 2006;31:433–448. [PubMed]
31. Kiyosawa N, Ando Y, Watanabe K, Niino N, Manabe S, Yamoto T. Scoring multiple toxicological endpoints using a toxicogenomic database. Toxicol. Lett. 2009;188:91–97. [PubMed]
32. Mah N, Thelin A, Lu T, Nikolaus S, Kuhbacher T, Gurbuz Y, Eickhoff H, Kloppel G, Lehrach H, Mellgard B, Costello CM, Schreiber S. A comparison of oligonucleotide and cDNA-based microarray systems. Physiol. Genomics. 2004;16:361–370. [PubMed]
33. Severgnini M, Bicciato S, Mangano E, Scarlatti F, Mezzelani A, Mattioli M, Ghidoni R, Peano C, Bonnal R, Viti F, Milanesi L, De Bellis G, Battaglia C. Strategies for comparing gene expression profiles from different microarray platforms: application to a case-control experiment. Anal. Biochem. 2006;353:43–56. [PubMed]
34. Waring JF, Ulrich RG, Flint N, Morfitt D, Kalkuhl A, Staedtler F, Lawton M, Beekman JM, Suter L. Interlaboratory evaluation of rat hepatic gene expression changes induced by methapyrilene. Environ. Health Perspect. 2004;112:439–448. [PMC free article] [PubMed]
35. Chu TM, Deng S, Wolfinger R, Paules RS, Hamadeh HK. Cross-site comparison of gene expression data reveals high similarity. Environ. Health Perspect. 2004;112:449–455. [PMC free article] [PubMed]
36. Fielden MR, Nie A, McMillian M, Elangbam CS, Trela BA, Yang Y, Dunn RT, II, Dragan Y, Fransson-Stehen R, Bogdanffy M, Adams SP, Foster WR, Chen SJ, Rossi P, Kasper P, Jacobson-Kram D, Tatsuoka KS, Wier PJ, Gollub J, Halbert DN, Roter A, Young JK, Sina JF, Marlowe J, Martus HJ, Aubrecht J, Olaharski AJ, Roome N, Nioi P, Pardo I, Snyder R, Perry R, Lord P, Mattes W, Car BD. Interlaboratory evaluation of genomic signatures for predicting carcinogenicity in the rat. Toxicol. Sci. 2008;103:28–34. [PubMed]
37. Beyer RP, Fry RC, Lasarev MR, McConnachie LA, Meira LB, Palmer VS, Powell CL, Ross PK, Bammler TK, Bradford BU, Cranson AB, Cunningham ML, Fannin RD, Higgins GM, Hurban P, Kayton RJ, Kerr KF, Kosyk O, Lobenhofer EK, Sieber SO, Vliet PA, Weis BK, Wolfinger R, Woods CG, Freedman JH, Linney E, Kaufmann WK, Kavanagh TJ, Paules RS, Rusyn I, Samson LD, Spencer PS, Suk W, Tennant RJ, Zarbl H. Multicenter study of acetaminophen hepatotoxicity reveals the importance of biological endpoints in genomic analyses. Toxicol. Sci. 2007;99:326–337. [PubMed]
38. Baker VA, Harries HM, Waring JF, Duggan CM, Ni HA, Jolly RA, Yoon LW, de Souza AT, Schmid JE, Brown RH, Ulrich RG, Rockett JC. Clofibrate-induced gene expression changes in rat liver: A cross-laboratory analysis using membrane cDNA arrays. Environ. Health Perspect. 2004;112:428–438. [PMC free article] [PubMed]
39. Ulrich RG, Rockett JC, Gibson GG, Pettit SD. Overview of an interlaboratory collaboration on evaluating the effects of model hepatotoxicants on hepatic gene expression. Environ. Health Perspect. 2004;112:423–427. [PMC free article] [PubMed]
40. Manoli T, Gretz N, Grone HJ, Kenzelmann M, Eils R, Brors B. Group testing for pathway analysis improves comparability of different microarray datasets. Bioinformatics. 2006;22:2500–2506. [PubMed]
41. Ma’ayan A. Insights into the organization of biochemical regulatory networks using graph theory analyses. J. Biol. Chem. 2009;284:5451–5455. [PMC free article] [PubMed]
42. Kiyosawa N, Manabe S, Sanbuissho A, Yamoto T. Gene set-level network analysis using a toxicogenomics database. Genomics. 2010;96:39–49. [PubMed]
43. Crzegorczyk M, Husmeier D, Werhli AV. Reverse engineering gene regulatory networks with various machine learning methods. In: Emmert-Streib F, Dehmer M, editors. Analysis of Microarray Data—A Network Approach. Wiley-VCH; Hoboken, NJ, USA: 2008.
44. Lau SS, Monks TJ. The contribution of bromobenzene to our current understanding of chemically-induced toxicities. Life Sci. 1988;42:1259–1269. [PubMed]

Articles from International Journal of Molecular Sciences are provided here courtesy of Multidisciplinary Digital Publishing Institute (MDPI)