PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Sci Transl Med. Author manuscript; available in PMC 2010 December 27.
Published in final edited form as:
PMCID: PMC3010398
NIHMSID: NIHMS212935

Metabolic Signatures of Exercise in Human Plasma

Abstract

Exercise provides numerous salutary effects, but our understanding of how these occur is limited. To gain a clearer picture of exercise-induced metabolic responses, we have developed comprehensive plasma metabolite signatures by using mass spectrometry to measure over 200 metabolites before and after exercise. We identified plasma indicators of glycogenolysis (glucose-6-phosphate), tricarboxylic acid (TCA) cycle span 2 expansion (succinate, malate, and fumarate), and lipolysis (glycerol), as well as modulators of insulin sensitivity (niacinamide) and fatty acid oxidation (pantothenic acid). Metabolites that were highly correlated with fitness parameters were found in subjects undergoing acute exercise testing, marathon running, and in 302 subjects from a longitudinal cohort study. Exercise-induced increases in glycerol were strongly related to fitness levels in normal individuals and were attenuated in subjects with myocardial ischemia. A combination of metabolites that increased in plasma in response to exercise (glycerol, niacinamide, glucose-6-phosphate, pantothenate, and succinate) upregulated the expression of nur77, a transcriptional regulator of glucose utilization and lipid metabolism genes in skeletal muscle. Plasma metabolic profiles obtained during exercise provide signatures of exercise performance and cardiovascular disease susceptibility, in addition to highlighting molecular pathways that may modulate the salutary effects of exercise.

Exercise can confer cardiovascular protection (1, 2), unmask occult organ dysfunction (3), and predict disease prognosis (4, 5), but how and why these effects occur is not entirely clear. An understanding of exercise-induced metabolic changes can begin to elucidate these issues. Samples of blood and more invasive biopsies of skeletal muscle from humans and animals during exercise have revealed changes in several subsets of metabolites (6-10). High intensity exercise increases the concentrations of lactate (6) and products of adenine nucleotide catabolism (11) in skeletal muscle and plasma, reflecting heightened anaerobic metabolism and ATP turnover, respectively. In skeletal muscle, acute exercise causes changes in amino acid concentrations, including a modest uptake of glutamate and release of glutamine and alanine to promote ammonia metabolism (8), with concordant changes in plasma concentrations of these metabolites (7, 10).

Less is known about effects of exercise on the relative intramuscular and plasma concentrations of other classes of metabolites. For example, skeletal muscle biopsies have demonstrated rapid expansion of span 2 tricarboxylic acid (TCA) cycle intermediates in response to exercise (9, 12), which augments aerobic energy production in cardiac and skeletal muscle (7, 13, 14). Similarly, glycogenolysis and lipolysis increase in skeletal muscle and adipose tissue, mobilizing substrates for oxidative phosphorylation (15). However, the extent to which these tissue changes in substrate utilization can be detected in plasma in response to exercise, and whether they reflect fitness or disease status remain poorly characterized.

Emerging metabolite profiling technologies have enhanced the feasibility of acquiring high throughput “snapshots” of a whole organism's metabolic status (16-19). In addition to serving as potential biomarker signatures of disease states (20), circulating metabolites may themselves have unanticipated roles as regulatory signals with hormone-like functions (21, 22). The objective of this study was to obtain a systematic view of the metabolic response to exercise by simultaneously assaying a large and diverse set of known metabolites. We developed a targeted liquid chromatography, mass spectrometry (LC/MS)-based metabolomics platform with high analyte specificity (23, 24) and applied it to serial blood samples obtained before and after exercise so that each individual could serve as his or her own control. We defined the magnitude, kinetics, and interrelatedness of plasma metabolic changes in response to acute exercise and their relationship to changes elicited by prolonged (marathon-running) exercise. We then tested whether metabolites altered in response to exercise were correlated with a key fitness related parameter in a prospective cohort. Finally, we examined whether metabolites that changed during exercise can modulate the expression of transcriptional regulators of metabolism in cell culture and animal models.

RESULTS

We first performed metabolic profiling in subjects referred for exercise treadmill testing (ETT) who had normal exercise capacity and no evidence of myocardial ischemia (Table 1). We obtained peripheral blood samples from subjects at baseline, peak exercise, and 60 minutes after completion of ETT to characterize the alterations associated with acute maximum exercise. Because unintentional overfitting of data is a concern in biomarker discovery studies (25), we studied metabolic changes in two independent groups of subjects [i.e., a derivation cohort (N=45) and a validation cohort (N=25)]. There were no significant clinical differences between subjects in the ETT derivation and validation cohorts (Table 1). Twenty-three metabolites changed significantly (nominal P<0.005) at the peak exercise time point in the derivation cohort (Table 2 and Fig. 1), with an estimated false discovery rate of <5%, or approximately one metabolite out of the 23 metabolites that changed (see Methods). In the validation cohort, significant changes were noted for 21 of the 23 metabolites. For the overall group of metabolites in Table 2, the magnitude of change in individual metabolites in the two cohorts was highly correlated (r=0.99, P<0.0001) (Table S1 provides additional data on metabolite changes in the combined cohorts).

Figure 1
Relative changes in metabolites in response to exercise
Table 1
Clinical characteristics
Table 2
Metabolite changes in plasma at the peak exercise time point

The observed changes in plasma metabolites immediately after cessation of exercise, which occurred approximately 10 minutes after baseline sampling, reflect rapid upregulation of several metabolic pathways responsible for skeletal muscle substrate utilization (Fig. 2, top panel). The metabolite profiling platform captured previously reported increases in plasma concentrations of glycolysis products (lactate, pyruvate), lipolysis products (glycerol) and amino acids (alanine and glutamine). Plasma concentrations of the ketone body acetoacetate fell in response to acute exercise, as described (26).

Figure 2
Fuel substrate mobilization during exercise

Our broad metabolomics approach enabled detection of coordinate changes in multiple components of various metabolic pathways. For example, we documented increases in sequential products of adenine nucleotide catabolism (e.g. AMP, inosine, hypoxanthine, xanthine), including phosphorylated metabolites that are usually confined to intracellular compartments (Fig. 3A). We observed particularly prominent changes in span 2 TCA cycle intermediates (Table 2 and Fig. 3B). Notably, the changes in individual TCA cycle intermediates in plasma closely mirrored previously reported changes in skeletal muscle obtained from invasive intramuscular biopsies taken during exercise testing (9).

Figure 3
(A) Enrichment of adenine nucleotide catabolites in pulmonary arterial blood during exercise. (Left) Intramuscular and extracellular metabolic reactions in the adenine nucleotide catabolism pathway. (Right) Patterns of change of individual metabolites ...

We also detected plasma metabolic changes in pathways not previously associated with exercise. For example, niacinamide, which enhances insulin release and improves glycemic control (27, 28), increased by 79% (IQR 42 to 182%). We observed heterogeneity in niacinamide elevation in response to exercise in that leaner individuals (subjects with BMI less than the median value of 28) had a more than two-fold greater increase in niacinamide in response to exercise than did individuals with BMI above the median (102%, IQR -1 to 117% vs. 41%, IQR -7 to 80%, respectively, P=0.04). Concentrations of allantoin, a product of uric acid oxidation that has been implicated as an indicator of oxidative stress (29), fell after acute exercise. Finally, plasma concentrations of glycogenolysis intermediates (3-phosphoglycerate, glucose-6-phosphate) increased with ETT. Hierarchical clustering of metabolites to determine the interrelatedness of metabolic changes with exercise is displayed in Fig. S2.

Although prior studies have documented that heart rate and hemodynamic parameters rapidly return to baseline after acute exercise (30, 31), we observed metabolic changes that persisted 60 minutes after the cessation of exercise (Table 3). Metabolites that were changed only at the 60 minute time point included TCA cycle intermediates (citrate/isocitrate and aconitate), a homocysteine metabolism pathway intermediate (methionine), and two adenine nucleotide degradation products (xanthosine and uric acid). Other metabolites that were changed only at the 60 minute time point included uridine, a pyrimidine nucleoside, and the amino acid, ornithine. Adenine nucleotides concordantly increased incrementally at 60 minutes compared to the peak exercise time point (Fig. 3A), as did span 1 TCA intermediates citrate/isocitrate and aconitic acid, whereas span 2 TCA intermediates decreased in a concerted fashion (Fig. 3B).

Table 3
Metabolite changes in plasma at 60 minutes after completion of exercise testing. Metabolites with P<0.005 in the derivation cohort are shown. Metabolites with identical retention times and parent-daughter ion pairs (e.g. citrate and isocitrate) ...

Localization of metabolic changes through multi-site blood sampling

We performed metabolic profiling on 8 subjects who underwent comprehensive cardiopulmonary exercise testing (CPET), which included bicycle ergometry with invasive hemodynamic monitoring and multi-site blood sampling to assess the cause of shortness of breath on exertion. The CPET subjects showed normal exercise responses, although estimated peak oxygen uptake (VO2) values were lower in the CPET cohort than in the ETT cohort (Table 1), commensurate with the smaller metabolic demand imposed by bicycle ergometry CPET than by treadmill ETT (32). We compared samples from the superior vena cava (SVC), which contains blood from the non-exercising upper body, with samples from the pulmonary artery (PA), which reflects the venous effluent from the exercising lower extremity skeletal muscle as well as cardiac muscle. Prior studies suggest that there is a 50% contribution of inferior vena cava blood at rest while upright on a bicycle, and a 73% contribution during bicycle ergometry exercise (33). Thus, we could assess instantaneous metabolite gradients in distinct vascular beds and so localize the site of metabolic changes. Analysis of bicycle ergometry also allowed us to test whether our findings from treadmill ETT could be generalized to other exercise modalities.

Cycle ergometry exercise yielded very similar metabolic changes to those observed with treadmill ergometry (Fig. 1 Heatmap and Fig. 3). At peak exercise, the majority of metabolites showed significantly larger changes in PA plasma than in SVC plasma, implicating a sub-diaphragmatic skeletal muscle or cardiac source of these metabolic changes, likely from the exercising lower limbs. As expected, steep instantaneous PA-to-SVC gradients were evident for the purine degradation products (ΔPA/ΔSVC ratio range: 2.2 to 9.0, Fig. 3A) and span 2 TCA cycle intermediates (ΔPA/ΔSVC ratio range 1.7 to 2.3, Fig. 3B). By contrast, a subset of metabolites that were significantly changed in both the ETT and CPET cohorts, including several amino acids, acetoacetate, and glucose-6-phosphate, were similarly changed in the SVC and PA samples. This may be due to endocrine-like signaling effects from the exercising muscle (34). At 60 minutes after exercise, significant instantaneous PA-to-SVC gradients were no longer present for any of the metabolite classes.

Relationship between metabolic changes and fitness in ETT

We next examined whether exercise-induced excursions of metabolites that were confirmed in both the ETT derivation and validation cohorts, either at peak exercise or 60 minutes (N=28 metabolites total), were correlated with exercise performance. For these analyses, we divided the ETT cohort of individuals with overall normal exercise capacity into two groups on the basis of median percent predicted peak oxygen uptake (%pVO2). The more- and less-fit subgroups did not differ with regard to age (58±11 vs. 59±14 years), sex (94 vs. 86% male), weight (193±35 vs. 194±31 lbs) or the product of heart rate and systolic blood pressure achieved (27,500±4770 vs. 26750±6900 mmHg/min), respectively (P>0.05 for all comparisons). At peak exercise, glycerol increased to a greater extent in the group of individuals who achieved higher %pVO2 (98%, IQR 38 to 143%) compared to individuals who achieved lower %pVO2 (48%, IQR, 24 to 69%, P<0.005, Fig. 4), suggesting that more fit individuals have a greater capacity for lipolysis in response to acute exercise. The magnitude of the changes in glycerol were most closely related to %pVO2 (r=0.54, P<0.001). We further investigated glycerol excursions in a distinct cohort of subjects with and without exercise-induced myocardial ischemia, with no differences in exercise exposure or cardiac risk factors including age, sex, BMI, or diabetes status (see Table S2). Exercise-induced increases in plasma glycerol concentrations were signficantly smaller in subjects with inducible ischemia than in controls (18%, IQR -10 to 79%, vs. 66%, IQR 24 to 109% in controls, P=0.0015). The evidence for reduced exercise-induced lipolysis in the ischemic and less-fit cohorts, as indicated by the smaller elevations in glycerol, may be indicative of maladaptive responses to exercise in which lipid utilization is impaired.

Figure 4
Fitness levels and differential metabolic changes during ETT

Pantothenate also increased to a greater extent in more-fit individuals, whereas methionine excursions were greater in the less-fit subgroup (Fig. 4). Changes in glutamine concentrations were also greater in the less-fit group, likely reflecting greater skeletal muscle release of ammonia during exercise. The metabolic changes in each of these metabolites persisted 60 minutes after cessation of exercise (Fig. 4). By contrast, lactate concentrations did not differ between more and less fit individuals at either peak exercise or 60 minutes after exercise. Notably, the differential changes in glycerol, pantothenate, glutamine and methionine between more and less fit subjects persisted after we adjusted for the amount of exercise performed (Table S3).

Relationship between metabolic changes and prolonged exercise

We acquired metabolic profiles of 25 subjects who ran the 26.2 mile Boston Marathon, with an average time of 247±46 minutes. As expected, extensive changes in plasma metabolite concentrations were evident at the end of the race as compared to pre-race concentrations (Table 4 and Fig. 2). We documented marked elevations in glycerol (1,128%, IQR 897-1315%, P<0.0001) and β-hydroxybutyrate (401%, IQR 224-1060%, P<0.0001, Fig. 2), consistent with extensive lipolysis and ketone body production, respectively, in subjects who completed the marathon. In contrast to our findings after acute exercise, we saw a reduction in gluconeogenic amino acids (alanine, threonine, serine, proline, valine, histidine, glutamine, asparagine, Table 4) and unexpected increases in tryptophan metabolites (kynurenate, quinolinate, anthranilate).

Table 4
Metabolite changes following completion of a 26.2 mile marathon. Metabolites with P<0.005 are shown. Metabolites with identical retention times and parent-daughter ion pairs (e.g. citrate/isocitrate) cannot be distinguished by the platform.

To investigate whether the metabolic changes that correlated with fitness after acute exercise also correlated with fitness after prolonged exercise, we stratified marathon runners into two groups on the basis of their finish time above and below the median (240 minutes). The groups did not differ in age (faster group, 41±11 vs. 42±7 years), sex (76% vs. 69% male), or BMI (25±4 vs. 24±2, all P>0.05). Fumarate changes, which correlated with fitness during acute exercise performance, were also related to marathon speed (faster group: +200%, IQR=110 to 281% vs. +108%, IQR=16 to 137%, P<0.05). Additional span 2 TCA intermediary metabolites with greater excursions in the faster marathoners included succinate (+227%, 194 to 411%, vs. +66%, 30 to 94%) and malate (+227%, 133 to 269% vs. 87%, 49 to 109%, all P<0.05). An equally weighted sum of TCA cycle intermediate concentrations after marathon completion differentiated the faster from the slower runners (Fig. 5). There were non-significant trends toward greater increases in both pantothenate (+40%, 6 to 88% vs. +22%, -15 to 40%,) and glycerol (+1172%, 993 to 1469% vs. 986%, 827 to 1295%,) in the faster marathon group, in the same direction as was observed in acute exercise.

Figure 5
TCA intermediate changes with marathon running

In the marathon runners, other metabolites that changed during acute exercise differed in the faster and slower runners. Faster runners had more modest increases in ketone bodies (β-hydroxybutyrate, +362% vs. +855%) and in allantoin (+9% vs. +28%), a marker of oxidative stress, than did slower runners. Citrulline, which modulates arginine bioavailability for nitric oxide synthesis, showed attenuated reduction in faster runners (-14% vs. -41%), while arginosuccinate concentrations (+79% vs. +25%) and niacinamide (+253% vs. +53%) were higher in faster marathon runners (P<0.05 for all).

Metabolic predictors of fitness in a large, prospective cohort

We next determined whether baseline plasma concentrations of metabolites that were altered in response to exercise (Table 2) were associated with cardiovascular fitness in an independent cohort of subjects from the Framingham Heart Study (N=302, see Table S3). Heart rate is a known independent predictor of exercise capacity (35) that has been directly related to cardiovascular outcomes in the Framingham Heart Study (36) and was measured at the time of blood sampling in this cohort.

We specifically evaluated whether the metabolites that were modulated by acute exercise were correlated with resting heart rate in this cohort. Of the acute exercise metabolites, 12 were also measured in a subset of subjects in the Framingham Heart Study cohort (Table S5). Glycerol concentrations were significantly correlated with resting heart rate (R=0.22, P=0.0002) whereas glutamine concentrations were inversely related to heart rate (R=-0.21, P=0.0002). Both of these differences remained significant after adjustment for age, BMI, and gender (Table S5). Subjects in the fourth heart rate quartile had 1.44-fold higher glycerol concentrations than those in the first quartile (P=0.001). Glutamate concentrations in the fourth heart rate quartile tended to be higher (1.21-fold, P=0.08), while glutamine concentrations were lower (0.87-fold, P=0.03). These data are notable given the greater increase in glycerol and attenuated decrease in glutamine in response to exercise in more fit subjects in the ETT cohort.

Activation of the nur77 pathway by exercise-induced metabolites

We further hypothesized that a subset of metabolites that increased in plasma after exercise could also modulate pathways relevant to cellular respiration and substrate utilization. We performed experiments with 6 exercise-induced metabolites of biological interest, specifically assaying their effects on 11 transcriptional regulators of metabolism in cultured myotubes. We found that a mixture of physiologically relevant concentrations of glycerol, niacinamide, glucose-6-phosphate, pantothenate, and succinate rapidly upregulates the expression of nr4a1c (or nur77) (Fig. 6A), a recently described transcriptional regulator of glucose utilization and lipid metabolism genes in skeletal muscle (37, 38). No individual metabolite triggered a similar response. Consistent with these findings, nur77 expression was induced 5-fold in mouse quadriceps by 30 minutes of exercise (Fig. 6B).

Figure 6
(A) Modulation of gene expression by metabolites. (Left) mRNA expression of indicated genes (36B4: Rplp0 ribosomal protein, large, P0, HPRT: hypoxanthine guanine phosphoribosyl transferase, CYCS: cytochrome c, somatic, COX5B: cytochrome c oxidase subunit ...

DISCUSSION

Metabolomics technologies can be used to systematically define phenotypic patterns of small molecules in blood and urine (23, 24, 39). Here, we applied an LC-MS-based platform to characterize the metabolic response to exercise. As expected, we identified changes in metabolites reflecting heightened utilization of fuel substrates in several metabolic pathways, including increased glycolysis (6), lipolysis (15, 26), adenine nucleotide catabolism (11), and amino acid catabolism (8). These data corroborate prior studies examining the metabolic response to exercise, which have typically focused on one metabolite or a discrete set of identified small molecules (40, 41). By contrast, the breadth of our approach also enabled the identification of previously undescribed metabolic changes in response to exercise including increases in indicators of glycogenolysis (glucose-6-phosphate and 3-phosphoglycerate) and small molecules that reflect oxidative stress (allantoin), and that modulate insulin sensitivity (niacinamide). The sensitivity of the platform also allowed us to monitor metabolic changes that had been previously documented only by invasive skeletal muscle sampling during exercise. Rigorous clinical phenotyping further enabled us to link distinct metabolic excursions with key clinical parameters, including exercise performance, myocardial ischemia, and resting heart rate. Finally, we found that a mixture of exercise-induced metabolites rapidly triggered expression of the transcription factor nur77, which controls glucose and lipid metabolism in skeletal muscle.

Plasma metabolic profiles reflect intracellular metabolism

Our method detected metabolites in plasma typically thought to be confined to intracellular compartments because of their phosphorylation state or site of bioactivity (9, 12). Although intracellular TCA cycle and adenine nucleotide changes in muscle during exercise are well characterized (9, 11, 13), our high sensitivity LC-MS platform allowed us to examine whether these changes occurred in plasma in a fitness-dependent way. Indeed the exercise-induced changes in plasma concentrations of TCA cycle intermediates that were measured were similar to those reported in skeletal muscle after a similar amount of exercise (9). Specifically, there were increases in individual TCA cycle span 2 constituents (2-4-fold increases for fumarate, malate, and succinate), with more modest increases in citrate/isocitrate, and no change in α-ketoglutarate (9, 12). Similarly, we found plasma signatures of sequential products of purine degradation and intramuscular glycogenolysis, that had previously been documented to change within muscle (11).

Furthermore, selective catheterization during exercise allowed us to localizemetabolic changes. The pulmonary artery to superior vena cava gradients at peak exercise of both purines and TCA intermediaries are consistent with marked release of these small molecules from exercising lower extremity skeletal muscle or cardiac muscle. By contrast, excursions in amino acids, acetoacetate, and glucose-6-phosphate were all induced by exercise but similarly so in the SVC and PA samples. This result suggests that these metabolites reflect coordinated metabolic changes in both exercising and non-exercising tissues—potentially triggered by circulating hormone-like molecules (34). However, the data from selective catheterization of the SVC and PA cannot rule out rapid modulation of metabolite concentrations by organs such as the liver that would attenuate differences between PA and SVC levels of metabolites.

Performance correlates of metabolic changes

In our studies, the most-fit individuals in both the ETT and marathon cohorts showed greater augmentation of pantothenate and fumarate concentrations than less fit individuals. Pantothenate, which has not been previously associated with acute exercise performance, plays an active role in fatty acid metabolism and facilitates acetyl CoA entry into the TCA cycle. Pantothenate deficiency is rare, but when present it is associated with postural hypotension, increased heart rate, and impaired cardiac pyruvate utilization (42, 43), which may reflect a reduced capacity to augment cardiac stroke volume for a given work-load.

The span 2 TCA intermediary, fumarate, also increased to a greater extent in the more fit patients, that is, those achieving a higher percent predicted peak VO2 during acute exercise. Furthermore, increases in fumarate and other span 2 TCA intermediates were associated with faster marathon times. It has been suggested that an increase in the total concentration of intramuscular TCA intermediates is necessary to augment and maintain TCA cycle flux during exercise (9, 12). In our studies, the size of the plasma TCA intermediate pool serves as a barometer of adequate muscle TCA pool size to maintain oxidative metabolism during prolonged exercise.

We identified previously undescribed exercise-induced increases in plasma concentrations of allantoin and niacinamide. Allantoin, is a marker of oxidative stress generated in humans by nonenzymatic oxidation of uric acid by reactive oxygen species (44, 45). Allantoin concentrations increased to a greater extent in less fit individuals compared to more fit individuals during prolonged exercise, indicating attenuated oxidative stress associated with fitness. Niacinamide is an amide of niacin that participates in intracellular respiration to oxidize fuel substrates. Niacinamide regulates insulin sensitivity, and studies in small cohorts suggest that niacinamide may promote glycemic control in early-stage diabetes (27, 28). Thus, the persistent increases in niacinamide after brief exercise, which were more marked in lean individuals, and the greater increase in niacinamide among faster marathon runners, provides a possible link between acute and chronic insulin-sensitizing effects of exercise.

Fatty acids and lipids are preferred substrates for exercising muscle. Fatty acids can undergo reesterification after lipolysis within adipocytes (47). By contrast, glycerol is irreversibly liberated by lipolysis and is thus a better reporter of lipolysis in plasma (48). Plasma glycerol has been previously reported to increase in response to resistance exercise (15, 26), and our data now show a greater increase in glycerol during exercise in more-fit individuals than in those who are less fit. The lipolysis pathway is a specific indicator of fitness, since alterations in glycolysis and purine metabolites did not differ significantly between more and less fit individuals. In addition, glycerol increased less in subjects with exercise-induced myocardial ischemia, when compared to healthy subjects matched for age, sex, and exercise exposure. This result suggests that more-fit individuals have a greater capacity to activate lipolysis during physical activity than those that are less-fit and may also reflect a shift away from fatty acids as a preferred substrate in subjects with ischemic myocardium (49).

Relationship between metabolite concentrations and fitness in a large prospective cohort

Resting heart rate is a fitness metric that predicts coronary heart disease outcomes in the Framingham Heart Study and other cohorts (36, 50). Plasma glycerol concentrations directly correlated with heart rate in the Framingham Heart Study indicating that higher circulating glycerol concentrations at rest were associated with lower fitness levels. Circulating plasma glycerol originates from lipolysis in adipose tissue (51, 52). Increased adipose mass and resistance to insulin's anti-lipolytic effects may contribute to the basal concentrations of glycerol in less fit individuals, although correction for BMI did not attenuate the relationship between heart rate and glycerol. In addition, inter-individual variability in β-adrenergic signaling, which modulates lipolysis (53), may contribute to the association between glycerol and heart rate in that heightened basal β-adrenergic signaling may also be associated with increased heart rate.

Plasma glycerol measured at rest and after physical activity could potentially serve as a biomarker of fitness. Individuals who can activate lipolysis in response to various forms of exercise may be more likely to remain lean, whereas high glycerol and limited activity-induced lipolysis may be a maladaptive phenotype. This biomarker would be analogous to heart rate itself, in that fit individuals demonstrate greater heart rate augmentation with exercise but lower baseline heart rates. Whether plasma levels of glycerol or glutamine (which was also associated with heart rate in the Framingham cohort), predict cardiovascular outcomes on top of existing biomarkers requires further study.

Unanticipated role of metabolites in modulating Nur77 transcription

To begin to understand whether circulating metabolites may mediate salutary effects of exercise, we hypothesized that metabolites altered by exercise in our study would induce expression of genes involved in regulating skeletal muscle metabolism. In fact, addition of physiologically relevant concentrations of metabolites to differentiated myotubes in cell culture resulted in a rapid 3-fold induction in the transcription factor Nur77, although no single metabolite augmented gene expression. Nur77, primarily studied for its role in apoptosis, is also an important metabolic regulator of glucose (37) and lipid metabolism (38) in skeletal muscle. Nur77 has also been suggested as a potential therapeutic target for the metabolic syndrome (54). The expression of Nur77 in skeletal muscle is reduced in several models of obesity and type 2 diabetes (ob/ob, db/db, and Zucker Diabetic Fatty rats) and is increased in response to insulin-sensitizing treatments (55). The endogenous agonists that modulate Nur77 expression are poorly understood.

Limitations

The sample size of our patient cohort with multi-site sampling was limited, due to the small number of patients who undergo invasive CPET who also have normal exercise capacity and hemodynamics. Nonetheless, the direction and magnitude of the changes accompanying cycle ergometry were highly consistent with those seen in subjects undergoing ETT, thus validating our observed findings across exercise modalities. In addition, our population was relatively homogeneous because we required normal exercise tolerance for inclusion in the study, likely attenuating correlations between metabolite concentrations and fitness levels.

Conclusions

Metabolic profiling in this study provides a comprehensive snapshot of human metabolism at rest and during exercise and provides small molecule reporters of fitness. During brief exposure to exercise (~10 minutes) circulating metabolite concentrations indicate rapid activation of a catabolic program consisting of heightened lipolysis, glycolysis, and glycogenolysis, as well as amino acid and purine catabolism that largely persists for at least 60 minutes after completion of exercise. This metabolic response is in contrast to the post-prandial anabolic state in which insulin suppresses lipolysis and promotes uptake of circulating amino acid and purine metabolites (24). Overactivation of this anabolic pathway contributes to obesity and insulin resistance (56). The entire spectrum of exercise-induced metabolic changes may therefore provide a favorable counterbalance to a frequently maladaptive net-anabolic physiologic state.

The exercise-induced metabolic changes that differentiate more and less fit individuals, in particular, represent attractive candidates asmediators of the salutary effects of exercise. By serially measuring individual metabolites, we found that during exercise more fit individuals activate lipolysis (glycerol), facilitate entry of fatty acids into the TCA cycle (pantothenate), and expand the TCA cycle intermediate pool (fumarate, malate, succinate) more robustly than less fit individuals, while incurring less oxidative stress (allantoin), attenuated release of a homocysteine precursor (methionine), and greater increases in the insulin-sensitizing small molecule niacinamide. Moreover, resting basal concentrations of a subset of metabolites that are altered in a fitness-dependent manner in response to exercise (i.e. glutamine and glycerol) are also related to fitness in a large prospective cohort

The capacity of circulating metabolites altered by exercise to activate a transcription factor that regulates glucose and lipid metabolism offers further mechanistic insight into potential salutary metabolic effects of exercise in humans Exercise-induced metabolic signals may overcome the suppression of Nur 77 that has been observed in animal models of obesity and diabetes (54, 55). Ultimately, a better understanding of exercise-induced metabolic changes might help us identify the salutary effects of exercise in individuals with and without cardiovascular disease and point to targets for therapeutic modulation.

METHODS

Study Cohorts and Study Protocol

All blood sampling was performed as part of human studies protocols approved by the Institutional Review Board Subcommittee on Human Studies of the Massachusetts General Hospital (MGH). Written informed consent was obtained from all subjects.

For exercise testing protocols, we recruited outpatients referred to the MGH Exercise Laboratory for either diagnostic treadmill ETT (n=70) or bicycle ergometry cardiopulmonary exercise testing (CPET, n=8). In order to define the normal metabolic response to maximum exercise, we selected subjects who met the following inclusion criteria: 1) normal exercise tolerance as defined by estimated peak VO2 greater than 70% predicted (57, 58); 2) evident maximum effort on the basis of heart rate response greater than 85% predicted in the absence of beta-blockade; and 3) pre-exercise fasting for at least 4 hours. Exclusion criteria included cessation of exercise by the test supervisor, reversible perfusion defects or electrocardiographic evidence of exercise-induced ischemia, mechanical limitation to exercise, diabetes, or left ventricular ejection fraction less than 50%.

The myocardial ischemia cohort consisted of subjects with exercise-induced myocardial ischemia (n=65), as determined by ischemic ST-segment response to exercise (>1mm horizontal or downsloping ST-segment depression versus none) (59), and moderate-to-severe reversible myocardial perfusion defects by 99mTc-sestaMIBI imaging. Control subjects (n=65) were matched on the basis of age-, sex- and exercise duration.

The Standard Bruce Protocol was used for treadmill ETT (60). CPET was performed by coupling 10-25 W/min incremental ramp cycle ergometry with measurement of respiratory gas exchange and continuous hemodynamic monitoring as previously described (61). Rest and peak heart rate and blood pressure, duration of the stress test, and either estimated (58) or directly measured peak VO2 were recorded.

For plasma sampling before and after completion of the Boston Marathon, amateur runners scheduled to participate in the Boston Marathon were recruited by open e-mail invitation sent to local running clubs. The initial 25 responders with no history of cardiovascular disease were enrolled in the study. Sample size was limited to enable prompt acquisition of blood samples at the finish line and to avoid subject discomfort related to prolonged delay after a marathon.

Finally, subjects from the Framingham Offspring Study who were initially enrolled in a study of metabolic risk prediction (n=382), in whom a subset of the metabolites were assayed, were examined. Subjects who were taking nodal agents (i.e. beta blockers, non-dihydropyridine calcium channel blockers, and digoxin) that influence resting heart rate were excluded.

Metabolic Profiling Analysis

Samples from ETT were obtained from a peripheral venous catheter immediately before initiation of exercise, at peak exercise, and 60 minutes after completion of exercise. CPET samples were obtained at the same intervals from catheters placed in the superior vena cava (SVC) and proximal pulmonary artery (PA). Samples were collected in 5ml K2EDTA-treated tubes (Becton Dickinson). All blood samples were immediately centrifuged at 2000 g for 10 minutes to pellet cellular elements. The supernatant plasma was then aliquoted and frozen at -80°C to minimize freeze-thaw degradation. For subjects completing the Boston Marathon, peripheral venous plasma samples were obtained within 10 minutes of marathon completion, immediately centrifuged, placed on ice and then transferred to -80°C within 120 minutes.

For metabolite profiling we incorporated metabolites that were potentially relevant to cardiovascular and metabolic disease and amenable to measurement by liquid chromatography mass spectrometry (LC-MS) as previously described (23). The detailed description of our LC-MS methodology, which was applied in this study, has been previously described (23) and is summarized in the Supplemental Methods section.

Gene Expression Profiling Experiments

Cell Culture

C2C12 cells were differentiated into myotubes for 7 days. Cells were then placed in Hanks Buffered Salt Solution for 24 hrs and treated with the pooled metabolites (succinate 9ug/ml, pantothenate 6ug/ml, niacinamide 0.3ug/ml, glucose-6-phosphate 45ug/ml, and glycerol 30ug/ml, to mimic relative plasma concentrations) (62) for 60 minutes. Cells were washed once in phosphate buffered saline and mRNA was isolated using a TurboCapture method (Qiagen, Valencia, CA). RNA was reverse transcribed, and qPCR performed, using an ABI 7700HT qPCR machine. Gene expression was normalized to expression of TBP.

Exercise in Mice

Mice were tested by forced running on motor-driven treadmills (Columbus Instruments). Mice were first acclimated for 5 minutes a day for three days, at a low rate of 14 meters/min and 0% incline. On the test day (day 4), the treadmill was set to a constant 10% incline and started at 10 meters/minute. Every two minutes, the speed was then increased by 2 meters/minute, and the mice were forced to run to exhaustion. Exhaustion was determined by the unwillingness of mice to keep running on the treadmill, despite stimulus by a small electric shock on the stationary platform of the treadmill. Once determined to be exhausted, mice were returned to their cage, and, after the indicated times, euthanized. Quadriceps was removed, and total RNA was isolated using the Trizol method. RNA was reverse transcribed, and qPCR performed, using an ABI 7700HT qPCR machine. Expression is normalized to expression of TBP.

Statistical Analysis

For clinical characteristics of subjects undergoing exercise in each cohort, continuous variables were compared using Student's t Test, and categorical variables were compared with Fisher's Exact Test. From the 70 subjects from whom peripheral samples were collected in the ETT study, 45 subjects were randomly selected for analysis as a derivation set. The significance of change in metabolite concentrations from pre-test to post-test values was assessed either by paired Student's t Test or Wilcoxon Signed-Rank Test, as appropriate. We used a significance threshold of P<0.005 in the derivation cohort, because this threshold would be expected to yield approximately one false positive discovery from 210 analyzed metabolites, assuming independent hypotheses. Conservatively high estimates of false discovery rate were obtained via the Benjamini-Hochberg procedure.(63) The procedure was applied to the set of 210 measured metabolites for the comparisons of peak and post-exercise with baseline, as well as post-marathon with baseline.

Metabolites that changed significantly at peak exercise or at the 60-minute time points in the ETT derivation cohort were selected as candidate exercise biomarkers for testing in the ETT validation cohort (25 patients) and the CPET validation cohort (8 patients). Criteria for validation were P<0.05 by Wilcoxon signed-rank test with the direction of change concordant with that observed in the derivation cohort. The conjoint probability of 0.005 × 0.05 = 2.5×10-4 approximates a Bonferroni correction (0.05/210 = 2.4 × 10-4). The relationship between changes in metabolites in the derivation and validation cohorts was assessed with a Spearman rank correlation coefficient. The marathon cohort was used only as a derivation group (n=25). For this group we applied the same significance threshold that was used for the ETT derivation group (P<0.005).

Further analysis was carried out in the subgroup of 8 patients undergoing CPET with simultaneous plasma sampling from the SVC and PA at baseline, peak exercise, and 60 minutes post-exercise in order to localize metabolite changes. During cycle ergometry lower-extremity exercise, there is a disproportionately large contribution of inferior vena cava blood (reflecting skeletal muscle venous effluent) compared to superior vena cava blood that constitutes mixed venous blood in the pulmonary artery. Previously reported relative contributions of IVC and SVC blood flow to overall venous return is 73% and 27%, respectively, during cycle ergometry (33). Therefore, to identify 2-fold IVC vs. SVC enrichment patterns for metabolic changes, we used criteria consisting of: a PA-to-SVC ratio of greater than 1.73:1 (2×0.73 + 1×0.27), and P<0.05 by Wilcoxon signed-rank test for relative changes in metabolites in the SVC and PA.

Metabolic excursions in subsets of patients stratified by median percent predicted peak VO2 or by finish time were compared by Wilcoxon signed-rank tests. Similarity between metabolites in terms of similar profiles of change across many subjects was assessed by hierarchical clustering. Pearson or Spearman correlation coefficients are reported depending on whether or not data was normally distributed.

STATA (Version 10.0, College Station, TX) and SAS version 9.1.3 (SAS Institute) software was used to perform statistical analyses. The authors had full access to and take full responsibility for the integrity of the data. All authors have read and agree to the manuscript as written.

Supplementary Material

1

List of Supplementary Material

Materials and Methods

Fig. S1. Relationship between plasma lactate concentrations measured by liquid chromatography-mass spectrometry and the MGH clinical chemistry laboratory.

Fig. 2. Dendrogram illustrating hierarchical clustering of metabolites that changed with exercise.

Table S1. Metabolites that change within plasma in response to exercise in the entire exercise tolerance test cohort (N=70, P<0.05).

Table S2. Clinical characteristics of subjects with exercise-induced myocardial ischemia and control subjects without inducible ischemia.

Table S3. Differential changes in metabolite concentrations per MET achieved during exercise tolerance testing, among more and less fit individuals.

Table S4. Clinical characteristics of Framingham Heart Study participants in whom metabolites were measured.

Table S5. Relationship between plasma metabolite concentrations and resting heart rate in Framingham Heart Study participants.

References

Acknowledgements

We thank the staff in the MGH Exercise Testing Laboratories and the individuals who assisted with orchestrating plasma sampling in the Boston Marathon study.

Funding: The authors gratefully acknowledge support from the NIH (K23HL091106 to GDL, R01 HL072872 to REG and MSS; U01HL083141 to REG, MSS, and FPR, and R01DK081572 to REG), the Donald W. Reynolds Foundation (to REG and MSS), Fondation Leducq (to REG), American Heart Association Fellow-to-Faculty Award (GDL), and Established Investigator Award (REG). This work was also supported by the National Heart, Lung and Blood Institute's Framingham Heart Study (Contract No. N01-HC-25195). AA was supported by a pre-doctoral award from the Sarnoff Cardiovascular Research Foundation. FPR was also supported in part by NIH/NHGRI grants HG003224, HG0017115, NS054052, HG004233 and by the Keck Foundation.

Footnotes

Competing interests: The authors declare that they have no competing interests.

References and Notes

1. Thompson PD, Buchner D, Pina IL, Balady GJ, Williams MA, Marcus BH, Berra K, Blair SN, Costa F, Franklin B, Fletcher GF, Gordon NF, Pate RR, Rodriguez BL, Yancey AK, Wenger NK. Circulation. 2003;107:3109. [PubMed]
2. Gibbons RJ, Balady GJ, Bricker JT, Chaitman BR, Fletcher GF, Froelicher VF, Mark DB, McCallister BD, Mooss AN, O'Reilly MG, Winters WL, Jr., Antman EM, Alpert JS, Faxon DP, Fuster V, Gregoratos G, Hiratzka LF, Jacobs AK, Russell RO, Smith SC., Jr. Circulation. 2002;106:1883. [PubMed]
3. McNeer JF, Margolis JR, Lee KL, Kisslo JA, Peter RH, Kong Y, Behar VS, Wallace AG, McCants CB, Rosati RA. Circulation. 1978;57:64. [PubMed]
4. Myers J, Prakash M, Froelicher V, Do D, Partington S, Atwood JE. N Engl J Med. 2002;346:793. [PubMed]
5. Snader CE, Marwick TH, Pashkow FJ, Harvey SA, Thomas JD, Lauer MS. J Am Coll Cardiol. 1997;30:641. [PubMed]
6. Van Hall G, Jensen-Urstad M, Rosdahl H, Holmberg HC, Saltin B, Calbet JA. Am J Physiol Endocrinol Metab. 2003;284:E193. [PubMed]
7. Sahlin K, Katz A, Broberg S. Am J Physiol. 1990;259:C834. [PubMed]
8. Henriksson J. J Exp Biol. 1991;160:149. [PubMed]
9. Gibala MJ, MacLean DA, Graham TE, Saltin B. Am J Physiol. 1998;275:E235. [PubMed]
10. Eriksson LS, Broberg S, Bjorkman O, Wahren J. Clin Physiol. 1985;5:325. [PubMed]
11. Sahlin K, Gorski J, Edstrom L. Am J Physiol. 1990;259:C409. [PubMed]
12. Gibala MJ, MacLean DA, Graham TE, Saltin B. J Physiol. 1997;502(Pt 3):703. [PubMed]
13. Hiltunen JK, Davis EJ. Biochim Biophys Acta. 1981;678:115. [PubMed]
14. Aragon JJ, Lowenstein JM. Eur J Biochem. 1980;110:371. [PubMed]
15. Ormsbee MJ, Thyfault JP, Johnson EA, Kraus RM, Choi MD, Hickner RC. J Appl Physiol. 2007;102:1767. [PubMed]
16. Nicholson JK, Wilson ID. Nat Rev Drug Discov. 2003;2:668. [PubMed]
17. Raamsdonk LM, Teusink B, Broadhurst D, Zhang N, Hayes A, Walsh MC, Berden JA, Brindle KM, Kell DB, Rowland JJ, Westerhoff HV, van Dam K, Oliver SG. Nat Biotechnol. 2001;19:45. [PubMed]
18. Allen J, Davey HM, Broadhurst D, Heald JK, Rowland JJ, Oliver SG, Kell DB. Nat Biotechnol. 2003;21:692. [PubMed]
19. An J, Muoio DM, Shiota M, Fujimoto Y, Cline GW, Shulman GI, Koves TR, Stevens R, Millington D, Newgard CB. Nat Med. 2004;10:268. [PubMed]
20. Sabatine MS, Liu E, Morrow DA, Heller E, McCarroll R, Wiegand R, Berriz GF, Roth FP, Gerszten RE. Circulation. 2005;112:3868. [PubMed]
21. Sapieha P, Sirinyan M, Hamel D, Zaniolo K, Joyal JS, Cho JH, Honore JC, Kermorvant-Duchemin E, Varma DR, Tremblay S, Leduc M, Rihakova L, Hardy P, Klein WH, Mu X, Mamer O, Lachapelle P, Di Polo A, Beausejour C, Andelfinger G, Mitchell G, Sennlaub F, Chemtob S. Nat Med. 2008;14:1067. [PubMed]
22. He W, Miao FJ, Lin DC, Schwandner RT, Wang Z, Gao J, Chen JL, Tian H, Ling L. Nature. 2004;429:188. [PubMed]
23. Lewis GD, Wei R, Liu E, Yang E, Shi X, Martinovic M, Farrell L, Asnani A, Cyrille M, Ramanathan A, Shaham O, Berriz G, Lowry PA, Palacios IF, Tasan M, Roth FP, Min J, Baumgartner C, Keshishian H, Addona T, Mootha VK, Rosenzweig A, Carr SA, Fifer MA, Sabatine MS, Gerszten RE. J Clin Invest. 2008;118:3503. [PMC free article] [PubMed]
24. Shaham O, Wei R, Wang TJ, Ricciardi C, Lewis GD, Vasan RS, Carr SA, Thadhani R, Gerszten RE, Mootha VK. Mol Syst Biol. 2008;4:214. [PMC free article] [PubMed]
25. Ransohoff DF. Nat Rev Cancer. 2004;4:309. [PubMed]
26. Goto K, Ishii N, Sugihara S, Yoshioka T, Takamatsu K. Med Sci Sports Exerc. 2007;39:308. [PubMed]
27. Crino A, Schiaffini R, Manfrini S, Mesturino C, Visalli N, Beretta Anguissola G, Suraci C, Pitocco D, Spera S, Corbi S, Matteoli MC, Patera IP, Manca Bitti ML, Bizzarri C, Pozzilli P. Eur J Endocrinol. 2004;150:719. [PubMed]
28. Pociot F, Reimers JI, Andersen HU. Diabetologia. 1993;36:574. [PubMed]
29. Grootveld M, Halliwell B. Biochem J. 1987;243:803. [PubMed]
30. Ellis K, Pothier CE, Blackstone EH, Lauer MS. Am Heart J. 2004;147:287. [PubMed]
31. Carnethon MR, Jacobs DR, Jr., Sidney S, Sternfeld B, Gidding SS, Shoushtari C, Liu K. Med Sci Sports Exerc. 2005;37:606. [PubMed]
32. Wasserman K, Hansen J, Sue D, Stringer W, Whipp B, editors. Principles of Exercise Testing and Interpretation. ed. 4. Lippincott Williams and Wilkins; Philadelphia, Pa: 2005. p. 217. 4.
33. Cheng CP, Herfkens RJ, Taylor CA. Am J Physiol Heart Circ Physiol. 2003;284:H1161. [PubMed]
34. Pedersen BK. J Appl Physiol. 2009;107:1006. [PubMed]
35. Laukkanen JA, Laaksonen D, Lakka TA, Savonen K, Rauramaa R, Makikallio T, Kurl S. Am J Cardiol. 2009;103:1598. [PubMed]
36. Kannel WB, Kannel C, Paffenbarger RS, Jr., Cupples LA. Am Heart J. 1987;113:1489. [PubMed]
37. Chao LC, Zhang Z, Pei L, Saito T, Tontonoz P, Pilch PF. Mol Endocrinol. 2007;21:2152. [PMC free article] [PubMed]
38. Maxwell MA, Cleasby ME, Harding A, Stark A, Cooney GJ, Muscat GE. J Biol Chem. 2005;280:12573. [PubMed]
39. Sreekumar A, Poisson LM, Rajendiran TM, Khan AP, Cao Q, Yu J, Laxman B, Mehra R, Lonigro RJ, Li Y, Nyati MK, Ahsan A, Kalyana-Sundaram S, Han B, Cao X, Byun J, Omenn GS, Ghosh D, Pennathur S, Alexander DC, Berger A, Shuster JR, Wei JT, Varambally S, Beecher C, Chinnaiyan AM. Nature. 2009;457:910. [PMC free article] [PubMed]
40. Chorell E, Moritz T, Branth S, Antti H, Svensson M. J Proteome Res. 2009 [PubMed]
41. Pohjanen E, Thysell E, Jonsson P, Eklund C, Silfver A, Carlsson IB, Lundgren K, Moritz T, Svensson MB, Antti H. J Proteome Res. 2007;6:2113. [PubMed]
42. Olson RE, Stare FJ. J Biol Chem. 1951;190:149. [PubMed]
43. paper presented at the Joint Food and Agriculture Organization/World Health Organization expert consultation; Bangkok, Thailand. 1998.
44. Tam LS, Li EK, Leung VY, Griffith JF, Benzie IF, Lim PL, Whitney B, Lee VW, Lee KK, Thomas GN, Tomlinson B. J Rheumatol. 2005;32:275. [PubMed]
45. Lagendijk J, Ubbink JB, Vermaak WJ. J Chromatogr Sci. 1995;33:186. [PubMed]
46. Doehner W, Schoene N, Rauchhaus M, Leyva-Leon F, Pavitt DV, Reaveley DA, Schuler G, Coats AJ, Anker SD, Hambrecht R. Circulation. 2002;105:2619. [PubMed]
47. Shapiro B, Chowers I, Rose G. Biochim Biophys Acta. 1957;23:115. [PubMed]
48. Vaughan M. J Biol Chem. 1962;237:3354. [PubMed]
49. Taegtmeyer H. Curr Probl Cardiol. 1994;19:59. [PubMed]
50. Nauman J, Nilsen TI, Wisloff U, Vatten LJ. J Epidemiol Community Health. 64:175. [PubMed]
51. Nurjhan N, Kennedy F, Consoli A, Martin C, Miles J, Gerich J. Metabolism. 1988;37:386. [PubMed]
52. Nurjhan N, Consoli A, Gerich J. J Clin Invest. 1992;89:169. [PMC free article] [PubMed]
53. Jocken JW, Blaak EE. Physiol Behav. 2008;94:219. [PubMed]
54. Smith AG, Muscat GE. Am J Physiol Cell Physiol. 2006;291:C203. [PubMed]
55. Lessard SJ, Rivas DA, Chen ZP, van Denderen BJ, Watt MJ, Koch LG, Britton SL, Kemp BE, Hawley JA. Endocrinology. 2009;150:4883. [PubMed]
56. Cnop M. Biochem Soc Trans. 2008;36:348. [PubMed]
57. Hansen JE, Sue DY, Wasserman K. Am Rev Respir Dis. 1984;129:S49. [PubMed]
58. Bruce RA, Kusumi F, Hosmer D. Am Heart J. 1973;85:546. [PubMed]
59. Mark DB, Hlatky MA, Harrell FE, Jr., Lee KL, Califf RM, Pryor DB. Ann Intern Med. 1987;106:793. [PubMed]
60. Fletcher GF, Balady GJ, Amsterdam EA, Chaitman B, Eckel R, Fleg J, Froelicher VF, Leon AS, Pina IL, Rodney R, Simons-Morton DA, Williams MA, Bazzarre T. Circulation. 2001;104:1694. [PubMed]
61. Lewis GD, Lachmann J, Camuso J, Lepore JJ, Shin J, Martinovic ME, Systrom DM, Bloch KD, Semigran MJ. Circulation. 2007;115:59. [PubMed]
62. Wishart DS, Tzur D, Knox C, Eisner R, Guo AC, Young N, Cheng D, Jewell K, Arndt D, Sawhney S, Fung C, Nikolai L, Lewis M, Coutouly MA, Forsythe I, Tang P, Shrivastava S, Jeroncic K, Stothard P, Amegbey G, Block D, Hau DD, Wagner J, Miniaci J, Clements M, Gebremedhin M, Guo N, Zhang Y, Duggan GE, Macinnis GD, Weljie AM, Dowlatabadi R, Bamforth F, Clive D, Greiner R, Li L, Marrie T, Sykes BD, Vogel HJ, Querengesser L. Nucleic Acids Res. 2007;35:D521. [PMC free article] [PubMed]
63. Benjamini Y, Hochberg Y. Journal of the Royal Statistical Society, Series B. 1995;57:289.