BACKGROUND AND OBJECTIVE:
Optimal treatment decisions in children require sufficient evidence on the safety and efficacy of pharmaceuticals in pediatric patients. However, there is concern that not enough trials are conducted in children and that pediatric trials differ from those performed in adults. Our objective was to measure the prevalence of pediatric studies among clinical drug trials and compare trial characteristics and quality indicators between pediatric and adult drug trials.
For conditions representing a high burden of pediatric disease, we identified all drug trials registered in ClinicalTrials.gov with start dates between 2006 and 2011 and tracked the resulting publications. We measured the proportion of pediatric trials and subjects for each condition and compared pediatric and adult trial characteristics and quality indicators.
For the conditions selected, 59.9% of the disease burden was attributable to children, but only 12.0% (292/2440) of trials were pediatric (P < .001). Among pediatric trials, 58.6% were conducted without industry funding compared with 35.0% of adult trials (P < .001).
Fewer pediatric compared with adult randomized trials examined safety outcomes (10.1% vs 16.9%, P = .008). Pediatric randomized trials were slightly more likely to be appropriately registered before study start (46.9% vs 39.3%, P = .04) and had a modestly higher probability of publication in the examined time frame (32.8% vs 23.2%, P = .04).
There is substantial discrepancy between pediatric burden of disease and the amount of clinical trial research devoted to pediatric populations. This may be related in part to trial funding, with pediatric trials relying primarily on government and nonprofit organizations.
clinical trials; evidence-based medicine; pediatrics; medication use; research subjects
The evaluation of 160 meta-analyses of animal studies on potential treatments for neurological disorders reveals that the number of statistically significant results was too large to be true, suggesting biases.
Animal studies generate valuable hypotheses that lead to the conduct of preventive or therapeutic clinical trials. We assessed whether there is evidence for excess statistical significance in results of animal studies on neurological disorders, suggesting biases. We used data from meta-analyses of interventions deposited in Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies (CAMARADES). The number of observed studies with statistically significant results (O) was compared with the expected number (E), based on the statistical power of each study under different assumptions for the plausible effect size. We assessed 4,445 datasets synthesized in 160 meta-analyses on Alzheimer disease (n = 2), experimental autoimmune encephalomyelitis (n = 34), focal ischemia (n = 16), intracerebral hemorrhage (n = 61), Parkinson disease (n = 45), and spinal cord injury (n = 2). 112 meta-analyses (70%) found nominally (p≤0.05) statistically significant summary fixed effects. Assuming the effect size in the most precise study to be a plausible effect, 919 out of 4,445 nominally significant results were expected versus 1,719 observed (p<10−9). Excess significance was present across all neurological disorders, in all subgroups defined by methodological characteristics, and also according to alternative plausible effects. Asymmetry tests also showed evidence of small-study effects in 74 (46%) meta-analyses. Significantly effective interventions with more than 500 animals, and no hints of bias were seen in eight (5%) meta-analyses. Overall, there are too many animal studies with statistically significant results in the literature of neurological disorders. This observation suggests strong biases, with selective analysis and outcome reporting biases being plausible explanations, and provides novel evidence on how these biases might influence the whole research domain of neurological animal literature.
Studies have shown that the results of animal biomedical experiments fail to translate into human clinical trials; this could be attributed either to real differences in the underlying biology between humans and animals, to shortcomings in the experimental design, or to bias in the reporting of results from the animal studies. We use a statistical technique to evaluate whether the number of published animal studies with “positive” (statistically significant) results is too large to be true. We assess 4,445 animal studies for 160 candidate treatments of neurological disorders, and observe that 1,719 of them have a “positive” result, whereas only 919 studies would a priori be expected to have such a result. According to our methodology, only eight of the 160 evaluated treatments should have been subsequently tested in humans. In summary, we judge that there are too many animal studies with “positive” results in the neurological disorder literature, and we discuss the reasons and potential remedies for this phenomenon.
Meta-analyses are increasingly popular. It is unknown whether this popularity is driven by specific countries and specific meta-analyses types. PubMed was used to identify meta-analyses since 1995 (last update 9/1/2012) and catalogue their types and country of origin. We focused more on meta-analyses from China (the current top producer of meta-analyses) versus the USA (top producer until recently). The annual number of meta-analyses from China increased 40-fold between 2003 and 2011 versus 2.4-fold for the USA. The growth of Chinese meta-analyses was driven by genetics (110-fold increase in 2011 versus 2003). The HuGE Navigator identified 612 meta-analyses of genetic association studies published in 2012 from China versus only 109 from the USA. We compared in-depth 50 genetic association meta-analyses from China versus 50 from USA in 2012. Meta-analyses from China almost always used only literature-based data (92%), and focused on one or two genes (94%) and variants (78%) identified with candidate gene approaches (88%), while many USA meta-analyses used genome-wide approaches and raw data. Both groups usually concluded favorably for the presence of genetic associations (80% versus 74%), but nominal significance (P<0.05) typically sufficed in the China group. Meta-analyses from China typically neglected genome-wide data, and often included candidate gene studies published in Chinese-language journals. Overall, there is an impressive rise of meta-analyses from China, particularly on genetic associations. Since most claimed candidate gene associations are likely false-positives, there is an urgent global need to incorporate genome-wide data and state-of-the art statistical inferences to avoid a flood of false-positive genetic meta-analyses.
personalized medicine; coronary artery disease; genotyping; SNP; preventive cardiology
We propose guidelines to evaluate the cumulative evidence of gene–environment (G × E) interactions in the causation of human cancer. Our approach has its roots in the HuGENet and IARC Monographs evaluation processes for genetic and environmental risk factors, respectively, and can be applied to common chronic diseases other than cancer. We first review issues of definitions of G × E interactions, discovery and modelling methods for G × E interactions, and issues in systematic reviews of evidence for G × E interactions, since these form the foundation for appraising the credibility of evidence in this contentious field. We then propose guidelines that include four steps: (i) score the strength of the evidence for main effects of the (a) environmental exposure and (b) genetic variant; (ii) establish a prior score category and decide on the pattern of interaction to be expected; (iii) score the strength of the evidence for interaction between the environmental exposure and the genetic variant; and (iv) examine the overall plausibility of interaction by combining the prior score and the strength of the evidence and interpret results. We finally apply the scheme to the interaction between NAT2 polymorphism and tobacco smoking in determining bladder cancer risk.
Genetics; environment; interactions; evaluations
There are no analyses of citations to books on epidemiological and statistical methods in the biomedical literature. Such analyses may shed light on how concepts and methods changed while biomedical research evolved. Our aim was to analyze the number and time trends of citations received from biomedical articles by books on epidemiological and statistical methods, and related disciplines.
Methods and Findings
The data source was the Web of Science. The study books were published between 1957 and 2010. The first year of publication of the citing articles was 1945. We identified 125 books that received at least 25 citations. Books first published in 1980–1989 had the highest total and median number of citations per year. Nine of the 10 most cited texts focused on statistical methods. Hosmer & Lemeshow's Applied logistic regression received the highest number of citations and highest average annual rate. It was followed by books by Fleiss, Armitage, et al., Rothman, et al., and Kalbfleisch and Prentice. Fifth in citations per year was Sackett, et al., Evidence-based medicine. The rise of multivariate methods, clinical epidemiology, or nutritional epidemiology was reflected in the citation trends. Educational textbooks, practice-oriented books, books on epidemiological substantive knowledge, and on theory and health policies were much less cited. None of the 25 top-cited books had the theoretical or sociopolitical scope of works by Cochrane, McKeown, Rose, or Morris.
Books were mainly cited to reference methods. Books first published in the 1980s continue to be most influential. Older books on theory and policies were rooted in societal and general medical concerns, while the most modern books are almost purely on methods.
MicroRNA (miR) expression may have prognostic value for many types of cancers. However, the miR literature comprises many small studies. We systematically reviewed and synthesized the evidence.
Using MEDLINE (last update December 2010), we identified English language studies that examined associations between miRs and cancer prognosis using tumor specimens for more than 10 patients during classifier development. We included studies that assessed a major clinical outcome (nodal disease, disease progression, response to therapy, metastasis, recurrence, or overall survival) in an agnostic fashion using either polymerase chain reaction or hybridized oligonucleotide microarrays.
Forty-six articles presenting results on 43 studies pertaining to 20 different types of malignancy were eligible for inclusion in this review. The median study size was 65 patients (interquartile range [IQR] = 34–129), the median number of miRs assayed was 328 (IQR = 250–470), and overall survival or recurrence were the most commonly measured outcomes (30 and 19 studies, respectively). External validation was performed in 21 studies, 20 of which reported at least one nominally statistically significant result for a miR classifier. The median hazard ratio for poor outcome in externally validated studies was 2.52 (IQR = 2.26–5.40). For all classifier miRs in studies that evaluated overall survival across diverse malignancies, the miRs most frequently associated with poor outcome after accounting for differences in miR assessment due to platform type were let-7 (decreased expression in patients with cancer) and miR 21 (increased expression).
MiR classifiers show promising prognostic associations with major cancer outcomes and specific miRs are consistently identified across diverse studies and platforms. These types of classifiers require careful external validation in large groups of cancer patients that have adequate protection from bias. –
Diseases such as type 2 diabetes (T2D) result from environmental and genetic factors, and risk varies considerably in the population. T2D-related genetic loci discovered to date explain only a small portion of the T2D heritability. Some heritability may be due to gene–environment interactions. However, documenting these interactions has been difficult due to low availability of concurrent genetic and environmental measures, selection bias, and challenges in controlling for multiple hypothesis testing. Through genome-wide association studies (GWAS), investigators have identified over 90 single nucleotide polymorphisms (SNPs) associated to T2D. Using a method analogous to GWAS [environment-wide association study (EWAS)], we found five environmental factors associated with the disease. By focusing on risk factors that emerge from GWAS and EWAS, it is possible to overcome difficulties in uncovering gene–environment interactions. Using data from the National Health and Nutrition Examination Survey (NHANES), we screened 18 SNPs and 5 serum-based environmental factors for interaction in association to T2D. We controlled for multiple hypotheses using false discovery rate (FDR) and Bonferroni correction and found four interactions with FDR <20 %. The interaction between rs13266634 (SLC30A8) and trans-β-carotene withstood Bonferroni correction (corrected p = 0.006, FDR <1.5 %). The per-risk-allele effect sizes in subjects with low levels of trans-β-carotene were 40 % greater than the marginal effect size [odds ratio (OR) 1.8, 95 % CI 1.3–2.6]. We hypothesize that impaired function driven by rs13266634 increases T2D risk when combined with serum levels of nutrients. Unbiased consideration of environmental and genetic factors may help identify larger and more relevant effect sizes for disease associations.
Electronic supplementary material
The online version of this article (doi:10.1007/s00439-012-1258-z) contains supplementary material, which is available to authorized users.
Two recent studies identified a mutation (p.Asp620Asn) in the vacuolar protein sorting 35 gene as a cause for an autosomal dominant form of Parkinson disease . Although additional missense variants were described, their pathogenic role yet remains inconclusive.
Methods and results
We performed the largest multi-center study to ascertain the frequency and pathogenicity of the reported vacuolar protein sorting 35 gene variants in more than 15,000 individuals worldwide. p.Asp620Asn was detected in 5 familial and 2 sporadic PD cases and not in healthy controls, p.Leu774Met in 6 cases and 1 control, p.Gly51Ser in 3 cases and 2 controls. Overall analyses did not reveal any significant increased risk for p.Leu774Met and p.Gly51Ser in our cohort.
Our study apart from identifying the p.Asp620Asn variant in familial cases also identified it in idiopathic Parkinson disease cases, and thus provides genetic evidence for a role of p.Asp620Asn in Parkinson disease in different populations worldwide.
Parkinson-s disease; Genome-wide; Genetics; Genetic epidemiology; Complex traits
Since 2007, genome-wide association (GWA) studies have identified numerous well-supported, novel genetic risk loci for common cancers; however, there are concerns that this technology is reaching its limits. We provide an overview of GWA-identified genetic associations with solid tumors. We simulated the distribution of population risk alleles for colorectal, prostate, testicular, and thyroid cancers based on genetic variants identified in GWA studies. We also evaluated whether statistical power to detect typical genetic effects could be improved with studies performing GWA analyses of all available samples rather than multistage designs. Fifty-six eligible articles yielded 92 eligible associations between cancer phenotypes and genetic variants with a median per-allele odds ratio (OR) of 1.22 (interquartile range = 1.15–1.36). Half of the associations pertained to prostate, colorectal, or breast cancer. Individuals at the upper quartile of simulated risk had only 2.1- to 4.2-fold higher relative risk than those in the lower quartile. Comprehensive evaluation of currently available samples with GWA platforms would yield few additional variants with per-allele OR = 1.4, but many more variants with OR = 1.2 could be detected; statistical power to detect weak associations (OR = 1.07) would still be negligible. The GWA approach is effective in identifying common genetic variants with moderate effect; however, identifying loci with very small effects and rare variants will require major new efforts. At present, the utility of GWA-identified risk loci in risk stratification for cancer is limited.
Pain influences sleep and vice versa. We performed an umbrella review of meta-analyses on treatments for diverse conditions in order to examine whether diverse medical treatments for different conditions have similar or divergent effects on pain and sleep.
We searched published systematic reviews with meta-analyses in the Cochrane Database of Systematic Reviews until October 20, 2011. We identified randomized trials (or meta-analyses thereof, when >1 trial was available) where both pain and sleep outcomes were examined. Pain outcomes were categorized as headache, musculoskeletal, abdominal, pelvic, generic or other pain. Sleep outcomes included insomnia, sleep disruption, and sleep disturbance. We estimated odds ratios for all outcomes and evaluated the concordance in the direction of effects between sleep and various types of pain and the correlation of treatment effects between sleep and pain outcomes.
151 comparisons with 385 different trials met our eligibility criteria. 96 comparisons had concordant direction of effects between each pain outcome and sleep, while in 55 the effect estimates were in opposite directions (P<0.0001). In the 20 comparisons with largest amount of evidence, the experimental drug always had worse sleep outcomes and tended to have worse pain outcomes in 17/20 cases. For headache and musculoskeletal pain, 69 comparisons showed concordant direction of effects with sleep outcomes and 36 showed discordant direction (P<0.0001). For the other 4 pain types there were overall 27 vs. 19 pairs with concordant vs. discordant direction of effects (P = 0.095). There was a weak correlation of the treatment effect sizes for sleep vs. headache/musculoskeletal pain (r = 0.17, P = 0.092).
Medical interventions tend to have effects in the same direction for pain and sleep outcomes, but exceptions occur. Concordance is primarily seen for sleep and headache or musculoskeletal pain where many drugs may both disturb sleep and cause pain.
Genetic effects for common variants affecting complex disease risk are subtle. Single genome-wide association (GWA) studies are typically underpowered to detect these effects, and combination of several GWA data sets is needed to enhance discovery. The authors investigated the properties of the discovery process in simulated cumulative meta-analyses of GWA study-derived signals allowing for potential genetic model misspecification and between-study heterogeneity. Variants with null effects on average (but also between-data set heterogeneity) could yield false-positive associations with seemingly homogeneous effects. Random effects had higher than appropriate false-positive rates when there were few data sets. The log-additive model had the lowest false-positive rate. Under heterogeneity, random-effects meta-analyses of 2–10 data sets averaging 1,000 cases/1,000 controls each did not increase power, or the meta-analysis was even less powerful than a single study (power desert). Upward bias in effect estimates and underestimation of between-study heterogeneity were common. Fixed-effects calculations avoided power deserts and maximized discovery of association signals at the expense of much higher false-positive rates. Therefore, random- and fixed-effects models are preferable for different purposes (fixed effects for initial screenings, random effects for generalizability applications). These results may have broader implications for the design and interpretation of large-scale multiteam collaborative studies discovering common gene variants.
epidemiology; genetics; genome-wide association study; Human Genome Project; meta-analysis; models, genetic; polymorphism, single nucleotide
Fifteen meta-analyses have been published between 1995 and 2011 to evaluate the efficacy/effectiveness and harms of diverse influenza vaccines—seasonal, H5N1 and 2009(H1N1) —in various age-classes (healthy children, adults or elderly). These meta-analyses have often adopted different analyses and study selection criteria. Because it is difficult to have a clear picture of vaccine benefits and harms examining single systematic reviews, we compiled the main findings and evaluated which could be the most reasonable explanations for some differences in findings (or their interpretation) across previously published meta-analyses. For each age group, we performed analyses that included all trials that had been included in at least one relevant meta-analysis, also exploring whether effect sizes changed over time. Although we identified several discrepancies among the meta-analyses on seasonal vaccines for children and elderly, overall most seasonal influenza vaccines showed statistically significant efficacy/effectiveness, which was acceptable or high for laboratory-confirmed cases and of modest magnitude for clinically-confirmed cases. The available evidence on parenteral inactivated vaccines for children aged < 2 y remains scarce. Pre-pandemic “avian” H5N1 and pandemic 2009 (H1N1) vaccines can achieve satisfactory immunogenicity, but no meta-analysis has addressed H1N1 vaccination impact on clinical outcomes. Data on harms are overall reassuring, but their value is diminished by inconsistent reporting.
Meta-analysis; influenza vaccine; vaccine efficacy; vaccine immunogenicity; vaccine safety
elderly; clopidogrel; glycoprotein IIb/IIIa blockers
Genetic and molecular epidemiology covers a vast area of research. Given the rapid changes in this field, discussing a research agenda is a precarious and ambitious task. A representative set of high‐priority concepts will be presented here, each of which alone could be the topic of a long series of essays. The wish list includes issues of full transparency and integration of information, dealing efficiently with complex multidimensional biology, juxtaposing the genome and environmental exposures, and using robust randomised trials to advance our knowledge and its application in this field.
Black box warnings (BBWs) are the strongest medication-related safety warnings in a drug’s labeling information and highlight major risks. Absence of a BBW or asynchronous addition of a BBW among same-class drugs could have major implications.
We identified the 20 top-selling drugs in 2008 (10 with BBWs and 10 without BBWs on their label) that belonged to different drug classes. We collected labeling information on all drugs belonging in these 20 classes, and recorded differences in the presence and timing of acquisition of BBWs for same-class drugs.
Across the 20 evaluated drug classes, we identified 176 different agents, of which 7 had been withdrawn for safety reasons. The reasons for the withdrawals became BBWs in other same-class agents only in two of the seven cases. Differences were identified in 9 of the 20 classes corresponding to 15 BBWs that were not present in all drugs of the same class. The information for 10 of the 15 different BBWs were included in the labels of same-class drugs as simple warnings or text, while it was absent entirely in 5 BBWs. The median interval from the time the BBW had appeared in another drug of the same class was 66 months.
Differences in BBW labeling in same-class drugs are common and shape impressions about the safety of similar agents. BBW labeling needs to become more systematic.
Electronic supplementary material
The online version of this article (doi:10.1007/s11606-011-1633-9) contains supplementary material, which is available to authorized users.
black-box warning; FDA; adverse events; harms
We studied the independent and joint effects of the genes encoding alpha-synuclein (SNCA) and microtubule associated protein tau (MAPT) in Parkinson's disease (PD) as part of a large meta-analysis of individual data from case-control studies participating in the Genetic Epidemiology of Parkinson's Disease (GEO-PD) consortium.
Participants of Caucasian ancestry were genotyped for a total of four SNCA (rs2583988, rs181489, rs356219, rs11931074) and two MAPT (rs1052553, rs242557) SNPs. Individual and joint effects of SNCA and MAPT SNPs were investigated using fixed- and random-effects logistic regression models. Interactions were studied both on a multiplicative and an additive scale, and using a case-control and case-only approach.
Fifteen GEO-PD sites contributed a total of 5302 cases and 4161 controls. All four SNCA SNPs and the MAPT H1-haplotype defining SNP (rs1052553) displayed a highly significant marginal association with PD at the significance level adjusted for multiple comparisons. For SNCA, the strongest associations were observed for SNPs located at the 3′ end of the gene. There was no evidence of statistical interaction between any of the four SNCA SNPs and rs1052553 or rs242557, neither on the multiplicative nor on the additive scale.
This study confirms the association between PD and both SNCA SNPs and the H1 MAPT haplotype. It shows, based on a variety of approaches, that the joint action of variants in these two loci is consistent with independent effects of the genes without additional interacting effects.
Parkinson disease; SNCA; MAPT; genetics; interaction; case-control
Proposed molecular classifiers may be overfit to idiosyncrasies of noisy genomic and proteomic data. Cross-validation methods are often used to obtain estimates of classification accuracy, but both simulations and case studies suggest that, when inappropriate methods are used, bias may ensue. Bias can be bypassed and generalizability can be tested by external (independent) validation. We evaluated 35 studies that have reported on external validation of a molecular classifier. We extracted information on study design and methodological features, and compared the performance of molecular classifiers in internal cross-validation versus external validation for 28 studies where both had been performed. We demonstrate that the majority of studies pursued cross-validation practices that are likely to overestimate classifier performance. Most studies were markedly underpowered to detect a 20% decrease in sensitivity or specificity between internal cross-validation and external validation [median power was 36% (IQR, 21–61%) and 29% (IQR, 15–65%), respectively]. The median reported classification performance for sensitivity and specificity was 94% and 98%, respectively, in cross-validation and 88% and 81% for independent validation. The relative diagnostic odds ratio was 3.26 (95% CI 2.04–5.21) for cross-validation versus independent validation. Finally, we reviewed all studies (n = 758) which cited those in our study sample, and identified only one instance of additional subsequent independent validation of these classifiers. In conclusion, these results document that many cross-validation practices employed in the literature are potentially biased and genuine progress in this field will require adoption of routine external validation of molecular classifiers, preferably in much larger studies than in current practice.
predictive medicine; genes; gene expression; proteomics
The rapid and continuing progress in gene discovery for complex diseases is fuelling interest in the potential application of genetic risk models for clinical and public health practice.The number of studies assessing the predictive ability is steadily increasing, but they vary widely in completeness of reporting and apparent quality.Transparent reporting of the strengths and weaknesses of these studies is important to facilitate the accumulation of evidence on genetic risk prediction.A multidisciplinary workshop sponsored by the Human Genome Epidemiology Network developed a checklist of 25 items recommended for strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS), building on the principles established by prior reporting guidelines.These recommendations aim to enhance the transparency, quality and completeness of study reporting, and thereby to improve the synthesis and application of information from multiple studies that might differ in design, conduct or analysis.
The authors evaluated whether there is an excess of statistically significant results in studies of genetic associations with Alzheimer's disease reflecting either between-study heterogeneity or bias. Among published articles on genetic associations entered into the comprehensive AlzGene database (www.alzgene.org) through January 31, 2007, 1,348 studies included in 175 meta-analyses with 3 or more studies each were analyzed. The number of observed studies (O) with statistically significant results (P = 0.05 threshold) was compared with the expected number (E) under different assumptions for the magnitude of the effect size. In the main analysis, the plausible effect size of each association was the summary effect presented in the respective meta-analysis. Overall, 19 meta-analyses (all with eventually nonsignificant summary effects) had a documented excess of O over E: Typically single studies had significant effects pointing in opposite directions and early summary effects were dissipated over time. Across the whole domain, O was 235 (17.4%), while E was 164.8 (12.2%) (P < 10−6). The excess showed a predilection for meta-analyses with nonsignificant summary effects and between-study heterogeneity. The excess was seen for all levels of statistical significance and also for studies with borderline P values (P = 0.05–0.10). The excess of significant findings may represent significance-chasing biases in a setting of massive testing.
Alzheimer disease; bias (epidemiology); genetic markers; genetics; meta-analysis; publication bias
Randomized evidence for vaccine immunogenicity and safety is urgently needed in the setting of pandemics with new emerging infectious agents. We carried out an observational survey to evaluate how many randomized controlled trials testing 2009 H1N1 vaccines were published among those registered, and what was the time lag from their start to publication and from their completion to publication.
PubMed, EMBASE and 9 clinical trial registries were searched for eligible randomized controlled trials. The units of the analysis were single randomized trials on any individual receiving influenza vaccines in any setting.
73 eligible trials were identified that had been registered in 2009–2010. By June 30, 2011 only 21 (29%) of these trials had been published, representing 38% of the randomized sample size (19905 of 52765). Trials starting later were published less rapidly (hazard ratio 0.42 per month; 95% Confidence Interval: 0.27 to 0.64; p<0.001). Similarly, trials completed later were published less rapidly (hazard ratio 0.43 per month; 95% CI: 0.27 to 0.67; p<0.001). Randomized controlled trials were completed promptly (median, 5 months from start to completion), but only a minority were subsequently published.
Most registered randomized trials on vaccines for the H1N1 pandemic are not published in the peer-reviewed literature.