We examined the association between study design characteristics and the results and conclusions of RCTs of head-to-head comparisons of statins with other drugs. We hypothesized that the results and conclusions of published trials would be more likely to favor the statin made by the sponsor of the study and that other design features, such as concealment of allocation, blinding, and sample size, would also be associated with results or conclusions that favor the statin. We found that the main factor associated with the results and conclusions of industry-sponsored research to compare statin drugs with statin or non-statin drugs is research sponsorship. Our study adds new information to the body of literature showing that pharmaceutical industry–sponsored studies comparing drug and placebo are more likely to favor the drug [14
]. Our finding suggests that favorable results and outcomes are associated with the specific sponsor of a study, even when all the studies are industry funded. This finding may help explain why well-designed head-to-head comparisons of statins and other drugs sometimes have contradictory results.
There are several possible explanations for our finding of the strong association between funding source and outcomes that are favorable to the drug company sponsor. First, it is possible that pharmaceutical companies selectively fund trials on drugs that are likely to produce a statistically significant result. This can be accomplished by selecting nonequivalent doses of drugs for testing [15
]. A recent review of 42 RCTs comparing the low-density lipoprotein–lowering ability of two or mores statins found that almost all of the trials compared nonequivalent doses of statins [32
]. Second, as we examined only published studies, publication bias, or the phenomenon of statistically significant results being published more frequently than statistically nonsignificant results, may explain the association of funding and outcome [40
]. Selective reporting of outcomes can also contribute to biased results and conclusions [4
]. In addition, industry sponsorship may be associated with multiple reporting of studies with favorable findings, emphasizing the imbalance towards statistically significant results in the published literature [18
]. Finally, more than one third of the studies in our sample had no disclosed sponsorship. It is possible that industry funders or industry-supported authors could fail to disclose the sponsorship of a published study if the findings do not support the sponsor's product. This, however, would have to be very prevalent to explain by itself the results in and concerning sponsorship. For example, nearly all the 29 studies with favorable conclusions and undisclosed funding would have to be actually funded by the comparator drug company, and nearly all the 45 studies without favorable conclusions would have to be funded by the test drug company in order to erase the difference shown in .
We identified a number of weaknesses common to published statin versus statin or non-statin drug comparisons that bring into question their clinical relevance. The most important weakness was the lack of patient-related clinical-outcome measures. Looking at the totality of available evidence, we found that almost all studies (98%, 189/192) used only surrogate outcome measures. Inadequate blinding, lack of concealment of allocation, poor follow-up, and lack of intention-to-treat analyses were common among these studies. These weaknesses suggest that these types of studies should be used with great caution by those making regulatory and purchasing decisions.
We found that adequate double blinding was an influential design feature in our sample. Adequately blinded studies were less likely to report results favoring the sponsor's product. Although RCTs of poorer quality are more likely to reach biased conclusions [10
], the unreliability of quality scores has led to the recommendation that trial quality be assessed for individual features that are aimed to reduce bias, such as concealment of allocation or blinding [45
]. Recent research further suggests that specific study design characteristics are not reliably associated with treatment effect sizes across different studies and medical areas [46
]. Thus, the study design characteristics associated with statistically significant results might vary with the type of research being examined. We used three items from the Chalmers et al. [38
] methodological quality assessment scale to assess study design features that might be associated with results and conclusions. These items focus on important aspects of trial design: (1) concealment of allocation, (2) control for whether all patients enrolled in the trial were included in the analysis (drop outs and intention-to-treat analyses), and (3) blinding of participants and investigators. Previous studies showed that these specific characteristics are associated with bias in clinical trials [10
]. For example, Schulz and colleagues found that estimates of treatment effects were exaggerated by 41% for inadequately concealed trials and by 17% for trials with inadequate double blinding [10
Finally, journal characteristics may influence the results and conclusions of articles as the quality of the reporting may vary with the journal. For example, articles published in peer-reviewed journals have superior quality compared to articles published in non-peer-reviewed journals [15
]. In our sample, we had no variability in peer-review status, but we did observe a small possible association between journal impact factors and results and conclusions that favored the test drug.
Our study has several limitations, including our ability to identify funding sources and financial ties. We categorized studies as industry funded or not based on each article's disclosure of a trial's funding source(s). Krimsky showed, however, that there is a lack of disclosure of industry research support and personal financial ties across a wide variety of journals [49
]. Thus, we may be underestimating the number of industry-sponsored studies and personal financial ties of investigators.
The market for statins is competitive. The National Cholesterol Education Program update of the Adult Treatment Panel III guidelines expanded both the scope and intensity of low-density lipoprotein–lowering therapy for prevention of cardiovascular disease [51
]. To achieve the goals in the guideline, millions of Americans would need to be placed on cholesterol-lowering medication in higher doses and for a longer period, thereby increasing the number of prescriptions for statin drugs [51
]. Eight of the nine members of the National Cholesterol Education Program panel had financial ties with pharmaceutical companies that manufactured statin drugs [51
]. Our findings suggest that available data on choosing between statins based on head-to-head comparisons may also be influenced by financial conflicts of interest. Our findings may be generalizable to other classes of drugs with competitive markets.
There is increasing concern that the funding source influences outcomes and conclusions of medical research [3
]. At the same time, industry support of biomedical research has increased dramatically during the past few decades [53
]. The growing proportion of industry-funded studies could shift the balance of published trials more towards studies that favor new drugs [55
]. This trend and our finding that, for one class of drugs, the results and conclusions of trials tend to favor the drug that is made by the sponsor raises important considerations for selecting drugs within a class. Sponsorship bias, even when controlling for other confounding study characteristics, may be the main explanation for contradictory findings of drug–drug comparison trials. This bias in drug–drug comparison trials should be considered when making health-policy decisions regarding drug choice, such as drug formulary decisions. Reviewers of published reports that disclose funding by the makers of the product being tested should be more critical of the methods than if the reports are not industry sponsored [56