We included meta-analyses published up to December 2004 that evaluated the effects of antihypertensive drugs on clinical outcomes in adults. The comparison group could include placebo, no treatment, usual care, or active therapy. We defined meta-analyses as systematic reviews that quantitatively combined data from at least two studies. We excluded meta-analyses on pregnant women and children because the mechanisms of hypertension in these populations differ from those in adults with chronically elevated blood pressure.
We also excluded meta-analyses that were duplicates or that overlapped considerably with one another. Duplicate meta-analyses were those that shared at least one author and evaluated the same trials and primary outcome measures. Overlapping meta-analyses were those that did not qualify as duplicate meta-analyses but shared at least one author, at least one trial, and the same topic (for example, the update of a pre-existing meta-analysis with new trial data). From each group of duplicate or overlapping meta-analyses we identified a representative meta-analysis to be included in the study, which was the meta-analysis that was published first. If two meta-analyses were published simultaneously, we randomly selected one for inclusion by rolling a dice.
We identified meta-analyses, without language restriction, by searching PubMed and the Cochrane Database of Systematic Reviews and by hand searching the reference lists of identified meta-analyses. A description of the search terms is available on bmj.com. One of us (VY) reviewed the titles and abstracts of all potential meta-analyses for inclusion. If the title was uninformative and no abstract was available, VY retrieved and reviewed the full text of the article to determine its eligibility.
Definition of financial ties
Financial ties were categorised as one drug company compared with all others. Information on financial ties was obtained from three sources: disclosures in the meta-analysis itself—sources of funding for the study or authors, or author affiliations; disclosures of industry or other sponsorship in the journal supplement in which a meta-analysis was published; and disclosures of financial ties in previous research articles on antihypertensive drugs by the first author of the included meta-analysis, arbitrarily going back three years before the publication date of the referent meta-analysis. Meta-analyses with financial ties to one drug company as disclosed in any one of the three sources, were defined as having financial ties to one drug company. We designed this definition of financial ties to be conservative. For example, a meta-analysis that we classified as having financial ties to one drug company on the basis of publication in an industry sponsored supplement could also have had financial ties to academia on the basis of the funding source of the meta-analysis. Academic financial ties might be expected to dilute the potential influence of industry, thus assuring that our findings would be conservative rather than inflated.
We carried out sensitivity analyses using different definitions of financial ties—using information disclosed only in the meta-analysis, or using information disclosed in the meta-analysis and in the supplement in which it was published. This shifted meta-analyses from the category of financial ties to one drug company to the category of all other financial ties.
We collected additional data on the financial ties of meta-analyses in the all other category. The subcategories for this category were defined as having financial ties to multiple drug companies; non-profit (academic, government, foundation, or professional) groups; any drug company (single or multiple) and non-profit; and no statement. We originally had two other subcategories of “no funding” and “only other,” but these contained only one meta-analysis each, so we combined the data with the no statement and non-profit subcategories, respectively. We collected these data with the a priori hypothesis that, even within the all other category, graded differences would exist between meta-analyses in the degree to which their results or conclusions were favourable towards the study drug, depending on their subcategory of financial tie.
Definition of the study drug and outcome measure for each meta-analysis
The study drug and outcome measure were defined by the authors of the meta-analyses or if left unspecified we defined them as the first treatment and outcome described in the results. The study drug could be single or combined therapy. If multiple primary outcome measures were explicitly identified, we deemed results or conclusions to be favourable if at least 50% of the outcome measures were favourable.
Primary outcome measures for this study
The primary outcome measures for this study were results, as determined by us, and conclusions, as stated by the authors of the meta-analyses. Our per protocol analyses were prespecified to use dichotomous coding of the results and conclusions as being favourable towards the study drug compared with not favourable. We collected additional data on subcategories of the not favourable group.
Results were coded from 1-5, with 1 being statistically in favour of the study drug, 2 being statistically against the study drug, 3 being statistically neither in favour nor against the study drug, 4 being unclear, and 5 being other. In accordance with our protocol we considered results coded as 1 to be favourable towards the study drug and those coded as 2-5 to be not favourable. We believe this coding is the most widely used for non-equivalency studies.
Conclusions were coded from 1-5, with 1 being in favour of the study drug, 2 being against the study drug, 3 being neutral towards the study drug, 4 being unclear, and 5 being other. In accordance with our protocol we considered conclusions coded as 1 to be favourable towards the study drug and those coded as 2-5 to be not favourable.
Other potentially relevant variables
We wanted to determine whether certain financial ties were associated with skewed results or conclusions, even after controlling for other variables. We therefore collected data on other characteristics of meta-analyses that our literature review suggested might be associated with favourable results or conclusions.
Better methodological quality of meta-analyses has not been consistently associated with favourable conclusions.25 26
We evaluated the methodological quality of each meta-analysis using a modified version of the Oxman and Guyatt quality instrument,27
which rates systematic reviews and meta-analyses on whether they include design features aimed at reducing bias, and assigns a summary score to each meta-analysis. The quality instrument has nine questions: Did the authors clearly describe their strategy for identifying primary research studies on the meta-analysis topic? Was the search strategy appropriate? Did the authors clearly report their criteria for deciding which studies to include and exclude? Were the inclusion and exclusion criteria appropriate? Did the authors clearly report their criteria for assessing the quality or validity of studies included? Was the validity assessment appropriate? Did the authors clearly report their strategy for quantitatively combining study results? Were study results combined appropriately? Were the stated conclusions supported by the data presented? The only feature evaluated by Oxman and Guyatt but not evaluated by our quality instrument was the overall scientific quality of the overview, which we thought would be redundant with the quality score. For each design feature the meta-analysis could receive a maximum of two points for fulfilling the criterion, one point for partially fulfilling it, or zero points for not fulfilling it. The quality score was the sum of these points, with the total possible being 18. We also evaluated whether the quality scores correlated significantly with any of the other characteristics of the meta-analyses.
Some evidence suggests that supplements from symposiums, especially those sponsored by drug companies, contain articles that are biased and of poor quality.28 29
We determined whether the meta-analyses were published in journal supplements.
We also collected data on additional characteristics: whether the meta-analyses involved literature searches or included studies in languages other than English, as well as in English, compared with in English only30 31
; included a description of the process of data abstraction compared with no description32,33,34
; included studies that were not randomised controlled trials compared with included only randomised controlled trials35
; included unpublished plus published studies compared with only published studies36
; included only studies that used placebo groups as the control group compared with studies that used other comparator groups (for example, no treatment, usual care, active drug control)37
; focused only on newer classes of drugs compared with older classes (newer drugs being, for example, angiotensin converting enzyme inhibitors, angiotensin II receptor blockers, or α blockers compared with older drugs, defined as β blockers and diuretics)9 38
; used surrogate outcomes only (for example, blood pressure, level of creatinine, levels of lipids) compared with morbidity and mortality outcomes (for example, myocardial infarction, dependence on dialysis, death)37 39
; used composite outcomes only (for example, myocardial infarction, stroke, and death) compared with distinct outcomes (for example, total mortality)40
; carried out evaluations of heterogeneity of included studies compared with no evaluations41
; and carried out sensitivity analyses of the results compared with no sensitivity analyses.42
We pretested our data extraction tool and quality instrument during a pilot study. This pilot study showed good intercoder reliability between the three reviewers in data extraction and quality assessment, despite one reviewer being unblinded and the other two being blinded to information on financial ties, as well as to author identity. One researcher (VY) unblinded to financial ties and author identity coded data collection items for the meta-analyses in our main study. A second coder (LAB), blinded to financial ties (according to our study definition) and author identity (name, affiliation, and address), coded a random sample of 24 (19%) of the meta-analyses in our main study. The degree of agreement between the two reviewers’ evaluations were κ=0.74 (substantial) for results and κ=0.60 (moderate) for conclusions.43
We used univariate logistic regression analyses to evaluate whether financial ties or other characteristics of meta-analyses were associated with favourable results or conclusions. Variables that were found to be significant to the level of P<0.05 on univariate analyses were then entered into exploratory multiple logistic regression models. All analyses were carried out using SAS version 9.1.