|Home | About | Journals | Submit | Contact Us | Français|
Gender and ethnicity are factors affecting the incidence and severity of vascular disease as well as subsequent treatment outcomes. Though well-studied in other fields, balanced enrollment of patients with relevant demographic characteristics in vascular surgery randomized-control trials (RCT) is not well known. This study describes the reporting of gender and ethnicity data in vascular surgery RCT and analyzes whether these studies adequately represent our diverse patient population.
We conducted a retrospective review of US-based RCT from 1983 through 2007 for three broadly defined vascular procedures: aortic aneurysm repair (AAR), carotid revascularization (CR), and lower extremity revascularization (LER). Included studies were examined for gender and ethnicity data, study parameters, funding source, and geographic region. The Nationwide Inpatient Sample (NIS) database was analyzed to obtain group-specific procedure frequency as an estimate of procedure frequency in the general population.
Seventy-seven studies were reviewed and 52 met our inclusion criteria. Of these, only 85% reported gender and 21% reported ethnicity. Reporting of ethnicity was strongly associated with larger (>280 subjects), multi-center, government-funded trials (P<0.001 for all). Women are disproportionately under-represented in RCT for all procedure categories (AAR: 9.0% vs 21.5%, CR: 30.0% vs 42.9%, LER: 22.4% vs 41.3%) while minorities are underrepresented in AAR studies (6.0% vs 10.7%) and CR studies (6.9% vs 9.5%) while they are over represented in LER studies (26.0% vs 21.8%, P<.001 for all).
Minority ethnicity and female gender are under-reported and under-represented in vascular surgery RCT, particularly in small, non-government funded and single-center trials. The generalizability of some trial results may not be applicable to these populations. Greater effort to enroll a balanced study population in RCT may yield more broadly applicable results.
Evidence-based medicine has revolutionized healthcare with the increased use of rigorous scientific analysis to address clinical questions. As a result, greater appreciation of the relative strengths of different types of clinical evidence has also affected the pursuit of new clinical analyses. Properly designed and conducted randomized clinical trials (RCT) are considered the highest level of evidence (Level I) available to guide clinical practice. The strength of the RCT lies in the randomization of patient to treatment options, since randomization diminishes the impact of known and unknown confounders. As with all clinical studies, one limitation of RCT is that the applicability of the results may not extend to patients that are underrepresented in the study sample.
Disparities exist between gender and between race/ethnicity with regard to the prevalence, presentation and outcome of vascular disease. There is evidence that African-Americans (AA) and Hispanic-Americans (HA), for example, have a higher prevalence of cardiovascular risk factors including hypertension and diabetes1-4 and peripheral arterial disease (PAD) as measured by ankle brachial index (ABI) compared to Caucasians.1, 5, 6 These disparities also extend to disease severity and treatment outcomes. AA and HA have more severe PAD as well as subsequent worse outcomes of bypass and greater need for amputation, compared to Caucasians.3, 4, 7 Furthermore, AA have higher risk-adjusted mortality after cardiac and vascular procedures including coronary artery bypass (CABG), aortic aneurysm repair (AAR), and carotid endarterectomy (CEA) compared to HA and Caucasians.8 Gender differences in presentation and outcome of vascular disease have also been demonstrated, with women having a higher ratio of emergency AAR compared to men,9 higher mortality, length of stay, and discharge to a nursing facility for both elective and emergent AAR.10 Both women and AA have been shown to experience lower graft patency after LER.11
Clinical studies should account for these gender and ethnicity disparities in vascular disease. Enrollment of a representative patient population in RCT is important to broaden the generalizability of the results. Other specialties, particularly medical oncology, have documented underrepresentation of women and minorities in randomized controlled trials.12, 13 However, little information is available to assess whether the study populations in vascular surgery in RCT are representative of the patients at risk for vascular disease.
The disparity in enrollment into studies and trials by gender and ethnicity was addressed in the National Institutes of Health (NIH) Revitalization Act of 1993 titled “Women and Minorities as Subjects in Clinical Research.”14, 15 This Act mandated representative inclusion of women and minorities as subjects in clinical research. It further specified that, particularly in diseases where pathophysiology or treatment may differ depending on gender or ethnicity, women and ethnic minorities must be adequately represented to detect differences in endpoints. Practically speaking, this Act significantly affects clinical research through its implications for trial design, subject recruitment, and study documentation. Implicit in these mandates are complete reporting of gender and ethnicity in all clinical research, even if adequate representation cannot be achieved.
We hypothesized that there is disproportionately low enrollment of women and ethnic minorities in RCT for the treatment of vascular disease in the United States. To evaluate our hypothesis, we carried out a retrospective analysis of published RCT for three common vascular surgery procedures: AAR, carotid revascularization (CR), and lower extremity revascularization (LER). We analyzed the reporting of gender and ethnicity in these RCT and compared the enrolled population mix to a contemporary estimate of procedure frequency based on a nationwide inpatient care database.
In August 2007 we conducted an electronic search of RCT published over the last 25 years using the Medline and Cochrane Clinical Trials databases. Our three categories of vascular procedure (AAR, CR and LER) were intended to be broadly inclusive so as to maximize the number of clinically relevant studies. The Medline database was searched for articles published in English between 1983 and August 2007 using a combined keyword and Medical Subject Heading (MeSH) strategy. The term “randomized controlled trial” was inclusively paired (“and”) with an array of search terms related to the three vascular procedures. For AAR this included “aortic aneurysm”, “thoracic aneurysm” and “abdominal aneurysm”. For CR this included “carotid”, “carotid stenosis” and “carotid endarterectomy”. For LER this included “claudication”, “peripheral vascular disease”, “critical limb ischemia” and “lower extremity bypass”. Similarly, the Cochrane Randomized Controlled Trial Database was searched for studies conducted between 1983 and 2007 that included the keywords “carotid”, “aortic” or “lower extremity”. Finally, six Cochrane Review articles generated by generalized Cochrane Database search of the above terms were reviewed for relevant randomized controlled trials (Appendix 1).
The resulting articles were individually screened to include only RCT that were based in the United States and included a vascular surgical or endovascular procedure as a component of the trial design. Studies that contained multiple procedures were included in the pooled analysis of gender and ethnicity reporting; however all of these multiple procedure trials only include procedures typically performed by a vascular surgeon. For example, a trial involving lower extremity bypass and aortic aneurysm repair would be included while a trial involving aortic aneurysm repair and aortic root repair would be excluded.
Following screening, all included studies were evaluated for the reporting of the gender and ethnicity of their subjects. Studies reporting gender and ethnicity were further analyzed for the proportion of women and minority race/ethnicity subjects enrolled. The number of subjects enrolled, the funding source of the research, the number of centers enrolling subjects and the publication year of each study was also recorded. For the purposes of our evaluation, “ethnic minority” is defined as an enrolled patient other than a “Caucasian” or “non-Hispanic white” as reported in each study.
The Nationwide Inpatient Sample (NIS) database is the largest all-payer inpatient care database publicly available in the U.S., and contains data from 5 to 8 million hospital stays form approximately 1000 hospitals representing a 20% stratified sample of U.S. hospitals. The 2004 and 2005 NIS databases (the two most current years available), were queried for combined diagnosis and procedures for each of the three categories (AAR, CR, LER). The patient sample with a pertinent code for each diagnosis (aortic aneurysm, carotid stenosis, PAD) was paired with an appropriate procedure code (for AAR, CR, LER). As a result, only patients with a diagnosis of aortic aneurysm, carotid stenosis or PAD who underwent treatment for that diagnosis were included in the dataset (Appendix 2).
The resulting raw 20% patient sample for each procedure category was subsequently stratified by gender and race/ethnicity, yielding a relative frequency estimate for both categories. These frequency estimates served as the expected demographic ratios and were then compared to the demographic data from RCT that reported gender and/or race/ethnicity demographic data. The NIS data collection and stratification was completed using SAS v. 9.1 (Cary, NC).
The reporting of gender and/or race/ethnicity was evaluated as a binomial variable using the Fisher exact test to evaluate the association between demographic reporting and study characteristics. These characteristics include publication year, number of centers enrolling subjects (single-center or multi-center), funding source, and study size. Funding sources were dichotomized between government funded studies (NIH and Veterans Affairs) and non-government funded studies (industry, private institutions, non-specified). Study size was dichotomized into “large” and “small” subgroups based on enrollment above or below the mean enrollment number (280 patients) among all trials. Studies that reported enrollment data by gender and/or race/ethnicity were further evaluated by comparison to the procedure frequency by gender and race/ethnicity, as estimated from the NIS database using the Fisher exact test. An alpha less than or equal to 0.05, corresponding to a P ≤ 0.05, was considered statistically significant. All statistical analyses were performed using SAS v. 9.1.
The initial literature search yielded 77 studies that were individually reviewed for project inclusion. Of these, 25 were excluded because of geographic (outside the United States) and surgical procedural characteristics (non-vascular surgical procedures), leaving 52 studies for our final analysis. These were divided by category of diagnosis (11 AAR studies, 18 CR studies, 13 LER studies and 10 multi-procedure studies) and evaluated for the reporting of gender and ethnicity in the study samples (Appendix 3). In total, 14,604 study subjects were randomized into the 52 studies. The mean number of subjects enrolled per RCT was 280 (median 100, range [9-2226]). Four studies enrolled greater than 1000 subjects. Included studies were published between 1983 and 2007 with a median publication year of 1998 and mode for publication year of 1997 (9 studies).
Forty-four of 52 studies (85%) reported the gender of subjects in the randomized sample. All government funded studies (N=12) and large studies (≥ 280 subjects, N=12) reported gender in their demographics. Both of these study characteristics showed a trend toward significance (P = 0.17) when compared to non-government funded studies and small studies (<280 subjects), respectively. There was no significant reporting differences between single or multi-center trials (N=17). Because the NIH mandated inclusion of race and gender data starting in 1994, publication year was also evaluated for differences of gender reporting. This demonstrated no difference between frequency of gender reporting before or after 1994 (P = 1.0, N=41 for post-1994 studies). To account for a delay between trial enrollment and publication, gender reporting was also compared in five separate analyses using years 1995 through 1999. There was likewise no significant difference in reporting for any of the publication years evaluated.
Eleven of 52 studies (21%) reported the race or ethnicity of enrolled subjects. Reporting of ethnicity was associated with large, multi-center trials, and government-funded trials (P< 0.001 for all). Ethnicity reporting was not associated with publication year of the study for any single year between 1994 and 1999. Aggregate reporting, enrollment and estimated procedure frequency data is summarized in Table 1.
RCT containing only one procedure (N=42) and also reporting gender (N=35) and/or ethnicity (N=10) were further analyzed for representative enrollment by comparison with nationwide procedural rates from the NIS database. In the U.S., women comprised approximately 21.5% of AAR, 42.9% of CR and 41.3% of LER procedures in 2004-2005. In contrast, women represented 9.0% of AAR, 30% of CR and 22.4% of LER subjects in RCT. In each of these procedure categories women were significantly underrepresented in RCT compared to the estimated population procedural incidence (P<0.001 for all, Figure 1A).
In the U.S., minorities comprised approximately 10.7% of AAR, 9.5% of CR and 21.8% of LER procedures in 2004-2005. In contrast, minorities represented 6.0% of AAR, 6.9% of CR and 26.0% of LER subjects in RCT. In AAR and CR studies, minorities were significantly underrepresented (P<0.001). Studies of LER enrolled a higher proportion of minorities compared to the estimated procedure frequency (Figure 1B, P<0.001).
Of the single procedure RCT in our study, 6 were from the Veterans Affairs (VA) healthcare system (1 for AAR, 2 for CR and 3 for LER). These studies tended to be larger in enrollment (median 598, range [189-1136]) and all reported gender demographic data. Because of the inherent gender imbalance in the VA population, we performed secondary gender enrollment analyses with these VA RCT excluded. This demonstrated female enrollment of 26.4% in AAR (vs. 21.5% of AAR procedures estimated from NIS), 33.3% in CR (vs. 42.9% of CR procedures) and 37.0% in LER studies (vs. 41.3% of LER procedures). These comparisons demonstrate that non-VA RCT of AAR enrolled significantly more women than estimated procedure frequency from NIS (P=0.008) while women remained underrepresented in non-VA studies of CR and LER (P=0.0001) even though the gap between enrollment and estimated procedure frequency narrowed markedly. The exclusion of VA-based trials from the evaluation of minority race/ethnicity enrollment was also performed and demonstrated similar percentages to procedure specific enrollment in all of the available studies related to CR and LER. Exclusion of VA-based trials from the group of AAR studies left us with no studies that reported race/ethnicity demographic data.
Vascular surgeons treat a diverse patient population and Level I evidence derived from RCT guides the standard of surgical care based on representative sampling of that population. In this study we have shown that complete demographic reporting of gender and race/ethnicity is absent from a large portion of vascular surgery RCT conducted in the U.S. Complete demographic reporting, particularly for ethnicity, was associated with larger, multicenter and/or government funded trials. There does not appear to be significant increases in reporting over time, particularly in the years following the NIH Revitalization Act of 1993. Among studies reporting gender and ethnicity data, women were significantly underrepresented in clinical trials for aortic, lower extremity and carotid artery procedures, while ethnic minorities were underrepresented in RCT involving aortic aneurysms and carotid revascularization.
The underreporting and underrepresentation of women and minorities occurs in a broad array of clinical research across multiple specialties. Even in specialties with known health disparities between gender and ethnicity, such as in diabetes, cardiac disease, HIV and cancer, there is persistent underreporting of complete demographic data in clinical research.16 There is also a demonstrated underrepresentation of women and minorities in clinical trials as well as underutilization of women and minorities in statistical models aimed at better understanding the disparities in disease prevalence and outcomes.17 Oddone, et al. evaluated trials containing an invasive arm and demonstrated that most enrolled fewer minority patients than expected. Interestingly, this same study showed that a majority of trials for diseases with disproportionate minority prevalence (diabetes, hypertension, end stage renal disease) enrolled more minority subjects than expected.18 As we have noted, PAD is more prevalent in AA and HA. As such, our study corroborates Oddone's findings in RCT of LER where minority enrollment exceeded estimated procedure frequency. One potential explanation for this over-representation may be that many RCT occur at large academic institutions that already care for a larger number of minority patients. Thus, the pool of trial participants is already over-represented in these institutions.
Significant consideration has been given to the causes of underrepresentation of minorities and women in RCT. There is a long-held belief that ethnic minorities are less willing to participate in human and clinical research, attributed to a mistrust of the medical and scientific community stemming from the Tuskegee syphilis and other similar experiments widely publicized in the media.19 However, this notion has become increasingly controversial as multiple recent studies have demonstrated equivalent willingness to participate in clinical trials regardless of ethnicity.20 Despite this, it is reported that minorities are more likely to believe that they had been previously treated in a trial without their consent.21 This suggests a persistent element of differing expectations and mistrust that needs to be overcome. In contrast, women, particularly those of child bearing age, have a long history of being excluded from RCT. Historically, this, in part, stems from a desire to limit the exposure of women of child bearing age to experimental treatments.22 However, with the recognition that gender is an important variable in the manifestation of disease and in the response to treatment, there has been significant effort toward balanced enrollment by gender into clinical trials. That there remains a persistent disparity in representation of women in a broad spectrum of clinical trials suggests the interplay of multiple subtle factors both in society and in clinical trial design and execution.23
Several limitations are present in this study. First, race/ethnicity is self-reported in RCT and in the NIS database. Furthermore, race/ethnicity is a complex topic with significant contributions from the fields of genetics, sociology, anthropology, and history. Not all minorities share similar backgrounds making broad categorization difficult, especially considering immigration patterns in the U.S. Despite these inherent difficulties in defining race/ethnicity, studies should at least try to report this important demographic in some manner. Presumably, gender is a more straightforward demographic variable to evaluate, yet many studies do not even report this data. It is possible that the studies not reporting gender or ethnicity may in fact have over-representation of these groups and thus the overall RCT enrollment distribution may be appropriate. However, when important demographic information is not available, no meaningful conclusions can be made.
Second, appropriate trial design may necessitate exclusion of portions of the population that have significant minority or gender representation. For example, end-stage renal disease is frequently a component of the exclusion criteria of vascular surgery RCT and it disproportionately affects AA.24, 25 As such, the goal of developing accurate endpoint measurement may have the unintended, and potentially unavoidable, consequence of under-representing specific patient populations.
Third, enrollment results are not entirely under the control of researchers. Very few RCT can draw enrollment from a nationally representative patient population. Even for multi-center studies, the local institutions conducting the trial may have distinct referral patterns. In particular, the VA patient population is not representative of the U.S. population in that it under represents women. However, many high quality vascular surgery RCT that guide clinical decision-making come from VA collaborations. Our sub-analysis excluding VA studies demonstrated attenuated disparities in enrollment. A more balanced incorporation of the VA population into the main analysis would require access to VA population and procedural data not available to the authors. Furthermore, with the increasing military enlistment of women and minorities in recent decades, the current veteran population may be more demographically similar to the entire U.S. population than in previous times.
Finally, the limited number of studies available in each category does not lend themselves to more sophisticated statistical measures. Because many of the parameters examined were strongly co-associated (for example, large trials and multi-center trials consisted of overlapping groups), multivariate logistic regression did not yield interpretable results and were therefore left out of this analysis.
In spite of these limitations, this study clearly demonstrates under-reporting of gender and race/ethnicity as well as disparities in the enrollment of women and minorities in vascular surgery RCT. This has significant implications for the interpretation and generalizability of these studies to the overall population. Recognition of these limitations may help surgeons assimilate other studies with better representation for their patient population. The authors of this study do not imply that the disparities in reporting and enrollment are intentional. Rather, patient concerns, unrecognized bias, and local population demographics likely contribute to these disparities. Nevertheless, this issue should be highlighted to promote compliance to NIH regulations in future studies. Further, in the interest of better understanding out diverse patient population, we feel these regulations should extend to all vascular surgery clinical studies, not just those funded by the NIH. With this recognition, researchers can focus on identifying and improving barriers to balanced recruitment and retention appropriate for their local environment. Such strategies can include focused recruitment and education about clinical research ethics and safeguards, recruitment of institutions with intrinsic minority representation into multi-center trials, and recruitment goals/caps based on estimates of disease incidence or procedural frequency.
RCT for aortic aneurysm repair, carotid revascularization, and lower extremity revascularization have under-reporting of gender and ethnicity demographics. Among studies with adequate reporting, disparities exist between the observed distribution of enrolled patients and expected enrollment based on national procedural frequencies. These findings affect the generalizability of RCT results and should serve as a stimulus to better comply with NIH clinical research guidelines.
This work was supported by NIH K23 Research Career Development Award (HL084386) (L.L.N.).
Presented at the 36th Annual Symposium of the Society for Clinical Vascular Surgery; Las Vegas, NV; March 5-8, 2008.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.