|Home | About | Journals | Submit | Contact Us | Français|
To determine both the intended and unintended effects of the National Institutes of Health (NIH) 2005 ethics rules by examining changes in publishing rates and the frequency of external relationships among NIH scientists.
After identifying eligible intramural scientists and administrators from institute’s web pages and central directories, a mailed survey was administered to 900 NIH research faculty between October 2008 and January 2009 (response rate 70.1%).A
Eighty percent of respondents believed the NIH ethics rules were too restrictive. While nearly half (45%) of respondents believed the rules positively impacted the public’s trust in the NIH, over three-quarters (77%) believed the rules hindered the NIH’s ability to complete its mission. Implementation of the ethics rules significantly decreased self-reported GIRs among NIH faculty (from 51.8% to 33.2%, P<.01), including significant drops in consulting (33.1% to 7.8%, P<.01) and scientific advisory board membership (31.5% to 16.0%, P<.01). The policy had limited impact on NIH faculty participation in non-industrial professional service roles and had no detectable change in publishing behavior (5.29 articles per researcher per year from 2002–2005 vs. 5.26 from 2005–2008, P = .88).
The NIH ethics rules accomplished much of what they were intended to do, limiting relationships with industry while maintaining NIH researchers’ association with external scientific and professional organizations. However, the rules negatively affected personnel morale and the perceived progress of research.
The current debate over industry’s role in biomedical research within government and academic settings revolves around a central disagreement about the balance of the risks and benefits of these relationships. One stance advocates a near total ban on relationships with industry, citing the need to curtail or eliminate all conflicts of interest in medicine. Proponents of this stance believe that the presence of any industrial relationships undermines the public’s trust in the medical establishment, increases secrecy in science, and biases medical decision-making towards company products and services.1,2 Opponents argue that industry relationships are simply one form of external scientific collaboration and has positive benefits for research and knowledge transfer. They believe any policies that restrict such relationships may isolate researchers and impede scientific advancements that may ultimately improve health outcomes and save lives.3
These opposing viewpoints were accentuated in 2004, when conflicts of interest within the intramural research program of the National Institutes of Health (NIH) became the subject of several newspaper investigations, federal inquiries, and congressional hearings.4–6 In response, then NIH Director Elias Zerhouni revamped the agency’s ethics policies, establishing final rules in August 2005 that effectively banned most (compensated or uncompensated) employment, consulting, and advising by NIH employees to biopharmaceutical firms, supported research institutions, and health care insurers and providers. The agency formed this policy based on three guiding principles:7
These principles emphasize the central importance of the public’s perception of government-industry relationships (GIRs) and conflicts of interest at the NIH for both scientists and administrators (herein, referred to as “faculty” of the NIH for ease of writing). But are these principles contradictory? By requiring that all relationships be submitted for approval, the regulations have created additional bureaucratic hurdles and may have inadvertently deterred “good” external scientific relationships. In an effort to reduce conflicts of interest, have the rules made the agency more insular by harming the professional discourse between NIH intramural faculty and external peers?
While the NIH conducted an internal survey of perceptions to the rules immediately following the introduction of these policies8, there has been no published study in the peer-reviewed literature of the subsequent impact of the NIH policy, especially on behaviors of NIH research personnel. The main purpose of this study was to determine how the rules affect the patenting and publishing performance of NIH scientists, as well as their collaborations with industry, professional organizations, and academia. In addition, we sought to gauge perceptions of conflict of interest and research integrity among NIH intramural faculty, and to examine how the 2005 Ethics Rules affected those perceptions. Emulating the guiding principles above, we also examined the perceived effect on public trust.
We collected the data presented here from a survey we administered to a sample of NIH faculty between October 2008 and January 2009. We identified eligible participants from the central NIH Directory and individual institutes’ web pages using criteria detailed below. These lists of potential participants were verified against individuals listed as principal investigators on intramural research over the previous two years on Computer Retrieval of Information on Scientific Projects (CRISP) (http://crisp.cit.nih.gov/). Because the primary focus of the NIH ethics rules was to ensure that research decisions were made without undue conflicts of interest, we limited the sample to NIH research scientists and administrators. Consequently, we removed from these lists any individual who was not listed as an investigator, scientist, or senior staff member, as well as faculty employed within policy/planning, scientific review, extramural grants management, communications, or veterinary sub-offices of the intramural program. This left 1636 eligible faculty from 21 divisions of the NIH. Each individual was assigned a random number and then sorted in ascending order. From this list, the first 900 NIH scientists and administrators were selected to ensure a random, representative sample of all faculty in the intramural program at the NIH.
We used a modified version of a survey instrument administered to 3080 life-science academic faculty in 2007,9 that we adapted and expanded based on 10 confidential personal interviews with current and former NIH faculty. In addition, experienced survey researchers pretested the new survey items using 11 cognitive interviews of NIH faculty that were not included in the final sample.
The survey focused on the faculty members’ knowledge, perceptions, and impact of the ethics rules, and also included questions on their rates of publishing, patenting, and external affiliations before and after their implantation. Because the ethics rules pertain to all external relationships, we queried respondents regarding both industry relationships and academic/professional relationships. Respondents were asked about each relationship individually over two time periods; for example, one set of questions asked, “Before the ethics rules did you serve in any of the following roles for companies as an NIH employee?” Types of industry relationships included serving as a company founder, board of directors, officer/executive, scientific advisory board, consultant, equity holder, recipient of funding for students, recipient of royalties, or corporate research collaboration. Academic and professional relationships included: journal editor, officer of a professional association, foundation board member, recipient of honoraria, or academic research collaboration. Questions regarding the faculty’s opinion on the various impacts of the ethics rules were based on a five-point Likert scale [large positive impact, small positive impact, no impact, small negative impact, large negative impact].
We asked respondents about all relationships, regardless of whether they were pre-approved by NIH administration; consequently some existing relationships, such as uncompensated scientific advisory board memberships, may be “authorized”. Because unapproved relationships are considered illegal activity that could cause the NIH scientist to be fired or prosecuted, we were concerned about the ability to reverse-identify any individual scientist. Therefore, we purposely did not ask respondents about gender, race/ethnicity, or worksite characteristics to protect confidentiality.
The Center for Survey Research at the University of Massachusetts, Boston conducted the survey by mail in a protocol identical to several previous projects.10,11 We conducted telephone follow-up to all individuals who did not return the separate postcard indicating participation. The phone calls had three main purposes: to make sure individuals had received the questionnaire and to correct addresses if they had not; to answer any questions the individual might have about the study; and to allow the individual to say he or she did not wish to participate simply by saying, “No.”. Those who declined to participate were coded as refusals and no follow-up contact was initiated. We mailed a final set of questionnaires to all those who had not returned the card and had not refused when reached by phone, with the aim of boosting the final response rate.
This study was approved by IRBs at both the Massachusetts General Hospital and the University of Massachusetts, Boston.
We divided the sampled individuals into three organizational categories: Administrators, Investigators, and Staff Scientists/Clinicians, based on the answer to the question, “What is your organizational title(s)? (Please check all that apply.)” Possible answers were Administrator (specify title), Senior Investigator, Investigator, Staff Scientist, Staff Clinician, and Other (specify title). The lead author (D.E.Z.) analyzed the data using standard statistical techniques. Bivariate analysis employed chi-squares for testing differences in simple proportions and analysis of variances for differences in means. Mutivariate logistic analyses were used to examine differences in respective proportions and means when adjusting for multiple predictor and control variables.
Of the 900 faculty in our sample, 93 were ineligible because they were retired, no longer intramural faculty, too ill to participate, or deceased. Of the remaining 807 faculty members, 566 completed the survey, for an overall response rate of 70.1%.12 The sensitive nature of the study and the subsequent need for anonymity of the questionnaires prevents us from a more detailed analysis of non-respondents. Although respondents were generally thorough and answered 97.8% of all possible questions, missing data from individually skipped questions slightly reduces the denominator of individual calculations and varies slightly across survey items.
Table 1 shows the characteristics of the 564 respondents (2 respondents chose not to answer questions about their professional characteristics). Of these faculty, 65 (11.5%) were administrators; 344 (61.0%) investigators; and 155 (27.5%) were staff scientists or staff clinicians. Nearly 95% (n = 541) were employed at the NIH before the introduction of new ethics rules in August 2005, and over half of all administrators and investigators had worked at the NIH for more than 15 years (P < .001 compared to scientists/clinicians). In terms of their educational background, 34% (n = 22) of administrators and 20% (n = 70) of investigators held MD degrees, compared to 9.7% (n = 15) of staff scientists (P < .001).
Of the faculty employed by the NIH since before the introduction of the ethics rules (n=541), 280 (51.8%) had some form of government-industry relationship (formal or informal, paid or unpaid) related to their area of scientific expertise during their career at the agency (Administrators: 70.0%; Investigators: 62.6%; Scientists/Clinicians: 21.2%; P<.05 for all comparisons). Of those with GIRs, 120 faculty members (42.9%) indicated that the industry relationship had contributed to their most important scientific work at the NIH.
Industry collaboration also demonstrated negative effects on relationships among NIH faculty (Table 2). Respondents indicated that in their experiences with other NIH scientists who had GIRs, 12.2% had concerns about the validity of the involved scientist’s work, 9.2% had been unable to replicate published research, 6.0% had a research idea stolen, and 5.6% had refused to share information, data, or materials with the involved scientist. These proportions did not differ significantly by NIH title or by past record of industry involvement.
Almost all respondents (n = 556, 98.2%) had at least some familiarity with the ethics rules. Of those, 444 (79.9%) believed the rules were “too restrictive” or “much too restrictive”; 91 (16.4%) felt they were “about right”; and one person (0.2%) felt they were “too loose” (16 respondents answered “don’t know” and 4 chose not to answer).
According to faculty, the NIH rules had a number of negative effects. Nearly half of faculty reported that the ethics rules had a negative impact on collaborations with industry (62%) and with academia (49%). Two-thirds cited a negative impact on their job satisfaction. Although a majority of faculty indicated that the ethics rules had no impact on their financial situation (61%), over a third (38%) felt that the rules negatively affected their finances and 28% were forced to divest from a financial holding because of the implementation of the ethics rules. At the same time, nearly half of all faculty respondents (45%) believed that the rules had a positive impact on the NIH’s credibility with the public (Figure 1).
Faculty with GIRs were significantly more negative about the effects of the new ethics rules compared to their peers without GIRs. Faculty with GIRs were less likely to say that the ethics rules had a positive impact on public credibility (OR = 0.66 [0.47–0.94], P = .02) and more than twice as likely to believe that the rules made it harder for the NIH to complete its mission (OR = 2.63 [1.72–4.01], P<.001). Those with GIRs were more likely than those without to believe that the rules had negative effects on both industrial (OR = 4.71 [3.20–6.95], P<.001) and academic collaborations (OR = 1.43 [1.02–2.01], P = .037), and were more likely to report negative personal effects derived from the rules (odds ratios between 2.18 and 4.07 for questions regarding effects on individual faculty, P<.001 for all).
In general, there were no significant differences between the responses of administrators and investigators, with two exceptions. First, 91.8% of administrators believed that the ethics rules made it more difficult for the NIH to complete its mission, compared to only 79% of investigators (P<.01). Second, administrators reported a greater effect on faculty recruitment, with 78% citing a negative impact vs. 53% for investigators (P<.01).
The ethics rules were associated with a significant decline in GIRs among NIH faculty employed both before and since their implementation (n=541). As seen in Figure 2, the number of responding faculty who consulted to industry was reduced by three-fourths (33.1% before vs. 7.8% after, P<.01) and the proportion of respondents serving on scientific advisory boards was cut in half (31.5% vs. 16.0%, P<.01). Significant declines were also seen for NIH faculty serving as a company founder or as a member of a board of directors, as well as significant declines in the receipt of royalties and the prevalence of industrial funding for students. Among those with an existing GIR (n=280), 38.0% faculty terminated an industrial collaboration, and 195 (73%) declined an honorarium from a company (results not shown).
Figure 2 also shows that the ethics rules had little perceived impact on professional service roles of NIH faculty, except for a decline in the proportion of faculty serving in advisory roles for foundations (32.0% before vs. 24.9% after, P = .01). Although the ethics rules did not generally cause faculty to stop performing these activities, 47.9% respondents indicated that the new rules made it harder to be an editor of a professional journal and 64.9% claimed it was more difficult to be an officer in a professional association. Over one-third (36.3%) responded that they had limited attendance at scientific meetings because of the ethics rules.
To assess the impact of faculty involvement with NIH-supported research institutions, we asked about their relationships with academic colleagues. Per the NIH rules, NIH faculty were no longer able to accept honoraria or speaking fees from universities or academic medical centers that receive support from the NIH. NIH faculty may have been forced to end collaborations with academia if they were paid as a consultant or the principle investigator had an irresolvable conflict of interest. Alternatively, NIH faculty may have ended a collaboration because the burden of administrative rules overwhelmed the benefit of the partnership. In comparison to the numbers for industrial collaborations above, only 10.0% of faculty terminated an academic collaboration while 71.2% declined an honorarium from an academic institution.
Faculty with GIRs averaged a significantly higher rate of publication than their peers who did not have GIRs (6.7 vs. 3.8 articles per year, P<.01, data not shown in tables). There was no change in publishing behavior of NIH faculty before and after the implementation of the ethics rules (5.29 articles per researcher per year from 2002–2005 vs. 5.26 articles per year from 2005–2008; P = .88). A differences-in-differences regression failed to detect any change in publishing productivity between faculty with and without previous GIR (P = .25). Similarly, there was no statistical difference in the proportion of faculty applying for a patent before and after the policy change (P = .39).
Beliefs among surveyed NIH intramural faculty (n=566) about the potential effects of relaxing the ethics rules were mixed. As shown in Table 2, 32% of respondents believed that a repeal of the ethics rules would increase bias in NIH science and 42% felt it would lead to more secrecy among faculty. While 30% staff members indicated that such repeal would be likely to overemphasize applied investigations, 84% believed it would lead to new research that would not otherwise be possible. When asked whether the ethics rules should be extended to academic scientists to reduce conflicts of interest, the majority of faculty believe they should apply to academic scientists who are NIH grantees (58.7%) and nearly half felt they should apply to all scientists in academic institutions (47.3%); these views about extending the NIH ethics rules did not differ significantly by title or prior relationship with industry.
The 2005 ethics rules implemented by the NIH represented a radical shift in managing industry–research relationships. Unlike open-disclosure practices commonly used to manage industry relationships in health care arenas such as physician practice patterns and continuing medical education, the NIH rules presume that all industrial collaborations are inappropriate unless proven otherwise and require advanced permission before they begin. As such, the policy fundamentally modifies the calculus by which the balance of risks and benefits are measured in research relationships.
This report makes three important contributions to this debate. First, this study assesses the perception of research integrity among NIH faculty. The overall results suggest a strong association between industrial research relationships and perceptions of conflict of interest. Approximately one in ten faculty questioned the validity of research as a result of a peer faculty’s involvement with industry, and nearly one-third believe that relaxing the ethics rules would lead to increased bias and secrecy, or place an undue emphasis on applied research.
The second major contribution of this study is documenting the prevalence of GIRs at the NIH, before and after the implementation of the 2005 ethics rules. While previous studies have shown the pervasiveness of corporate affiliations within academia, this is the first empirical study to document a strikingly similar proportion of GIRs among NIH faculty. A comparable survey of 3080 academics from the top 50 grant-receiving institutions in 2006–2007 suggested that 52.8% of academic research faculty maintained some form of relationship with industry, while the data presented above show that 51.8% of all NIH research faculty were linked with industry before the introduction of 2005 ethics rules.9 Our study further documents that even in the government research sector, the most productive scientists are most likely to have relationships with industry; NIH faculty linked to industry published at nearly twice the rate of their peers (6.7 vs. 3.8 articles per year), a ratio that is nearly identical to that seen in academia.9 Unfortunately, the cross-sectional nature of this study does not allow us to test whether industry relationships make scientists more productive or whether industry seeks out the most productive scientists with whom to collaborate.
Third, this study helps estimate the impact of the 2005 Ethics Rules. In general, the policy seems to have accomplished much of what it was intended to do: limit relationships with pharmaceutical, medical device, and related organizations while maintaining the researchers’ association with non-industry related external scientific and professional organizations. The rules did not eliminate GIRs – one-third of all NIH faculty still work with industry in some capacity (and it is important to note that this does not imply these faculty are in violation of federal regulations). This large remainder may reflect the agency’s recognition of the importance of limited external partnerships to clinical development and technology transfer. But the proportion of faculty who consulted was reduced by three-fourths and the fraction serving on scientific advisory boards was cut in half. Simultaneously, there was no significant drop in the researchers’ participation in professional service roles. Our findings suggest that a ban on industry relationships can be achieved without harming other important relationships—at least in the governmental setting.
However, the 2005 ethics rules evoked several unintended consequences that may lead to negative long-term implications for the NIH. Most appreciably, they have substantially reduced personnel morale, job satisfaction, and the progress of faculty research. Over 80% of all faculty believe the new rules are too restrictive, including 72.7% of those without prior industry relationships.13 While documented instances of recruiting and retention problems are limited,14 we found that NIH personnel, especially administrators, believe the rules are hampering successful recruitment of new faculty to the agency, presenting potential future workforce challenges. This may become even more important as the current generation of aging researchers near retirement.15
Although the new rules did not eliminate professional and scientific affiliations, faculty believe that the rules have strained these connections. A large proportion of NIH faculty have terminated academic collaborations, limited attendance at scientific meetings, and reported that serving in professional roles has become more difficult. As indicated in the guiding principles of the ethics rules, this decreased interaction may hinder the ability of NIH faculty to stay on the forefront of pertinent scientific advancements.
Further, it is important to consider whether the ethics rules have limited valuable lines of new research and clinical development. Of the faculty who had industrial relationships, nearly half claimed their past industry relationships contributed to their most important work at the NIH.
The vast majority, including those without industry affiliations, believe that relaxing the conflict of interest rules would encourage new types of research. However, our analysis of average publication rates before and after the implementation of the ethics rules did not detect a significant decline in scientists’ productivity. Additional research is needed to more fully understand the long-term effects of the 2005 rules on the nature, productivity, and impact of NIH intramural research.
Several limitations should be noted. As with all survey research, this study suffers from weaknesses inherent with self-administered questionnaires. Respondents may have under-reported socially-undesirable relationships or behaviors, and their perceptions of whether or not their colleagues have industry relationships may not have been accurate. Further, the study sample consisted of currently-employed intramural faculty and the results are not directly applicable outside this population. To the extent that NIH websites were not updated regularly, the sample may have under-represented new faculty and excluded any scientist who left NIH because of the ethics rules. Lastly, it is important to emphasize that interim conflict of interest policies were discussed and promulgated within the NIH prior to enacting the final ethics rules in August 2005.
The issue of industry relationships and conflicts of interest has become increasing salient for universities and academic medical centers, especially in light of proposed federal regulations for NIH extramural grantees.16 These institutions may look to the NIH experience for guidance as they develop relationships with industry and disclosure policies. While the parallels to academic and other research settings are considerable, differences do remain. Public perceptions of credibility may be more important at the NIH, since it is funded through direct public investment. On the other hand, NIH faculty are not required to continually find and renew grants to support their salaries like many academics, potentially making them less susceptible to industry influences. Our data suggest that the NIH faculty themselves are evenly split in their own assessment of the applicability of these rules in a wider setting.
Overall, our results further underscore the delicate balance of the benefits and the risks of industry relationships with researchers. Our findings suggest that the same policy that increased faculty’s perception of public credibility of the NIH also made it more difficult for the organization to complete its mission. The success of more stringent conflict-of-interest policies will center on their ability to mitigate the negative aspects of industry relationships while maintaining incentives for collaboration and scientific development. These policies must consider both the intended and unintended consequences of industry research collaborations.
Dr. Darren E. Zinner, Heller School of Social Policy and Management, Brandeis University, Waltham, Massachusetts.
Dr. Catherine M. DesRoches, Mongan Institute for Health Policy, Massachusetts General Hospital, Boston, Massachusetts.
Ms. Steffanie J. Bristol, Institute for Health Policy, Massachusetts General Hospital, Boston, Massachusetts, at the time of this research.
Dr. Brian Clarridge, Center for Survey Research, University of Massachusetts, Boston, Massachusetts.
Dr. Eric G. Campbell, Mongan Institute for Health Policy, Massachusetts General Hospital, Boston, Massachusetts.