|Home | About | Journals | Submit | Contact Us | Français|
To encourage evidence-based practice, an Annals of Internal Medicine editorial called for a new professional on clinical teams: an informationist trained in science or medicine as well as information science.
The study explored the effects of informationists on information behaviors of clinical research teams, specifically, frequency of seeking information for clinical or research decisions, range of resources consulted, perceptions about access to information, confidence in adequacy of literature searches, and effects on decision making and practice. It also explored perceptions about training and experience needed for successful informationists.
Exploratory focus groups and key interviews were followed by baseline and follow-up surveys conducted with researchers and clinicians receiving the service. Survey data were analyzed with Pearson's chi-square or Fisher's exact test.
Comparing 2006 to 2004 survey responses, the researchers found that study participants reported: seeking answers to questions more frequently, spending more time seeking or using information, believing time was less of an obstacle to finding or using information, using more information resources, and feeling greater satisfaction with their ability to find answers. Participants' opinions on informationists' qualifications evolved to include both subject knowledge and information searching expertise.
Over time, clinical research teams with informationists demonstrated changes in their information behaviors, and they valued an informationist's subject matter expertise more.
In their 2000 Annals of Internal Medicine editorial, Davidoff and Florance called for a new role on the clinical care team: the informationist . This role was needed, they believed, to bring evidence to clinical practices facing continued growth of published literature, patient safety concerns, and general lack of time available to health care professionals. The Institute of Medicine (IOM) also noted that training and encouraging clinicians to identify and apply evidence was not the complete solution to improving practice . They too suggested that an informationist be part of the clinical team. Davidoff and Florance and the IOM thought clinical knowledge and experience, as well as strong information science and related technology skills, were required to perform this function.
The National Institutes of Health (NIH) Library hired its first two informationists—librarians with extensive expertise in a clinical or research specialty—in 2001 and assigned them to research teams in their fields of expertise. Over the intervening years, the informationist program grew, and currently the library has fifteen informationists who are members of more than forty NIH clinical and/or basic science research teams. In addition to being expert information scientists, some of the NIH informationists have doctoral degrees in basic biomedical sciences such as molecular biology, biochemistry, and neuroscience. All of them are expected to continue their education by taking graduate-level courses in the specialties they support and participating in the specialties' professional meetings, in addition to maintaining their skills as information scientists. (A more detailed description of the NIH informationist program was published in 2008 .) From the start, informal feedback from the research teams was positive; however, as the commitment to the program increased, a formal evaluation was suggested. The study reported here looks at the effects the NIH informationists have had on participating clinical research teams.
Because every patient at NIH is enrolled in a clinical trial that takes place over an extended period with several return visits and the costs of participation—including transportation, lodging, and meals—are covered by NIH, traditional measures of information effects in a hospital—such as differences in patient length of stay, money saved, or patient outcome—would not provide the data needed to evaluate the NIH Library's informationist program. As a result, the evaluation looked instead at changes over time in the information behaviors of the participating clinical research teams, not specific individuals. The training and experience needed to perform the informationist role effectively were also explored.
Numerous studies exploring the information behaviors of health professionals have underscored the need for better support for their information needs. Covell found that many information needs were not recognized by practicing physicians and others went unanswered . In 1991, analyzing information requests during clinical teaching, Osheroff reported frequent requests, but many required the synthesis of patient information and medical knowledge and thus were difficult to answer . A more recent taxonomy of fifty-nine barriers to information use illustrated the complexity of developing successful interventions . However, Gorman showed how the biomedical literature of 1994 could answer primary care information needs, and, more recently, Westbrook found that use of online information resources enhanced the accuracy of answers provided to typical clinical problems by experienced clinicians [7, 8]. Nonetheless, online databases and libraries, even when available, continue to be infrequently consulted due to lack of time and search skills .
The 2000 Annals of Internal Medicine editorial proved a stimulus to librarians already exploring ways to better integrate information into the clinical context. In the United States and internationally, academic medical centers and other health organizations established model informationist programs [3, 10–16]. A recent literature review found what appears to be two informationist maturation models: First is an embedded model, usually in a clinical setting, that initially focuses on traditional information services but later progresses to support the team's technical and informatics information needs as well. Second is the consultant or bioinformatics model, which begins with a strong technical focus and, over time, provides more personalized service . While a few institutions were true innovators, the literature review also confirmed that informationist programs, when considered as innovations, remained in the early adopter stage, with libraries trying out the idea “in a careful way” .
Findings from studies in clinical settings suggest that embedding informationists encourages questioning and that trained informationist librarians can perform critical appraisal of the literature comparable to clinicians [14, 19, 20]. These studies also show that program success requires technical expertise, service excellence, and commitment by management [3, 13]. In addition, success requires informationists to demonstrate domain knowledge, engage in continuous learning, and be completely embedded in the team . Studies of the bioinformatics informationist model, on the other hand, have been descriptive but suggest that subject knowledge (molecular biology in particular) in support of project-specific consultation, generalized training, and development of web portals for easy access to sequence analysis tools and other resources are key elements for success [11, 12].
Evaluation of programs employing both informationist models is needed but is challenged by the fact that these programs are inherently customized and targeted to small groups . However, it is still the case, as Schacher observed, that while the benefits of having the literature available at the point of care are clear, more and better data on the impact of informationists are needed to secure routine positioning of these professionals on health care teams . This study attempts to provide evidence of the impact that NIH's informationists have had on the clinical research teams with whom they work.
Specifically, this study attempted to answer the following questions:
NIH not only funds translational clinical and basic research through extramural grants and contracts, but also conducts laboratory research and hundreds of phase I or II clinical trials in its own intramural research program. Eighteen of the 27 NIH institutes and centers have intramural research programs. Most NIH clinical trials take place at the Bethesda, Maryland, NIH Clinical Research Center, a 242-bed, in-patient hospital with 90 day-stations for out-patient visits. Ninety-five percent of the researchers working in these laboratories and clinics reported seeking information themselves, and 91% reported they preferred doing so according to a 2005 information needs assessment . Institutional review board approval was not required for this study, because NIH does not require it for program evaluations conducted among its staff by central service organizations such as the NIH Library.
The study began with exploratory work to identify and describe NIH researchers' expectations, perceptions, and experience with the informationist concept. In October 2003, a qualitative research consultant conducted three ninety-minute focus groups with ten scientists, representing both clinical and laboratory staff, from one institute. About half had worked closely with an informationist. Topics included discovering new ways that an informationist might contribute to their work, gathering information to enhance the informationist role, and informing the design of the planned quantitative study that would reach a larger group receiving informationist services. Using the same semi-structured interview guide, the consultant also conducted three key informant interviews with the same institute's leadership. Focus groups and interview data were transcribed, and content analysis was conducted by the consultant to identify themes and patterns.
Following the focus groups and key informant interviews, the NIH Library contracted with a market research and consulting firm to develop and then implement a survey of NIH staff who had, or were about to have, an informationist on their team. The focus group findings informed the content and format of the survey. For example, the web survey format was chosen because it offered flexibility for the clinical researchers being studied, something noted as particularly critical by focus group participants. The survey method allowed for two iterations: a baseline and a follow-up survey eighteen to twenty-four months later. In January 2004, the survey was pretested by a small group of clinical researchers who worked with an informationist. The final survey was posted to the web in February.
To encourage survey response, lead researchers and other principal contacts from the participating clinical research teams were asked to send their teams a link to the web survey. While the survey was underway, when an informationist joined a new team, its members were asked to complete the survey; ultimately, individuals from 9 teams could respond to the survey (~150–200 individuals). A total of 74 surveys were completed by February 2005, when the baseline survey ended. Respondents were anonymous and were prevented from answering more than once. A definitive response rate is unknown, because the actual number receiving the survey was determined by the research team leaders and not shared with the study coordinators. However, given the size of the potential survey population (permanent staff and fellows who rotate in and out every few months), the estimated response rate was 40%–50%.
Initially, an attempt was made to use the new teams as a control group, so that comparisons could be made between groups who had worked with informationists for a year or more and those who had never worked with one. The intention was that new teams would complete the survey before the informationist came on board. However, team members were slow in responding. Ultimately, the time allowed for completing the survey meant that some or all members of every team had worked with an informationist before they responded to the survey. Consequently, no control was established, and valid inter-group comparisons were not possible.
The same survey was repeated during 4 weeks from mid-May to mid-June 2006. Because of the time required to obtain responses to the baseline web survey (13 months), the follow-up survey was conducted by telephone by a consultant who specialized in telephone surveys. To give the consultant an appropriate “call list,” the informationists were asked to identify all members of the clinical research team with whom they worked. Study coordinators validated the final list. One hundred seventy members of the 9 original participating teams were sent an email inviting them to participate in the survey. Eighty-four responded, a 49% response rate.
All NIH clinical research teams consist of a principal investigator (PI), one or more co-PIs, and several postgraduate research fellows, who spend several weeks or months with a team as part of their rotation through various intramural research groups in an institute. About half of all fellows at NIH are foreign nationals and are at NIH for two to five years as part of an international training program. PIs and co-PIs have medical degrees (MDs) and/or doctoral degrees (PhDs) and are the largest segment of a clinical research team. Fellows assigned to clinical research teams typically have PhDs; a few have MDs. They make up the second largest segment of a team, but they are transient. Other members of a clinical research team can include a research nurse or study coordinator, a staff nurse, a statistician, a data manager, a pharmacist, and a nutritionist.
Because of the nature of their appointments, the same fellows who responded to the survey in 2004 are highly unlikely to have taken the survey again in 2006. However, all other categories of respondents are permanent employees who generally remain part of the clinical research team for many years. Therefore, the likelihood that PIs or co-PIs, research nurses, and the other members of the team who responded to the survey in 2004 also responded in 2006 is high. Because the surveys were anonymous, there is no way to confirm this.
Responses to the survey questions were first analyzed for inter-year comparisons by the market research consultants who developed the survey (Table 1). Their analysis drew on their broad knowledge of the information management function but did not include detailed statistical analysis, such as cross-tabulation of two or more questions, and did not include significance testing. To conduct the desired analyses, the authors converted the consultants' Excel spreadsheets into JMP statistical software files (SAS, Cary, NC). For example, cross-tabulations were done between the question about the respondent's role on the team and several of the twenty-two other questions on the survey. Responses were analyzed using Pearson's chi-square test. In cases where cell values were less than five, Fisher's exact test was used (noted by †). Statistics were calculated using JMP, version 7.0.
Despite expressing overall satisfaction with their information-gathering capabilities, focus group participants reported difficulty sorting through the plethora of information and finding specific information that they needed. The suggestion that an informationist could be a solution to this dilemma was greeted with skepticism by participants who had no experience with the program. However, the few participants who had used this or a similar service remarked on the competence and trustworthiness of informationists and generally gave them high praise. The consultant compared this to Federal Express's early market research that indicated low interest or perceived need for overnight delivery. It was not until the service existed and people began using it that users understood its value.
After colleagues endorsed the concept, skeptics were more willing to discuss the potential benefits of a librarian or informationist on their team. Many saw advantages to informationists attending rounds and staff meetings. Customizing information services to the team's specific needs was a general preference.
Participants had difficulty articulating the personal characteristics or training that an informationist should have, given that most had never encountered one. They were more comfortable talking about desired skills, such as competence with technology, critical thinking, and knowledge of the scientific method. The one trait they did identify was “initiative.” The consultant conducting the interviews noted that the idea of an informationist approaching them in their workplace was “hugely appealing.” They especially welcomed someone who could suggest better ways to search, retrieve, and organize information.
For both survey iterations, PIs or co-PIs, fellows, and nurse researchers or study coordinators constituted the majority of respondents. In 2004, the proportion of respondents in each of these 3 categories was comparable. In 2006, however, almost 3 times as many PIs and co-PIs responded (45/84) than had in 2004 (16/74), while about a fourth as many nurse administrators responded (3/84) than had responded in 2004 (5/74). A similar number of fellows responded both years, 11/84 in 2006 and 14/74 in 2004. The number of respondents in the other categories of research team members remained about the same.
To address study questions related to whether having an informationist on a clinical research team resulted in information behavior changes, several questions were asked of team members about their ability to find answers to questions that arose in their work that they thought could be answered by a search of the literature. By 2006, 79% of all respondents reported pursuing answers to these types of questions more than 60% of the time, compared to only 42% of respondents in the baseline survey (P<0.001) (study question 1a) (question 2.1, Table 2). Much of this increase was driven by PIs and co-PIs. In 2004, 56% (9/16) of the PIs and co-PIs reported pursing answers in the literature to questions that arose in their work 60% or more of the time. In 2006, 84% (38/45) of PIs and co-PIs reported doing so. Fellows also were more likely to pursue answers that could be found by searching the literature. In 2004, 78% of fellows (11/14) reported pursuing answers to these types of questions 60% or more of the time; in 2006, 100% of fellows reported doing so. Looking at other team members—such as research and staff nurses, data managers, statisticians, nutritionists, and pharmacists—as a group, their likelihood of seeking information remained unchanged, 35% (6/17) in 2004, compared to 38% (5/13) in 2006.
Respondents also reported spending 37% more time each week searching the literature, retrieving materials, and reading 9.2 hours per week on average, compared to 6.7 hours per week in 2004. Despite this, they were significantly less likely to identify “time to look” as an obstacle in 2006 than they had in 2004 (47% in 2004 to 29% in 2006) (P=0.01) (question 2.2, Table 2). Further, respondents were as likely in 2006 to seek information themselves, rather than have someone else do it for them, as they were in 2004: 85% of the time in 2004 and 81% of the time in 2006. This is a somewhat lower percentage than the general population of NIH researchers, perhaps explained by the larger number of clinicians in the survey population than in the general NIH research population surveyed in 2005 . Respondents were more likely to use “electronic journals and databases” to find information in 2006 (80%) than in 2004 (69%) (question 2.3, Table 2).
When asked which databases they used to find information to answer questions that arose in their work (study question 1b), in 2004, 95% of all respondents reported using PubMed/MEDLINE (question 2.4, Table 2), and the range of reported use of other databases was 1%–27% (median 11%). By 2006, use of databases other than PubMed was much higher (range 8%–54%, median 24%). For example, Web of Science (P=0.001), Cochrane Library Reports (P=0.001), and GenBank (P=0.01) were used significantly more often in 2006 than in 2004. Large increases also were reported in the use of other protein and molecular biology databases in 2006 compared to 2004.
Looking just at PIs and co-PIs, the likelihood that they would consult Web of Science went from 25% (4/16) in 2004 to 62% (28/45) in 2006; 19% of PIs and co-PIs (3/16) reported using Cochrane to answer questions in 2004, compared to 58% (26/45) in 2006. Similar increases in use of both Web of Science and Cochrane were noted for fellows. Use of GenBank increased among PIs and co-PIs, and fellows as well. In 2004, 13% of PIs and co-PIs (2/16) and 36% of fellows (5/14) reported seeking answers to questions there. In 2006, 42% of PIs and co-PIs (19/45) and 64% of fellows (7/11) reported using it. Similar rates of increase were noted for other molecular biology–related databases. There was little or no difference in the rates at which other members of the team used additional databases. In 2006, PubMed was still the most often reported database consulted, with 96% (81/84) responding that they used it when questions that could be answered by the literature arose.
To explore whether informationists facilitated or improved access to information (study question 1c), respondents were asked to rate the frequency with which 3 factors caused unsuccessful information seeking (question 2.5, Table 2). In 2006, “lack of time” (P=0.02), “difficulty finding information” (P=0.01), and “insufficient training” (P=0.01) were all significantly lesser issues than they had been in 2004. Consistent with these findings, respondents felt “more satisfied” (P=0.01) with their ability to obtain answers in 2006 than they had in 2004 (question 2.6, Table 2), suggesting that informationists had increased the confidence of clinical research teams in adequately researching the literature (study question 1d).
Several survey questions examined researchers' perceptions and acceptance of the informationist program. To explore the informationists' role in clinical decision making (study question 1e), respondents were asked in which team tasks and activities their informationists participated (question 3.1, Table 3). There was a significant increase over time in the levels at which informationists participated in team activities, including “going on rounds,” “helping with searches,” and “summarizing and screening the literature.” Further, a notable percentage (more than 36% in 2006) reported their informationist was engaged in “other” tasks not on the list of options, suggesting that informationists were participating in a broader array of team activities than anticipated when the service began. Involvement in nearly all activities increased over time, again suggesting that informationists were consulted more often and for more types of tasks. Another indicator of increased team responsibilities was that, in 2004, only 40% of all respondents (30/74) reported working directly with the informationist as opposed to merely having an informationist assigned to their team, while, in 2006, 69% of respondents (58/84) reported working directly with the informationist.
To measure acceptance of the informationist on the team, the researchers asked those who had reported working directly with informationists if they would do it again (question 3.2, Table 3) and whether they would recommend an informationist to others (question 3.3, Table 3). Responses were highly positive on both iterations of the survey, but even more so in 2006 than 2004, for example, with 100% in 2006 reporting they would use the service again, compared to 80% in 2004.
Whether they reported using the service or not, respondents were presented with a list of potential benefits of the program and asked to select those they thought informationists offered (question 3.4, Table 3). Multiple responses were allowed. By 2006, there were significant increases in perceptions of benefits, including “added thoroughness,” “providing expertise in finding information,” and “help finding additional information.” The one option where an informationist was not generally seen as providing a benefit, “helping find information for patients,” is a function normally reserved for the nurse educator on NIH clinical research teams and therefore was not unexpected.
To answer the last study question about requisite training for informationists, the survey asked respondents to rank six competencies that informationists should have to be effective team members (question 3.5, Table 3). In both years, the first choice was consistent: “expertise searching information sources relevant to my clinical/research area.” Interestingly, by 2006 “specific knowledge of my clinical/research area” increased from the fourth to the second most important competency.
Findings from the survey responses supplied full or partial answers to the study questions and provided specific information about how informationists affected their teams and how the informationist program has matured over time. Generally, findings indicated that the presence of an informationist in the clinical research environment did help PIs or co-PIs and fellows effectively utilize both the growing number and increasingly complex biomedical resources.
Over the period between baseline and follow-up surveys, clinical researchers were more likely to pursue answers to their questions. Although time has been cited frequently as a leading obstacle for clinicians seeking information [6, 9], this study indicated time became less of an obstacle. This is even more notable considering that busy PIs and co-PIs made up a higher percentage of respondents in 2006 than in 2004. As was demonstrated in both the 2004 and 2006 survey results, it was generally the lot of fellows to spend a large percentage of their time engaged in information seeking to find answers to clinical questions. However, after 2 or more years with an informationist, PIs and co-PIs increased the likelihood that they would seek answers themselves when questions arose. In addition, information was viewed as easier to find despite both the constantly increasing numbers of journals, articles, and genetic sequences available in online databases, as well as the periodic changes in search features and interface design. These findings were especially interesting because the survey responses also showed time spent on information-related tasks actually increased by 37%. By 2006, survey respondents reported they were significantly more likely to pursue answers to questions and they were more satisfied with their ability to find needed information, findings suggestive of a positive effect over time from having an informationist as a team member.
Informationists also appear to have had a positive effect on the range of information services and resources consulted by PIs or co-PIs and fellows in response to an information need. In 2004, most survey respondents relied solely on PubMed as a source of information, but in 2006, the range and frequency with which other databases were consulted by PIs or co-PIs and fellows rose significantly. This change cannot be attributed to increased availability or classroom training, because all the listed resources had been available to NIH staff for several years prior to the first survey and because classroom training and one-on-one tutorials in the major end-user databases including GenBank were offered many times prior to the first survey. For example, Web of Science had been available at NIH since 1998, 6 years prior to the start of the study, with numerous training sessions held yearly, yet the data show use of Web of Science by the groups in this study doubled (54% reported using it in 2006, compared with 27% in 2004) over the 2-year period. Increased use of the Cochrane database (45% of group members reported using in 2006, compared to just 19% in 2004) is even more likely to be attributable to the informationists, because no formal classes in this resource had been offered at the NIH Library prior to or during the study period. Increased use of genetics databases is noteworthy but is probably more a reflection of the increasing importance of molecular biology and genetics to clinical researchers generally than of the influence of informationists.
Whether an informationist improved the clinical or research decision making and practice of team members was not clear. What was evident was that, over time, the vast majority of investigators thought their informationist added needed expertise, found information that they otherwise would not have found, added thoroughness, saved them time, and reduced their workload burden. To the extent that these perceived benefits improved decision making, informationists had an effect.
While expertise in searching information sources relevant to the team was consistently the informationists' most valued knowledge or skill, by 2006, the importance of the informationists' subject knowledge also was recognized. This finding suggests that teams initially adopted informationists because they wanted better access to the literature. Over time, however, the informationist's subject knowledge was recognized as necessary if all the perceived benefits were to be achieved. While training informationists in science and medicine had always been an essential component of the NIH informationist program , for many other informationist programs, it was merely desirable [24, 25]. The current study showed that clinical research teams themselves viewed it as key to the informationists' preparation.
Over time, the evolution and uptake of informationists' services was apparent (question 3.1, Table 3). While NIH informationists reached the entire team when they made presentations or participated in rounds, they also worked extensively with individual team members. By 2006, more individual team members, primarily PIs or co-PIs and fellows, were working with the informationist than had in 2004, showing that informationists had achieved greater penetration among MDs and PhDs on the team. Researchers also reported valuing several informationist program features significantly more in 2006 than in 2004. In addition, nearly all respondents reported they would use an informationist again and recommend one to others. The increased use and recognition of potential benefits no doubt fed each other, the result of the long-term relationships that informationists embedded in teams were able to build.
One of the more intriguing findings from the surveys was that for a large number of respondents, informationists were engaged in “other” team activities beyond those anticipated. Although the survey itself did not provide insight into what these “other” activities might be, the authors' experience with the program indicated that these activities included such things as creating large citation databases, conducting bibliometric analyses of grant-funded publications, developing web pages and wikis to facilitate communication within and outside the teams, compiling and indexing a video database demonstrating movement disorders, and facilitating use of protocol authoring software . Informationists' adoption of more of these duties with their groups supported the finding of the recent systematic review  that, with maturity, clinical informationist programs evolve to support more technical and informatics needs of clinicians and researchers.
Respondents were volunteers and, therefore, were not necessarily representative of the groups of which they were members. Also, survey methodology addresses people's perceptions rather than actual events. Thus, while most of the study questions related directly to the survey questions, some questions were answered only by inference. For example, whether having an informationist on the team improved decision making could only be inferred from respondents' perceptions of the informationist. This issue requires further study. In addition, conducting the two iterations of the survey in different ways—by web and by telephone—might have influenced responses. Further, the larger percentage of PIs and co-PIs responding to the second survey might have influenced results. The lack of a control group against which to compare the findings limited the ability to attribute changes in information behavior solely to the informationists' presence on the teams. For instance, while the number of electronic resources available to researchers at NIH did not increase appreciably during this period, the size, and therefore usefulness, of databases such as GenBank did change and might have influenced their use. Therefore, while the presence of informationists on NIH clinical research teams might be statistically associated with changes in the information behavior of clinical research teams over time, it was not necessarily causal.
Given the limitations of the surveys, the conclusions drawn from the findings were conservative. However, over time, informationists on NIH clinical research teams did appear to make a difference in scientists' information behaviors, particularly PIs and co-PIs, the people with whom they worked most. For example, there were significant differences in information behaviors between 2004 respondents and 2006 respondents in pursuing answers to questions.
Further, the study indicated that informationists are made, not born. It not only took time for them to become accepted, contributing members of the clinical care or research team, but their education and experience in the subject areas and settings in which they practice as informationists contributed to their involvement in team activities. Acknowledging this, it was still their expertise as information scientists that was most highly valued.
The findings lead to the conclusion that any library considering establishing its own informationist-type program should be prepared to invest staff time and money to maintain the information science expertise of their librarians, as well as develop their subject matter knowledge in the specialty areas of the clinical groups they support. Further, given the growing importance of molecular biology and bioinformatics to clinical researchers, shown by the increase in reliance on the key databases in the field, as the informationist's relationship with the clinical team develops, they may need to add bioinformatics skills to their armamentarium.
This study has attempted to demonstrate the impact over time that informationists can have on the groups with whom they work. Future research should focus on gaining a deeper understanding of the informationist on clinical and/or research teams, particularly the larger effects on health care quality and health economics. A controlled comparison of health teams with and without informationist members would contribute to this understanding. Additionally, qualitative studies, using interviews and observation, for example, could provide more details on why researchers choose to work with informationists and what they value about them. As the informationist role develops in health care settings nationally and internationally, it should be informed and shaped through both local and multisite studies.
The authors thank Douglas Joubert, NIH Library, and Jason Broadway, National Library of Medicine, for help organizing and analyzing data from the surveys.
*This research was supported in part by the National Institutes of Health Director's set-aside funds for program evaluation and by the National Library of Medicine.