|Home | About | Journals | Submit | Contact Us | Français|
Objectives: The objectives were to examine the effectiveness of a curriculum designed to increase bioinformatics competencies of librarians with diverse backgrounds and to identify demographic factors that may affect the learning process.
Methods: Sixteen professional staff participated in a 12-week training course consisting of 5 distinct modules: molecular biology, genetic analysis, biotechnology, research literature, and databases. Participants completed a 120-question pretest and an 88-question posttest designed to evaluate the effectiveness of the training.
Results: Training was deemed successful as all participants scored higher on the posttest than the pretest. Data analysis was conducted in relation to participant background. Holding a biology degree did not seem to affect posttest results. Years of experience, however, had an impact on final scores in the databases section, as senior team members had greater difficulty learning the material.
Discussion: As the need for specialized information in the area of molecular biology and genetics becomes more central for the effectiveness of organizations, it is crucial for libraries to quickly align with those needs by having a clear vision for increasing the skills and competencies of their staff in this subject area. This paper describes an effective model for learning that was developed and tested by the Eskind Biomedical Library.
In the Association of American Medical Colleges report Learning Objectives for Medical Student Education, which discusses “the attributes that medical students should possess at the time of graduation,” the authors stress the importance of engaging future physicians in “lifelong learning to stay abreast of relevant scientific advances, especially in the disciplines of genetics and molecular biology” [1, 2]. The rapid advances in the area of bioinformatics and molecular biology create opportunities and challenges for librarians, because maintaining the high level of skills and competencies necessary to teach the access, retrieval, and use of relevant information is becoming more and more complex.
The role of the medical librarian in providing genomic and other biological information is still developing, and the success of biomedical libraries might well depend on their ability to expand traditional roles to meet these new needs [3–5]. Previous studies have indicated that librarians can provide a wide range of biological information services to researchers and that bioinformatics training and support is a widespread need, but such services require an increased expertise and knowledge in the subject area [6–11]. Work by Yarfitz and Ketchell indicates that less than 20% of bioinformatics questions could be answered by librarians without either a molecular biology background or specialized training due to the technical expertise required to answer these questions . The Eskind Biomedical Library (EBL) has recognized this need for specialized training as an important area for growth and expansion and has consequently spent the last three years creating a qualified library staff able to provide information support and education in bioinformatics. This paper reports on the EBL's effort, via a carefully designed learning plan, to increase the overall bioinformatics skills and competencies of its librarians.
The Eskind Biomedical Library is committed to “an institutional culture of lifelong learning” , as the competencies and skill sets of its employees are crucial to a quick, strategic adaptation to and alignment with the medical center [14–16]. The dynamic environment created by a philosophy of constant learning and skills enhancement ensures the continued relevance of individual expertise. This overall philosophy of learning and the formal training program were introduced in 1996 and are largely based on the concepts of adult learning and self-directedness [17, 18]. The training program accommodates the varying sets of skills that adult learners bring to the environment by allowing trainees to “test out” of modules or portions thereof, when relevant competencies and skills are demonstrated. To foster continual learning beyond the individual's initial learning plan, the EBL engages its employees in ongoing professional learning activities organized in-house as well as those offered throughout the medical center.
A group of highly focused, highly visible new initiatives has resulted from the EBL training program, and these initiatives have been important conduits for the continuing professional development of EBL staff. These projects place a strong emphasis on outreach and have successfully moved information specialists into clinical and research settings outside the library walls. In addition to their key role in the continuing training experience, these projects have greatly increased the visibility of the EBL in the Vanderbilt University Medical Center and the community at large, making librarians vital additions to multidisciplinary teams. The library's Clinical Informatics Consult Service (CICS) integrates EBL librarians into clinical teams to provide critical appraisal and synthesis of the literature in response to complex clinical requests, thereby facilitating evidence-based practices in clinical decision making [19–21]. To align EBL with the medical center goal of preeminent patient care, the library's Patient Informatics Consult Service (PICS) extends the concept of librarians as information consultants to provide appropriate information to health care consumers, such as patients and their families . Adapting the CICS model, the Research Informatics Consult Service (RICS) utilizes electronic resources, online information delivery, and outreach methods to provide information services to health sciences researchers at the point of need . The library “has long subscribed to the idea that librarians are uniquely qualified to locate and evaluate medical information”  and has attempted to meet some of these needs through programs such as CICS, PICS, and RICS. Through knowledge gained from administering these programs, EBL has recognized that for librarians to effectively and accurately meet patrons' increasing need for bioinformatics assistance, in-depth training in molecular biology and genetics would be needed.
In 2000, the Eskind Library started tackling the issue of bioinformatics training for librarians by developing and administering an eight-week introductory course. This overview course was primarily intended to expose librarians to the vocabulary and concepts of genomics and genetics and raise awareness of genetics databases .
Following this introductory course, three librarians and one library intern were selected, based on their willingness and interest, to pursue in-depth individual training with the instructor to increase the number of library staff with advanced molecular biology expertise. Conducted over one year, this high-level, two-stage training process focused on learning concepts in context . During the first stage, advanced trainees concentrated on reading and evaluating assigned research articles and focused on identifying unfamiliar terms and concepts that served as points of discussion with the trainer. Building on the acquired knowledge, trainees next honed database-searching and information-retrieval skills by finding answers to actual complex molecular biology questions previously received from users .
More recently, to align with medical center priorities for advancing the genomics research and education agendas, library leadership determined that the level of skills and competencies of information specialists needed to be more advanced and ubiquitous. Moreover, for librarians to have an impact in the medical center, researchers must understand and trust librarians' familiarity with and knowledge of bioinformatics and its resources. As a result, a more in-depth, formalized course was designed to: (1) elevate and update the overall level of skills and establish a broader base of molecular biology expertise in the library and (2) design and evaluate the effectiveness of a workable model for training librarians interested in fulfilling the role of the new bioinformationist.
Sixteen professional library staff members participated in the course. The study group consisted of fifteen individuals with master's degrees in library science and one professional nonlibrarian at the highest rank in the EBL health information analyst job family . Undergraduate degrees spanned a wide variety of disciplines, including literature and arts, social sciences, and natural sciences. For the purpose of blinding, Table 1reports these data as biology or non-biology degrees. The number of years of medical librarianship experience varied. Participants reported having taken zero to four previous molecular biology or genetics courses; a “course” was defined as an undergraduate class or previous EBL molecular biology training. Table 1summarizes study participants' demographic data.
One of the authors (Lyon), a librarian trained as an intern at EBL and with three years of library experience, designed and taught this in-house course based on her in-depth knowledge and experience in the areas of genetics and molecular biology. Prior to receiving her master's of library science degree, she completed a master's of science in molecular biology and had eight years of research laboratory experience. She has trained medical librarians in molecular biology for the Medical Library Association and the National Center for Biotechnology Information (NCBI) at the National Library of Medicine. Additionally, she contributed to the development of the NCBI's Advanced Workshop for Bioinformatics Information Specialists (NAWBIS) online course.†
The EBL Molecular Biology Module goals were to introduce basic science knowledge, provide an understanding of research literature, and present an overview of molecular biology databases with a focus on those maintained by the NCBI. Following a well-established and tested EBL rationale, this course allowed for gradual but in-depth learning of concepts and principles in genetics and molecular biology. The five sections of the course were arranged so that students progressed from vocabulary and basic subject knowledge to the ability to search, retrieve, and extract relevant information from articles and data sources via a filtering process. Specific sections of the module included: molecular biology, genetic analysis, biotechnology, research literature, and databases. Participants completed a pretest and posttest to establish baseline knowledge and level of knowledge gained through the course. Module content was finalized based on pretest results (see the appendix for additional details of course content).
All participants completed a pretest designed to ascertain knowledge levels in the course content and, particularly, absence of knowledge. In conjunction with the pretest, students filled out a background questionnaire to collect demographic information.
The pretest consisted of 120 written questions divided among the five previously identified sections. The answers required written prose by each of the trainees. To more accurately assess the subjects' preexisting knowledge, a multiple-choice format was intentionally avoided to prevent the participants from guessing. Questions varied in complexity and difficulty. Examples of pretest questions are provided in Table 2.‡
The pretest was administered as a closed-book exam through an EBL-developed online tool, the Learning Module Shell. The shell is a flexible, Web-based tool for presenting educational materials incorporating interactive elements such as multimedia and quizzes . The instructor scored each section separately; a point score and percentage score for each participant were recorded in the online module tool. All participants were allowed to view their own scores and were provided with feedback on their answers. A twelve-lecture series was then created based on the needs identified by the pretest.
The series of Molecular Biology Module lectures were offered one hour per week for twelve weeks. To ensure each participant completed the series, each lecture was offered twice a week and make-up sessions were conducted as needed. The lectures were taught in short, modular sections covering the same five main topics as the pretest: (1) molecular biology, (2) genetic analysis, (3) biotechnology, (4) research literature, and (5) databases.
Online modules containing the lecture content were made available through the Learning Module Shell and served as a teaching tool both for the sessions and further study. Each online module contained written text, illustrations, and practice quiz questions as well as reading assignments. The database portions of the module were linked directly to the corresponding sections of the NAWBIS online course. Course assignments included reading materials (textbook and articles), puzzles (crossword and word search), exercises, and literature evaluation. These assignments were designed to assist in the learning process in a more relaxed, fun way, particularly regarding the assimilation of new terminology.
Participants who scored 80% or more on any pretest section were allowed to opt out of the corresponding lecture or lectures and told of the requirement of taking the corresponding posttest section. They were encouraged to meet with the course instructor to discuss any questions they had on the subject material and to review the relevant online and text materials on their own time. All participants reported spending an average of one to two hours per week studying independently for the course.
Following the twelve-week series of lectures, the course instructor administered an online posttest of eighty-eight questions that closely adhered to the material used during the course lectures. As in the pretest, questions varied in difficulty. The posttest was also administered as a closed-book online exam through the Learning Module Shell and was divided into the same topics as the pretest and module.
The databases section was subdivided to serve as a more in-depth verification of the material taught. While this section of the pretest was designed to determine which databases the participants had knowledge of, the corresponding section of the posttest was designed to assess familiarity with the specific databases taught in the course.
The instructor scored each section separately; a point score and percentage score for each participant was recorded in the online module tool. At the end of the posttest, a fifteen-question survey was administered for input on future design and refinement of the course.
Based on pretest results, three librarians tested out of the module lectures as follows: participant #7 tested out of all sections except research literature; participant #11 tested out of both genetic analysis and databases; and participant #15 tested out of all sections. These three individuals had received in-depth individual training in the past two years as previously discussed.
All trainees showed improvement in their scores between the pretest and the posttest. Figure 1 shows the average percentage scores for all sixteen participants by test section for both tests. For example, on the molecular biology section, the average pretest score was 46%, which increased to an average of 89% on the posttest. The most dramatic increase in the average percentage score was in the research literature section (17% pretest to 85% posttest), while the smallest increase in the average score was seen in the databases section (31% pretest to 66% posttest).
Figure 2 shows a more detailed analysis of individual progress for all sixteen participants. An increase in posttest scores for all participants averaged over all sections of the module is clearly demonstrated. Three participants (#7, #11, and #15) scored at least 90% on the posttest. Three participants scored less than 10% on the pretest and increased their posttest scores to greater than 60%. One of the three (#16) scored more than 70% on the posttest and thus showed the largest overall increase. The participants (#7, #11, and #15) who tested out of sections of the course also improved their posttest scores. The trends of improvement were the same across all basic science sections of the module: molecular biology, genetic analysis, and biotechnology.
The research literature section, when compared to the other module sections, showed the lowest pretest scores across all participants (Figure 3). These results were most likely due to the lack of previous exposure to this specific type of published literature. The questions in this section focused on unique aspects of basic science research articles, in particular evaluating new types and formats of data. Participant #16 had a score of 0 for this section pretest and increased to 95% on the posttest. Following our previously stated model for adult training, participant #15 having scored more than 80% on this section's pretest was exempted from the formal lecture on the research literature component. The results show that on the posttest, this individual failed to achieve the highest score, as participants #7, #8, #11, and #16 scored better than #15 on the posttest.
The lowest posttest scores were noted in the analysis of the databases section results (Figure 4). Participants who tested out of this module section (#7, #11, and #15) showed slight improvement in posttest scores. Six participants who scored 10% or less on the pretest showed marked improvement of at least thirty-five percentage points on the posttest.
A second data analysis, based on an observation by the instructor, was conducted on the posttest results from the databases section. Interactions during the training with the senior members of the team (more than 10 years' professional experience in medical librarianship) indicated that these participants had greater difficulty learning the material than the junior members. The results of the data supported this early impression as the posttest results for the senior members of the team were clearly lower than the junior members of the team (5 years' or less professional experience). An aggregate of these results are reported in Figure 5.
A final analysis of all pre- and posttest data was conducted in relation to participant background. The data was stratified by two factors: type of undergraduate degree (biology versus non-biology) and years of professional experience. Figure 6 shows the average pre- and posttest scores for participants grouped by these factors. Compared to the other groups, participants with biology undergraduate degrees and 5 years or less of professional experience performed better on the pretest. This group had only a slight advantage on the posttest when compared to participants with non-biology degrees and similar years of professional experience. Overall, the presence or absence of a biology degree did not seem to have a major effect on posttest results.
As information becomes increasingly specialized and new forms of data and databases are developed, librarians must adapt their skills and expertise to position themselves to meet their organizations' existing goals and to anticipate future needs. These concerns are critical in the area of biomedicine where rapidly accumulating genetic information is increasingly affecting the practice of medicine. It is therefore imperative that medical librarians proactively prepare to meet the changing information requirements of health care professionals.
The model of training described in this paper is enabled by a strong culture of proactive learning. It is essential that trainees are willing to incorporate the principles of lifelong learning into their daily professional practice. Further, leadership support is vital to the success of training efforts of this magnitude, given the time and resources required from all the personnel involved. Although the structure and the set-up for learning is likely to be unique in each library, the authors feel that implementing the training program described in this paper will be difficult without the support of a learning environment.
The results from this study demonstrate that medical librarians are capable of assimilating new and complex knowledge. Significantly, all participants improved their level of comprehension of molecular biology concepts and resources (Figure 1). Therefore, background factors such as years of professional experience in medical librarianship and level of previous education in the subject area should not deter the training of any individuals committed to learning. However, the possible impact of these factors should be considered in the process of developing the training course.
A question that is often asked is whether a previous degree in the biological sciences is necessary to develop skills in bioinformatics. The results of this study indicate that this is not the case, as shown in Figure 6. While a background in biology provided an advantage on the pretest for some participants; on the posttest, the advantage disappeared. Independent of undergraduate degree, if given the opportunity, all trainees were capable of increasing their competencies in molecular biology to similar levels.
An important observation was made during the study, because the results seemed to suggest that years of experience had an impact on the learning process. As discussed above, junior participants scored higher on the posttest than their more senior colleagues, as shown in Figures 5 and and6.6. The authors speculate that as the number of years of experience searching bibliographic databases increases, the transfer of searching skills to molecular biology databases that are very different in content and use becomes more challenging. The participants with five years or less of professional experience may have had higher average databases scores, because they are still familiarizing themselves with searching all types of databases. While the data support this hypothesis, this interpretation might be limited by sample size and will require further verification.
Interestingly, it was also noted that, in spite of the option of testing out of module sections, there was some evidence that the participants who were exempted would have benefited from attending the formal lectures. As reported in the results, a participant (#15) who tested out of the research literature section improved by only six percentage points and was surpassed on the posttest by other trainees (Figure 3). Although this is only a single example, it suggests that individuals with prior training may still benefit from repeating course material to reinforce understanding, especially in a dynamic area such as molecular biology.
The fact that the pre- and posttests consisted of subject-oriented factual questions is a limitation of the study. The training was not designed to simulate real life scenarios but rather to develop subject knowledge, as this foundation is clearly fundamental for understanding concepts and terminology. The absence of a clear understanding of basic concepts will have a negative impact on the ability of the information specialist to communicate effectively with users and earn their respect and trust. The team realizes the importance of practice with actual user questions and plans to address this in future training.
To maintain and continue the development of knowledge over time, additional training opportunities are being planned as an ongoing effort to constantly target skills and competencies in the field of molecular biology and genetics. Periodic open-book retention tests will be followed by refresher sessions that will emphasize material not well retained. Open-book format has been chosen, because it more closely mimics the actual situation in which librarians have information resources available. Therefore, this approach will evaluate trainees' abilities to find information at the time of need. Also, user query practice will be provided as part of the library's established continuing education sessions. The current plan is to make training in genetics and molecular biology mandatory in the learning plans of all new interns hired by the library. Given the complex and dynamic nature of the subject material, the course will be constantly refined based on the results of this study and comments received from all participants during training sessions.
The authors gratefully acknowledge the assistance of Margaret Westlake and Nila Sathe in the preparation of this manuscript.
Online resources include:
*Based on a presentation at MLA '03, the 103rd Annual Meeting of the Medical Library Association, San Diego, California; May 5, 2003.
†Information about the Advanced Workshop for Bioinformatics Information Specialists (NAWBIS) online course offered by the National Center for Biotechnology Information at the National Library of Medicine may be viewed at http://www.imcac.com/NAWBIS/.
‡Full copies of the tests are available by request by contacting Jennifer Lyon atJennifer.Lyon/at/vanderbilt.edu.