A power information user (PIU) model to promote information integration in Tennessee's public health community

J Med Libr Assoc. 2004 October; 92(4): 459–464.

PMCID: PMC521517

A power information user (PIU) model to promote information integration in Tennessee's public health community^*

Nila A. Sathe, MA, MLIS, Assistant Director for Research, Patricia Lee, MLS, Assistant Director for Fee-Based Services, and Nunzia Bettinsoli Giuse, MD, MLS, AHIP, Library Director Professor of Biomedical Informatics

Eskind Biomedical Library Vanderbilt University Medical Center Nashville, Tennessee 37232

Nila A. Sathe: nila.sathe/at/vanderbilt.edu; Patricia Lee: patricia.lee/at/vanderbilt.edu; Nunzia Bettinsoli Giuse: nunzia.giuse/at/vanderbilt.edu

Received January 2004; Accepted June 2004.

This article has been cited by other articles in PMC.

Abstract

Observation and immersion in the user community are critical factors in designing and implementing informatics solutions; such practices ensure relevant interventions and promote user acceptance. Libraries can adapt these strategies to developing instruction and outreach. While needs assessment is typically a core facet of library instruction, sustained, iterative assessment underlying the development of user-centered instruction is key to integrating resource use into the workflow. This paper describes the Eskind Biomedical Library's (EBL's) recent work with the Tennessee public health community to articulate a training model centered around developing power information users (PIUs). PIUs are community-based individuals with an advanced understanding of information seeking and resource use and are committed to championing information integration. As model development was informed by observation of PIU workflow and information needs, it also allowed for informal testing of the applicability of assessment via domain immersion in library outreach. Though the number of PIUs involved in the project was small, evaluation indicated that the model was useful for promoting information use in PIU workgroups and that the concept of domain immersion was relevant to library-related projects. Moreover, EBL continues to employ principles of domain understanding inherent in the PIU model to develop further interventions for the public health community and library users.

INTRODUCTION

Education of users is a core function of libraries, and promoting the effective use of resources and understanding of information concepts is central to the library's mission. Scherrer notes, in an analysis of the 1990–2000 annual statistics compiled by the Association of Academic Health Sciences Libraries, that “the average number of education sessions has doubled since the beginning of the decade, increasing from 74 in 1990 to 143 in 2000. Attendance at these sessions has shown a similar growth pattern” [1]. Similarly, Earl found in a 1996 survey that 75% of health sciences libraries offer formal instruction [2].

Traditional instruction in information skills typically entails didactic presentation of resource specifics or information concepts using problem-based health sciences examples and hands-on practice for trainees. While these methods facilitate raising awareness of resources, such single sessions are often not effective for developing robust understanding or promoting integration of information-seeking concepts and resource use into trainees' workflow. Other investigators have assessed the effectiveness of information skills training with similar conclusions: In a systematic review of quantitative and qualitative studies examining information skills training for health care professionals, Brettle notes that while participants generally value training and believe that their skills improve, there is “very limited evidence to show that training does improve search skills” [3]. Similarly, Garg finds that training has “some effectiveness in improving health professionals' searching skills,” though the small number of studies included in the review, and the limited quality of the studies themselves, could not demonstrate clear evidence of the training's effect or the longevity of gained knowledge [4].

Further, training's effectiveness may be hampered by the necessity of teaching to heterogeneous user populations composed of varying disciplines, levels of technical skill, and information needs. These factors make it more challenging to make training highly relevant by pinpointing user contexts and needs. Thus, training sessions may more often resemble the oak tree than the acorn users seek.

The Eskind Biomedical Library's (EBL's) recent work with the public health community in Tennessee underscores the need to examine traditional training practice. This paper describes the EBL's development of an educational model that incorporates best practices from traditional information skills training with deep domain immersion and understanding to facilitate long-term integration of resources into workflow.

BACKGROUND

In 1999–2000, EBL embarked on a project to raise awareness of electronic resources in the Tennessee public health community with an outreach grant from the National Library of Medicine (NLM) Partners in Information Access program. Project activities included an extensive assessment of information needs and dialogue with the community via focus groups to guide training sessions and Website development [5, 6]. During the project, EBL trained 198 public health staff in resources addressing critical needs. Immediate post-training evaluations revealed a high level of satisfaction with the sessions; however, the limited number (n = 60) of returned evaluations 6 months after training indicated that, though half of respondents reported retaining 40% of the session material, few participants ranked their knowledge of resources covered in the training sessions as “sufficient” or “considerable” after 6 months. Table 1 provides a summary of selected 6-month-post-training data.

Table 1 Self-ranking of expertise for selected resources from participants' evaluation of training six months later (N = 60)

Such data support the intuition that the typical “catch-all” training session is not effective for developing a robust level of understanding. While such training sessions are highly useful for raising awareness of resources and formally introducing concepts of information seeking, without continual practice, new knowledge inevitably fades. Furthermore, full integration of knowledge necessitates changing work habits in many user populations. Speaking of the public health population, Rambo similarly notes, “incorporating … external information resources into established work patterns is, after all, a change in behavior for many public health practitioners. Changing behavior requires considerable motivation, which may include endorsement or adoption by opinion leaders” [7]. To elucidate a model for training that better encompasses user workflow and promotes long-term integration of information use, EBL investigators extended their work with Tennessee's public health community to develop a model for long-term integration of information resources.

OVERCOMING KNOWLEDGE FADE: THE POWER INFORMATION USER MODEL

Beginning in 2001, EBL targeted selected public health professionals to train as power information users (PIUs)—members of the public health community with an advanced understanding of information use and resources and the ability to share their knowledge with others [8]. Potential PIUs were identified by state health department officials based on aptitude for learning, willingness to act as a peer resource, position as community leaders, or position on project teams that could benefit from information integration. Though five candidates were initially targeted for the PIU training, conflicting schedules and changing resource demands allowed investigators to work extensively with only two PIUs, both of whom held influential positions in their departments.

Training development

Using ethnographic methods, EBL investigators observed PIUs at work in numerous workgroup settings. Nearly 100 hours of observation over the project period enabled the development of need-driven process analyses, breakdowns of critical work functions that supported the essential goals and objectives of the targeted group and enabled investigators to identify key areas of workflow that would benefit from information integration. The appendix shows one such analysis.

Training and consultation in the post-observation period combined a theoretical focus on concepts of information seeking and data- and evidence-driven problem solving as well as applied training in relevant resources. The process analyses typically focused around specific PIU tasks and, informed by the investigators' observation of workgroup meetings and general workflow, guided the development of highly relevant, context-specific training. This contextual training generally occurred over multiple sessions and involved investigators' querying PIUs to determine the sources they would ordinarily use to accomplish their assigned tasks. Investigators then focused the task by dissecting major concepts and formulating specific research questions. Framing a research question was an unfamiliar exercise to the PIUs, but, in many cases, this process identified the primary sources for needed data—a surprise for the PIUs. Table 2 illustrates several of the resources and concepts discussed in training sessions.

Table 2 Selected material from power information user (PIU) curriculum

Similarly, while the PIUs had extensive knowledge of public health agencies and the data and publications produced by these agencies, they often did not understand how the digital information environment was organized or how they should select and use optimal retrieval tools. Training helped to coalesce the PIUs' knowledge and make more evident the links between information needs, specific strategies for information seeking, and sources of information. This dual focus helped to promote a mindset embracing lifelong learning as well as key skills needed to navigate the information environment.

Evaluation

Evaluation of PIU training and information integration included qualitative measures such as feedback via questionnaires, interviews, and a survey of PIU progress expressed by work products. Investigators collected data using taped interviews, questionnaires, and a survey of Websites and bookmarks on PIUs' computers. As noted in other studies, a combination of methods yielded both a longitudinal perspective on PIU information use as well as assessment of recent activities [9, 10].

Whereas the PIU model proved to require a greater time commitment and attention to logistics than investigators anticipated, PIUs reported incorporating many of the information fundamentals they learned into their workflow. For example, PIUs noted locating, assessing, and synthesizing more than seventy-five unique journal articles and documents that directly contributed to the development of a statewide report on health care access for individuals with limited English proficiency. PIUs also altered their work practices to better incorporate knowledge sharing and reuse. In addition to establishing a central repository for work products, PIUs also shared information with their communities by contributing information expertise and resources to work products in responsibilities beyond their immediate purview.

Though PIUs were interested in knowledge management practices, evaluation of the PIU model's efficacy in this small group also underscored the importance of an organizational culture that supported learning to achieve optimal diffusion of gained knowledge. Knowledge management practices, such as sharing a central server housing materials for reuse in producing or enhancing work, were an innovation for PIUs. Previously, groups worked on project aspects— such as collecting primary data, interpreting resources, and synthesizing and applying gained knowledge—in isolation from other potentially useful resources or data gathered by others. Depositing individual materials to create a pooled collection accessible to all members transferred ownership and raised concerns about individual contributions and recognition. PIUs, however, acknowledged productivity gains from such knowledge sharing.

Additionally, investigators conducted extended evaluation of the PIU model by examining the impact PIUs had on peers' information seeking through interviews with individuals identified by PIUs. Given changes in the political structure of the state and health department, few individuals who had worked directly with PIUs were still employed by the Tennessee Department of Health at the time of this evaluation. Investigators were, however, able to interview nine individuals identified by the original PIUs.

Interviews revealed that the PIU concept raised awareness and use of resources and helped to make primary literature more easily accessible to this small, targeted group. Interviewees remarked on locating and accessing documents to support work in areas such as bioterrorism and grant writing and noted using novel resources. PIU interaction also facilitated interest in concepts of information organization and management. Individuals who had worked with PIUs expressed an understanding of and further interest in advanced concepts such as knowledge management and data analysis.

Moreover, sustained consultation with PIUs should continue to promote information sharing. Such continuing relationships with librarians have been cited as key to long-term use of information in practice in other areas of health sciences [11]. Transitioning the PIU model from onsite consultation to remote, just-in-time conferencing has enabled librarians to keep abreast of issues and needs in the public health community and should allow PIUs to continue to build their information skills.

DISCUSSION

Though the number of individuals involved in the PIU Project was small and consultation periods were fragmented by PIUs' frequent, job-related travel, this intensive level of interaction provided investigators with a fuller understanding of the day-to-day workings of public health practitioners. Such domain knowledge has long been used in the informatics arena to inform the development of focused interventions. Forsythe, for example, participated in numerous studies illustrating the usefulness of an independent observer in gaining domain understanding [12–16]. Similarly, numerous investigators have used observation and work sampling to implement and evaluate informatics interventions, particularly order entry systems [17–31].

The PIU model incorporates sustained observation to develop a sound understanding of user needs and illustrates the applicability of domain immersion for developing library instruction and outreach. The PIU model also fosters in-house sources of expertise—individuals within the community itself who are committed to acting as information resources and championing information integration. Cultivating “champions” to facilitate user buy-in is likewise critical to longstanding success in integrating interventions, be they physical systems or new ways of thinking.

Several investigators described the importance of involved users in designing and implementing order systems relevant to users needs [32–35], and other authors highlighted the necessity of acceptance and ownership from the target population in promoting learning. In projects to foster Grateful Med use, both Burnham and Dorsch noted that onsite involvement and onsite liaisons who were particularly invested in understanding and using information technology were key to ensuring success [36, 37]. Participants at an NLM-funded forum discussing public health outreach projects also advocated that “a thorough understanding of information needs…should underlie all outreach” [38].

EBL has incorporated these principles of immersion and meeting of information needs in context in programs such as the Clinical Informatics Consult Service (CICS), which integrates informationists on inpatient rounding teams, where they field complex questions and deliver highly targeted, synthesized evidence [39–42]. Lessons learned from the PIU model and the CICS program have also supported EBL's rethinking of educational outreach in all arenas. Instead of providing regularly scheduled classes focused on single resources, EBL now offers instruction largely by appointment with content customized to individual trainees or specific workgroups. Librarians provide intensive training directly focused on needs identified by trainees, bringing the librarians' expertise to bear on the user's particular questions and expectations for the resource and noting, when relevant, additional resources or methods to integrate use into workflow. Such intensive work with trainees fosters connections between the library and users and increases the librarians' understanding of user workflow and the relevance of future interventions. Further, this type of training draws on the PIU model of creating local cells of expertise that serve to diffuse knowledge among workgroups.

EBL's recent creation of a public health–centric digital library^† also draws on the PIU experience. Knowledge about the community's workflow and information-seeking styles are incorporated in the design of this portal to relevant resources. The library has also applied lessons learned from the PIU project in adapting CICS consultation to the outpatient arena. Several months of observation of the workflow in Vanderbilt University Medical Center's (VUMC's) adult primary care clinic, as well as discussions with clinicians, have guided the library's modification of traditional CICS practice of rounding by an informationist. This observation has confirmed that typical CICS rounding does not mesh with the workflow of the outpatient clinic; however, a remote model that integrates with VUMC's electronic medical record system is proving successful in enabling just-in-time provision of evidence in this setting [43].

CONCLUSION

The PIU model, implemented on a small scale in this project, conceptualizes educational methods adapted from key informatics and library science practices. Librarians have been quick to adopt effective instructional measures such as problem-based learning and just-in-time electronic tutorials. Principles of integration in the user's domain and the formalized incorporation of in-depth analysis of workflow, as employed in the PIU project and by numerous other investigators, may also effectively inform and extend library interventions. While this type of analysis necessarily entails a significant amount of time and resources, the library must understand its users' needs thoroughly to provide truly responsive, relevant services.

Acknowledgments

The authors gratefully acknowledge the support of Wendy Long, deputy commissioner of the Bureau of Health Services, Tennessee Department of Health, for her invaluable support of the PIU project and Annette Williams and Rebecca Jerome for their editing assistance.

APPENDIX

Sample process analysis for the Limited English Proficiency (LEP) Workgroup

Assessment and evaluation of language needs of affected populations

Identify languages that are likely to be encountered in the health department and estimate the numbers of individuals who speak the identified languages
Identify the points of contact in the health department where language assistance is necessary
Identify location and availability of necessary resources to provide effective language assistance
Develop means to assess resources

Development of a comprehensive written policy on language access

Develop policies and procedures for obtaining and providing trained and competent interpreters and other language assistance services in a timely manner
–Hire bilingual staff trained in interpreting
–Hire staff interpreters
–Contract with an interpreter service for trained and competent interpreters
–Develop a formal arrangement for the services of voluntary community interpreters
Translate written materials
–Provide translated written materials including vital documents for prioritized language groups
–Develop mechanisms to provide oral translation of documents
Develop methods for providing notice of the right to access to translated materials to LEP persons
–Develop language identification cards that allow LEP persons to identify their language needs to staff
–Post and maintain signs in regularly encountered languages at points of entry
–Develop uniform procedures for timely and effective telephone communication between staff and LEP persons
–Ensure inclusion of statements about the services available and the right to free language assistance services in materials routinely disseminated to the public

Training of staff

Disseminate policy to all employees likely to have contact with LEP persons
Develop cultural competency training and training follow-up mechanisms for staff

Monitoring

Develop consumer satisfaction survey in appropriate languages to monitor and obtain feedback about services provided to LEP individuals at the health department
Monitor language assistance programs to assess the current LEP makeup
Assess efficacy of existing assistance in meeting needs of LEP persons
Evaluate staff knowledge of LEP policies and procedures

Footnotes

The Power Information User Project was supported by National Library of Medicine grant number N01-LM-6-3522, administered by the University of Maryland, Baltimore.

†

The Eskind Biomedical Library's public health–centric digital library may be viewed at http://www.mc.vanderbilt.edu/phdiglib/.

REFERENCES

Scherrer CS, Jacobson S. New measures for new roles: defining and measuring the current practices of health sciences librarians. J Med Libr Assoc. 2002. Apr; 90(2):164–72. [PMC free article] [PubMed]
Earl MF. Library instruction in the medical school curriculum: a survey of medical college libraries. Bull Med Libr Assoc. 1996. Apr; 84(2):191–5. [PMC free article] [PubMed]
Brettle A. Information skills training: a systematic review of the literature. Health Info Libr J. 2003. Jun; 20(Suppl 1):3–9. [PubMed]
Garg A, Turtle KM. Effectiveness of training health professionals in literature search skills using electronic health databases—a critical appraisal. Health Info Libr J. 2003. Jun; 20(1):33–41. [PubMed]
Lee P, Giuse NB, and Sathe NA. Benchmarking information needs and use in the Tennessee public health community. J Med Libr Assoc. 2003. Jul; 91(3):322–36. [PMC free article] [PubMed]
Lee P, Sathe NA, Lynch F, Martin S, and Giuse NG. The role of collaborative partnerships in a public health outreach program. Paper presented at: MLA/CHLA/ABSC 2000, 100th Medical Library Association Annual Meeting; Vancouver, BC; May 2000.
Rambo N, Zenan JS, Alpi KM, Burroughs CM, Cahn MA, and Rankin J. Public Health Outreach Forum: lessons learned [special report]. Bull Med Libr Assoc. 2001. Oct; 89(4):403–6. [PMC free article] [PubMed]
Lee P, Giuse NB, and Sathe N. Adapting ethnographic methodology to investigate workflow process to promote knowledge management practice in a public health environment. Paper presented at: SCMLA 2002, Southern Chapter of the Medical Library Association 52nd Annual Meeting; Nashville, TN; 2002.
Ash JS, Gorman PN, Lavelle M, Payne TH, Massaro TA, Frantz GL, and Lyman JA. A cross-site qualitative study of physician order entry. J Am Med Inform Assoc. 2003. Mar– Apr; 10(2):188–200. [PMC free article] [PubMed]
Ash JS, Stavri PZ, Dykstra R, and Fournier L. Implementing computerized physician order entry: the importance of special people. Int J Med Inf. 2003. Mar; 69(2–3):235–50. [PubMed]
Powell CA, Case-Smith J. Information literacy skills of occupational therapy graduates: a survey of learning outcomes. J Med Libr Assoc. 2003. Oct; 91(4):468–77. [PMC free article] [PubMed]
Forsythe DE, Buchanan BG, Osheroff JA, and Miller RA. Expanding the concept of medical information: an observational study of physicians' information needs. Comput Biomed Res. 1992. Apr; 25(2):181–200. [PubMed]
Osheroff JA, Forsythe DE, Buchanan BG, Bankowitz RA, Blumenfeld BH, and Miller RA. Physicians' information needs: analysis of questions posed during clinical teaching. Ann Intern Med. 1991. Apr 1; 114(7):576–81. [PubMed]
Aydin CE, Forsythe DE. Implementing computers in ambulatory care: implications of physician practice patterns for system design. Proc AMIA Annu Fall Symp 1997;677–81. [PMC free article] [PubMed]
Buchanan BG, Moore JD, Forsythe DE, Carenini G, Ohlsson S, and Banks G. An intelligent interactive system for delivering individualized information to patients. Artif Intell Med. 1995. Apr; 7(2):117–54. [PubMed]
Forsythe DE. Using ethnography to build a working system: rethinking basic design assumptions. Proc Annu Symp Comput Appl Med Care 1992;505–9. [PMC free article] [PubMed]
Geissbuhler A, Miller RA. A new approach to the implementation of direct care-provider order entry. Proc AMIA Annu Fall Symp 1996;689–93. [PMC free article] [PubMed]
Giuse DA.. Provider order entry with integrated decision support: from academia to industry. Methods Inf Med. 2003;42(1):45–50. [PubMed]
Murray MD, Rupp MT, Overhage JM, Ebbeler DE, Main JW, and Tierney WM. Multidimensional work sampling in an outpatient pharmacy. Pharm Pract Manag Q. 1995. Oct; 15(3):44–56. [PubMed]
Ash JS, Stavri PZ, Dykstra R, and Fournier L. A cross-site qualitative study of physician order entry. Int J Med Inf. 2003. Mar; 69.
Ash JS, Stavri PZ, Dykstra R, and Fournier L. Implementing computerized physician order entry: the importance of special people. Int J Med Inf. 2003. Mar; 69. [PubMed]
Ash J, Gorman P, Lavelle M, Lyman J, Fournier L.. Investigating physician order entry in the field: lessons learned in a multi-center study. Medinfo. 2001;10(Pt 2):1107–11. [PubMed]
Ash JS, Gorman PN, Hersh WR, Lavelle M, and Poulsen SB. Perceptions of house officers who use physician order entry. Proc AMIA Symp 1999;471–5. [PMC free article] [PubMed]
Korst LM, Eusebio-Angeja AC, Chamorro T, Aydin CE, and Gregory KD. Nursing documentation time during implementation of an electronic medical record. J Nurs Adm. 2003. Jan; 33(1):24–30. [PubMed]
Gamm LD, Barsukiewicz CK, Dansky KH, Vasey JJ, Bisordi JE, and Thompson PC. Pre- and post-control model research on end-users' satisfaction with an electronic medical record: preliminary results. Proc AMIA Symp 1998;225–9. [PMC free article] [PubMed]
Tange H, Van Der Linden H, Sas P, Beusmans G, Talmon J, Van Oosterhout E, and Hasman A. Towards a PropeR combination of patient records and protocols. Int J Med Inf. 2003. Jul; 70(2–3):141–8. [PubMed]
Buckeridge DL, Mason R, Robertson A, Frank J, Glazier R, Purdon L, Amrhein CG, Chaudhuri N, Fuller-Thomson E, Gozdyra P, Hulchanski D, Moldofsky B, Thompson M, and Wright R. Making health data maps: a case study of a community/university research collaboration. Soc Sci Med. 2002. Oct; 55(7):1189–206. [PubMed]
Tafazzoli AG, Altmann U, Burkle T, Holzer S, and Dudeck J. Integrated decision support in a hospital cancer registry. Artif Intell Med. 2002. Mar; 24(3):243–55. [PubMed]
Fontaine BR, Speedie S, Abelson D, and Wold C. A work-sampling tool to measure the effect of electronic medical record implementation on health care workers. J Ambul Care Manage. 2000. Jan; 23(1):71–85. [PubMed]
Chan W. Increasing the success of physician order entry through human factors engineering. J Healthc Inf Manag. 2002. Winter; 16(1):71–9. [PubMed]
Burkle T, Kuch R, Prokosch HU, and Dudeck J. Stepwise evaluation of information systems in an university hospital. Methods Inf Med. 1999. Mar; 38(1):9–15. [PubMed]
Schuster DM, Hall SE, Couse CB, Swayngim DS, and Kohatsu KY. Involving users in the implementation of an imaging order entry system. J Am Med Inform Assoc. 2003. Jul– Aug; 10(4):315–21. [PMC free article] [PubMed]
Massaro TA. Introducing physician order entry at a major academic medical center: II. impact on medical education. Acad Med. 1993. Jan; 68(1):25–30. [PubMed]
Ahmad A, Teater P, Bentley TD, Kuehn L, Kumar RR, Thomas A, and Mekhjian HS. Key attributes of a successful physician order entry system implementation in a multi-hospital environment. J Am Med Inform Assoc. 2002. Jan–Feb; 9(1):16–24. [PMC free article] [PubMed]
Larson RL, Blake JP. Achieving order entry by physicians in a computerized medical record. Hosp Pharm. 1988. Jun; 23(6):551–3. [PubMed]
Burnham JF, Perry M. Promotion of health information access via Grateful Med and Loansome Doc: why isn't it working? Bull Med Libr Assoc. 1996. Oct; 84(4):498–506. [PMC free article] [PubMed]
Dorsch JL, Landwirth TK. Document needs in a rural GRATEFUL MED outreach project. Bull Med Libr Assoc. 1994. Oct; 82(4):357–62. [PMC free article] [PubMed]
Rambo N, Zenan JS, Alpi KM, Burroughs CM, Cahn MA, and Rankin J. Public Health Outreach Forum: lessons learned [special report]. Bull Med Libr Assoc. 2001. Oct; 89(4):406. [PMC free article] [PubMed]
Giuse NB. Advancing the practice of clinical medical librarianship [editorial]. Bull Med Libr Assoc. 1997. Oct; 85(4):437–8. [PMC free article] [PubMed]
Giuse NB, Kafantaris SR, Miller MD, Wilder KS, Martin SL, Sathe NA, and Campbell JD. Clinical medical librarianship: the Vanderbilt experience. Bull Med Libr Assoc. 1998. Jul; 86(3):412–6. [PMC free article] [PubMed]
Florance V, Giuse NB, and Ketchell DS. Information in context: integrating information specialists into practice settings. J Med Libr Assoc. 2002. Jan; 90(1):49–58. [PMC free article] [PubMed]
Davidoff F, Florance V. The informationist: a new health profession? Ann Intern Med. 2000. Jun 20; 132(12):996–8. [PubMed]
Cahall M, Giuse NB, and Giuse DA. Transitioning a successful clinical informationist model from an inpatient to an outpatient setting. Poster presented at: MLA '04, the 104th Medical Library Association Annual Meeting; Washington, DC; May 2004.

Articles from Journal of the Medical Library Association : JMLA are provided here courtesy of
Medical Library Association