|Home | About | Journals | Submit | Contact Us | Français|
In a prospective study to explore connections between clinical information delivery and information retrieval, 41 Canadian family physicians searched an Electronic Knowledge Resource as needed for practice. Research software, called the Information Assessment Method prompted family physicians to report on the situational relevance, perceived cognitive impact, and application of their retrieved information hits. Both the Information Assessment Method and the Electronic Knowledge Resource needed periodic updating to properly address our research questions.
To determine the frequency of software updates when manual or semi-automatic approaches are used by family physicians.
Each family physician received a handheld computer (PDA) that ran the Windows Mobile 6 operating system. For technical reasons, the Information Assessment Method and the Electronic Knowledge Resource were accessed offline on PDA. To update the Electronic Knowledge Resource and the Information Assessment Method, family physicians were asked to synchronize their PDA to their PC. Updating the Information Assessment Method was a manual process, whereas updating the Electronic Knowledge Resource was semi-automatic.
We found: (1) about 25% of family physicians never or rarely updated PDA software on their own (2) a large number of software updates were never installed, and (3) the semi-automatic method was associated with a small increase in the proportion of installed software updates (58.9% versus 48.6% for the manual method).
When a wireless Internet connection is not used to update PDA software, sociotechnical issues complicate mobile data collection and data transfer.
“The only problem is software updating. I hate synchronizing my PDA to my computer, even more I find updating a daunting process.”Quote from one participant, received July 29 2008.
Implicit in any move towards evidence based practice is awareness of and access to current evidence. Electronic Knowledge Resources on mobile computer (PDA) are increasingly used by health professionals, with much variability among the types and methods of implementation and resulting effectiveness.1 Given our changing knowledge base, Electronic Knowledge Resources are dynamic and periodically updated. Methods for updating Electronic Knowledge Resources on PDA can be classified as manual, semi-automatic or wireless. Manual updates are initiated by the user. Semi-automatic approaches prompt users to update their PDA software upon synchronizing to a personal computer. Wireless installation allows end users to install new applications or update already installed PDA software without physically connecting to a computer. 2
While numerous empirical studies describe the use of Electronic Knowledge Resources and PDAs in clinical practice, none scrutinizes the frequency of PDA software updating. In this paper, we describe how often family physicians updated PDA software, and compare manual vs. semi automatic updating methods. Our findings are relevant to information providers and researchers considering mobile data collection, medical educators involved in e-learning projects and designers of clinical computing projects.
In a prospective observational study, a cohort of 41 family physicians searched an Electronic Knowledge Resource as needed for clinical information. They were asked to update two inter-related pieces of software on their PDA3. Our research objectives were to (1) examine to what extent family physicians retrieved clinical information they previously received on email, (2) compare ratings of clinical information received on email (push) versus retrieved on PDA (pull), and (3) explore whether family physicians purposefully or by serendipity, retrieved clinical information previously received on email. The study was approved by the McGill University Faculty of Medicine Institutional Review Board.
Our study software, the Information Assessment Method hooked into a commercial Electronic Knowledge Resource (Essential Evidence Plus, http://www.essentialevidenceplus.com) to allow rating information such as synopses of clinical research retrieved to address practice-based questions. The Information Assessment Method prompted participants to rate the situational relevance, perceived cognitive impact, and any application to practice of these information hits. The Information Assessment Method is the product of an 8-year research and development program4, summarized at http://iam2009.pbworks.com.
We emailed new synopses of published clinical research to family physicians, as they were released by the publisher.
At roughly two month intervals, the publisher released Electronic Knowledge Resource updates containing these new synopses, which family physicians could then retrieve as needed for practice. Since we wished to examine how push influenced pull through the retrieval of synopses that were previously read on email, family physicians were asked to use the most recently released versions of the Electronic Knowledge Resource and the Information Assessment Method. Failure to update the Electronic Knowledge Resource left family physicians without PDA access to recently released synopses. Failure to update the Information Assessment Method prevented evaluation of new synopses.
41 family physicians from nine of 10 Canadian provinces consented to the study. 36 were certificants of the College of Family Physicians of Canada. There were 24 men and 17 women, ranging in age from 28–70 years (median 44). All were in active practice. One family physician had no Internet access in their main patient setting, 37 (90.2%) reported high speed access, and three did not know what type of connection they had. In terms of computer self-efficacy, eight (19.5%) rated their level of skill as advanced, 32 (78.1%) as intermediate, and one as beginner. Early on, one participant dropped out before updating any software.
All participants were offered an HTC Touch Smartphone. However, 17 chose a device with no phone (the hp iPAQ 110). Both devices were Wi-Fi enabled, and used Windows Mobile 6. We performed the initial software installation. Family physicians were trained to synchronize their PDA to their PC for software updating and data transfer to our study server. Thus, no wireless connection was used and study software residedon the PDA. Upon synchronization of PDA with partner PC, an updater application automatically reminded participants to install Electronic Knowledge Resource updates.
Software to implement the Information Assessment Method on PDA was developed and pilot-tested with guidance from a private firm. By email and telephone, we reminded participants to install Information Assessment Method updates, released alongside each Electronic Knowledge Resource update. No updater application was available for the Information Assessment Method on the partner PC. Thus, updating the Information Assessment Method was a manual process requiring synchronization with a partner PC to reinstall six components (e.g., Microsoft SQL Mobile 2005).
Entry into the study began December 2007 and continued until May 2008. Data collection ended September 8, 2008.
From December 2007 to September 2008, six Electronic Knowledge Resource and five Information Assessment Method updates were released. We counted the number of updates installed by each participant in a time window spanning their start date until the end of data collection. We classified updates as either unassisted or assisted. Unassisted updates were defined as updates done by the participant without our help. For each participant, we compared the number of unassisted updates with the number of available updates for both the Electronic Knowledge Resource and the Information Assessment Method. We also counted assisted updates, defined as updates requiring the assistance of our research team. In the assisted situation, for each participant we calculated proportions for the number of updates done divided by the number of updates available.
Between December 10 2007, and September 8 2008, we received 1,374 rated searches containing 2,634 rated hits. Most family physicians (n=28, 70%) reported at least one sociotechnical problem, such as: (1) problems with synchronization software (2) trouble synchronizing at work due to lack of administrative rights, and (3) changes in PC operating system. On 19 occasions, these problems led us to recall PDAs to the research centre to update study software.
The variable start date created different numbers of updates available for each family physician. Figures 1 and and22 reveal the number of Electronic Knowledge Resource and Information Assessment Method updates made unassisted by each family physician (without help from the research team). In both figures, the number of installed updates is represented by the black bars, and ranked from least to most. We observed (1) about 25% of participants never or rarely updated on their own, (2) a large number of updates were never installed (light grey bars), and (3) the semi-automatic method was associated with a 10% increase in the proportion of installed software updates (n= 123/209, (58.9%) versus 83/171 (48.6%)).
On at least one occasion, eight of 40 participants needed assistance to update their Electronic Knowledge Resource (table 1). Eleven of 40 needed help to update the Information Assessment Method. With assistance, all participants updated at least once. After multiple email reminders and our ongoing support, 26 participants (65%) succeeded in downloading 60% or more of all Electronic Knowledge Resource updates. Only nine of 40 participants (22.5%) installed the final Information Assessment Method update in the last 30 days of data collection.
Some experts believe few physicians will manually update PDA software5. Our results support this belief. In this observational study, many participants did not update their PDA software, even though they consented and were trained to do so. The semi-automatic updating method was not much better than the manual method, perhaps because any potential advantage of semi-automatic updating can be realized only when personal and organizational factors are aligned. For example, a motivated family physician may not be permitted to install software in the workplace.
Sociotechnical approaches to IT implementation in health care consider the importance of learning how people, technologies and the process of care interact6. From an organizational perspective, IT problems can be ‘controlled at the source’ by restricting administrative rights to members of a technical support team. This approach to network security was one obstacle to mobile data transfer in our study.
From an individual perspective, the task of updating software for research is outside normal workflow. To optimize mobile data collection and data transfer for research, a wireless connection7 should be used. While some PDA software vendors now provide wireless mobile access to frequently updated content, this was not an option for us, as the Electronic Knowledge Resource we used was available offline only.
One limitation concerns generalizability, as our study sample was chosen by convenience. A sub-group of family physicians with high computer skills may have high rates of software updating on PDA. Secondly, the use of PDAs for research data collection cannot be understood in isolation from the organizational context. Our method of software updating was workable in our hands, but not in the hands of many participants within different organizations.
In searching bibliographic databases for studies reporting on frequency of PDA software updating (search strategy available on request), we found no empirical primary care studies on this topic. However, a growing body of literature describes the use of Electronic Knowledge Resources in health care, or the use of PDAs in clinical practice8,9,10,11,12,13. Many articles examine sociotechnical issues and how theoretical models are relevant to technology acceptance 14,15,16.
Our results suggest better methods are needed for PDA data collection in evaluation and research. The capacity for wireless updating of medical software on PDA seems essential.
In this paper, we observed how often family physicians updated two types of PDA software when a wireless connection is not used. In this context, sociotechnical issues complicate mobile data collection and data transfer, requiring frequent assistance. Our findings will hopefully alert researchers and developers to seek better solutions to software updating given the work context of health professionals17.
We thank Dr. Scott Strayer for reviewing and commenting on an earlier version of the manuscript.
Authors declare no conflict of interest with respect to any of the commercial products listed in this paper.
Roland Grad, Family Medicine, McGill University, 3755 Cote Ste Catherine Road Montreal Quebec H3T 1E2.
Pierre Pluye, Family Medicine, McGill University, 517 Pine Avenue West, Montreal Quebec H2W 1S4.
Vera Granikov, Information Science, Herzl Family Practice Centre, 3755 Cote Ste Catherine Road Montreal, Quebec H3T 1E2.
Janique Johnson-Lafleur, Family Medicine, McGill University, 3755 Cote Ste Catherine Road Montreal, Quebec H3T 1E2.