Search tips
Search criteria

Results 1-25 (1312239)

Clipboard (0)

Related Articles

1.  Managing symptoms during cancer treatments: evaluating the implementation of evidence-informed remote support protocols 
Management of cancer treatment-related symptoms is an important safety issue given that symptoms can become life-threatening and often occur when patients are at home. With funding from the Canadian Partnership Against Cancer, a pan-Canadian steering committee was established with representation from eight provinces to develop symptom protocols using a rigorous methodology (CAN-IMPLEMENT©). Each protocol is based on a systematic review of the literature to identify relevant clinical practice guidelines. Protocols were validated by cancer nurses from across Canada. The aim of this study is to build an effective and sustainable approach for implementing evidence-informed protocols for nurses to use when providing remote symptom assessment, triage, and guidance in self-management for patients experiencing symptoms while undergoing cancer treatments.
A prospective mixed-methods study design will be used. Guided by the Knowledge to Action Framework, the study will involve (a) establishing an advisory knowledge user team in each of three targeted settings; (b) assessing factors influencing nurses’ use of protocols using interviews/focus groups and a standardized survey instrument; (c) adapting protocols for local use, ensuring fidelity of the content; (d) selecting intervention strategies to overcome known barriers and implementing the protocols; (e) conducting think-aloud usability testing; (f) evaluating protocol use and outcomes by conducting an audit of 100 randomly selected charts at each of the three settings; and (g) assessing satisfaction with remote support using symptom protocols and change in nurses’ barriers to use using survey instruments. The primary outcome is sustained use of the protocols, defined as use in 75% of the calls. Descriptive analysis will be conducted for the barriers, use of protocols, and chart audit outcomes. Content analysis will be conducted on interviews/focus groups and usability testing with comparisons across settings.
Given the importance of patient safety, patient-centered care, and delivery of quality services, learning how to effectively implement evidence-informed symptom protocols in oncology healthcare services is essential for ensuring safe, consistent, and effective care for individuals with cancer. This study is likely to have a significant contribution to the delivery of remote oncology services, as well as influence symptom management by patients at home.
PMCID: PMC3527220  PMID: 23164244
Cancer; Symptom management; Implementation science; Mixed-methods study; Guidelines based
2.  The Development of a Mobile Monitoring and Feedback Tool to Stimulate Physical Activity of People With a Chronic Disease in Primary Care: A User-Centered Design 
JMIR mHealth and uHealth  2013;1(2):e8.
Physical activity is an important aspect in the treatment of patients with chronic obstructive pulmonary disease or type-2 diabetes. A monitoring and feedback tool combined with guidance by a primary care provider might be a successful method to enhance the level of physical activity in these patients. As a prerequisite for useful technology, it is important to involve the end-users in the design process from an early stage.
The aim of this study was to investigate the user requirements for a tool to stimulate physical activity, embedded in primary care practice. The leading principle of this tool is to change behavior by self-monitoring, goal-setting, and feedback.
The research team collected qualitative data among 15 patients, 16 care professionals, and several experts. A prototype was developed in three stages. In stage 1, the literature was searched to identify end-users and context. In stage 2, the literature, experts and patient representatives were consulted to set up a use case with the general idea of the innovation. In stage 3, individual interviews and focus groups were held to identify the end-user requirements. Based on these requirements a prototype was built by the engineering team.
The development process has led to a tool that generally meets the requirements of the end-users. A tri-axial activity sensor, worn on the hip, is connected by Bluetooth to a smartphone. In an app, quantitative feedback is given about the amount of activity and goals reached by means of graphical visualization, and an image shows a sun when the goal is reached. Overviews about activity per half an hour, per day, week, and month are provided. In the menu of the app and on a secured website, patients can enter information in individual sessions or read feedback messages generated by the system. The practice nurse can see the results of all patients on a secure webpage and can then discuss the results and set personalized goals in consultation with the patient.
This study demonstrates that a user-centered approach brings in valuable details (such as the requirements for feedback in activity minutes per day) to improve the fit between the user, technology, and the organization of care, which is important for the usability and acceptability of the tool. The tool embedded in primary care will be evaluated in a randomized controlled trial.
PMCID: PMC4114510  PMID: 25099556
user-centered design; self-management; physical activity; accelerometry; remote sensing technology; primary health care
3.  Designing and evaluating an interprofessional shared decision-making and goal-setting decision aid for patients with diabetes in clinical care - systematic decision aid development and study protocol 
Care of patients with diabetes often occurs in the context of other chronic illness. Competing disease priorities and competing patient-physician priorities present challenges in the provision of care for the complex patient. Guideline implementation interventions to date do not acknowledge these intricacies of clinical practice. As a result, patients and providers are left overwhelmed and paralyzed by the sheer volume of recommendations and tasks. An individualized approach to the patient with diabetes and multiple comorbid conditions using shared decision-making (SDM) and goal setting has been advocated as a patient-centred approach that may facilitate prioritization of treatment options. Furthermore, incorporating interprofessional integration into practice may overcome barriers to implementation. However, these strategies have not been taken up extensively in clinical practice.
To systematically develop and test an interprofessional SDM and goal-setting toolkit for patients with diabetes and other chronic diseases, following the Knowledge to Action framework.
1. Feasibility study: Individual interviews with primary care physicians, nurses, dietitians, pharmacists, and patients with diabetes will be conducted, exploring their experiences with shared decision-making and priority-setting, including facilitators and barriers, the relevance of a decision aid and toolkit for priority-setting, and how best to integrate it into practice.
2. Toolkit development: Based on this data, an evidence-based multi-component SDM toolkit will be developed. The toolkit will be reviewed by content experts (primary care, endocrinology, geriatricians, nurses, dietitians, pharmacists, patients) for accuracy and comprehensiveness.
3. Heuristic evaluation: A human factors engineer will review the toolkit and identify, list and categorize usability issues by severity.
4. Usability testing: This will be done using cognitive task analysis.
5. Iterative refinement: Throughout the development process, the toolkit will be refined through several iterative cycles of feedback and redesign.
Interprofessional shared decision-making regarding priority-setting with the use of a decision aid toolkit may help prioritize care of individuals with multiple comorbid conditions. Adhering to principles of user-centered design, we will develop and refine a toolkit to assess the feasibility of this approach.
PMCID: PMC3937124  PMID: 24450385
Shared decision-making; Priority setting; Patient decision aid; Interprofessional care; Diabetes mellitus; Patient education; Medical informatics; Toolkit development; Study protocol; User-centred design; Qualitative methods
4.  Designing a patient-centered personal health record to promote preventive care 
Evidence-based preventive services offer profound health benefits, yet Americans receive only half of indicated care. A variety of government and specialty society policy initiatives are promoting the adoption of information technologies to engage patients in their care, such as personal health records, but current systems may not utilize the technology's full potential.
Using a previously described model to make information technology more patient-centered, we developed an interactive preventive health record (IPHR) designed to more deeply engage patients in preventive care and health promotion. We recruited 14 primary care practices to promote the IPHR to all adult patients and sought practice and patient input in designing the IPHR to ensure its usability, salience, and generalizability. The input involved patient usability tests, practice workflow observations, learning collaboratives, and patient feedback. Use of the IPHR was measured using practice appointment and IPHR databases.
The IPHR that emerged from this process generates tailored patient recommendations based on guidelines from the U.S. Preventive Services Task Force and other organizations. It extracts clinical data from the practices' electronic medical record and obtains health risk assessment information from patients. Clinical content is translated and explained in lay language. Recommendations review the benefits and uncertainties of services and possible actions for patients and clinicians. Embedded in recommendations are self management tools, risk calculators, decision aids, and community resources - selected to match patient's clinical circumstances. Within six months, practices had encouraged 14.4% of patients to use the IPHR (ranging from 1.5% to 28.3% across the 14 practices). Practices successfully incorporated the IPHR into workflow, using it to prepare patients for visits, augment health behavior counseling, explain test results, automatically issue patient reminders for overdue services, prompt clinicians about needed services, and formulate personalized prevention plans.
The IPHR demonstrates that a patient-centered personal health record that interfaces with the electronic medical record can give patients a high level of individualized guidance and be successfully adopted by busy primary care practices. Further study and refinement are necessary to make information systems even more patient-centered and to demonstrate their impact on care.
Trial Registration identifier: NCT00589173
PMCID: PMC3250934  PMID: 22115059
5.  Usability Testing of a Digital Pen and Paper System in Nursing Documentation 
Usability testing was used to evaluate whether a new technology, a digital pen and paper system, would be usable for hospital nurses. Twenty-one nurses in a Labor and Delivery unit were randomly assigned into two groups, and a crossover design was used to compare the digital pen and paper system to conventional pens. Data collection included observations, interviews, and a questionnaire. Results showed that nurses had a positive attitude toward the system and could foresee its potential benefits, but they found that in its current design the system had poor usability and interfered with nurses’ work practices. Usability testing provided important insight into the needs of nurses and the suitability of this technology. This study is an example of how a user-centered approach can improve our understanding of the real needs of nurses and contribute to the design of useful and usable technologies for healthcare.
PMCID: PMC1560675  PMID: 16779159
6.  User-Centered Design and Interactive Health Technologies for Patients 
Despite recommendations that patients be involved in the design and testing of health technologies, few reports describe how to involve patients in systematic and meaningful ways to ensure that applications are customized to meet their needs. User-centered design (UCD) is an approach that involves end-users throughout the development process so that technology support tasks, are easy to operate, and are of value to users. In this paper we provide an overview of UCD and use the development of Pocket Personal Assistant for Tracking Health (Pocket PATH), to illustrate how these principles and techniques were applied to involve patients in the development of this interactive health technology. Involving patient-users in the design and testing ensured functionality and usability, therefore increasing the likelihood of promoting the intended health outcomes.
PMCID: PMC2818536  PMID: 19411947
user-centered design; interactive health technologies; lung transplantation; self-monitoring; handheld computers
7.  Efficiency and Usability of a Near Field Communication-Enabled Tablet for Medication Administration 
JMIR mHealth and uHealth  2014;2(2):e26.
Barcode-based technology coupled with the electronic medication administration record (e-MAR) reduces medication errors and potential adverse drug events (ADEs). However, many current barcode-enabled medication administration (BCMA) systems are difficult to maneuver and often require multiple barcode scans. We developed a prototype, next generation near field communication-enabled medication administration (NFCMA) system using a tablet.
We compared the efficiency and usability of the prototype NFCMA system with the traditional BCMA system.
We used a mixed-methods design using a randomized observational cross-over study, a survey, and one-on-one interviews to compare the prototype NFCMA system with a traditional BCMA system. The study took place at an academic medical simulation center. Twenty nurses with BCMA experience participated in two simulated patient medication administration scenarios: one using the BCMA system, and the other using the prototype NFCMA system. We collected overall scenario completion time and number of medication scanning attempts per scenario, and compared those using paired t tests. We also collected participant feedback on the prototype NFCMA system using the modified International Business Machines (IBM) Post-Study System Usability Questionnaire (PSSUQ) and a semistructured interview. We performed descriptive statistics on participant characteristics and responses to the IBM PSSUQ. Interview data was analyzed using content analysis with a qualitative description approach to review and categorize feedback from participants.
Mean total time to complete the scenarios using the NFCMA and the BCMA systems was 202 seconds and 182 seconds, respectively (P=.09). Mean scan attempts with the NFCMA was 7.6 attempts compared with 6.5 attempts with the BCMA system (P=.12). In the usability survey, 95% (19/20) of participants agreed that the prototype NFCMA system was easy to use and easy to learn, with a pleasant interface. Participants expressed interest in using the NFCMA tablet in the hospital; suggestions focused on implementation issues, such as storage of the mobile devices and infection control methods.
The NFCMA system had similar efficiency to the BCMA system in a simulated scenario. The prototype NFCMA system was well received by nurses and offers promise to improve nurse medication administration efficiency.
PMCID: PMC4114445  PMID: 25100043
medication systems; medication errors; mobile applications; automatic data processing; nursing
8.  Usability Study of a Computer-Based Self-Management System for Older Adults with Chronic Diseases 
JMIR Research Protocols  2012;1(2):e13.
Usability can influence patients’ acceptance and adoption of a health information technology. However, little research has been conducted to study the usability of a self-management health care system, especially one geared toward elderly patients.
This usability study evaluated a new computer-based self-management system interface for older adults with chronic diseases, using a paper prototype approach.
Fifty older adults with different chronic diseases participated. Two usability evaluation methods were involved: (1) a heuristics evaluation and (2) end-user testing with a think-aloud testing method, audio recording, videotaping, and interviewing. A set of usability metrics was employed to determine the overall system usability, including task incompletion rate, task completion time, frequency of error, frequency of help, satisfaction, perceived usefulness, and perceived ease of use. Interviews were used to elicit participants’ comments on the system design. The quantitative data were analyzed using descriptive statistics and the qualitative data were analyzed for content.
The participants were able to perform the predesigned self-management tasks with the current system design and they expressed mostly positive responses about the perceived usability measures regarding the system interface. However, the heuristics evaluation, performance measures, and interviews revealed a number of usability problems related to system navigation, information search and interpretation, information presentation, and readability. Design recommendations for further system interface modifications were discussed.
This study verified the usability of the self-management system developed for older adults with chronic diseases. Also, we demonstrated that our usability evaluation approach could be used to quickly and effectively identify usability problems in a health care information system at an early stage of the system development process using a paper prototype. Conducting a usability evaluation is an essential step in system development to ensure that the system features match the users’ true needs, expectations, and characteristics, and also to minimize the likelihood of the users committing user errors and having difficulties using the system.
PMCID: PMC3626148  PMID: 23612015
Usability evaluation; self-management; patient participation; chronic disease
9.  Rationale, design, and implementation protocol of an electronic health record integrated clinical prediction rule (iCPR) randomized trial in primary care 
Clinical prediction rules (CPRs) represent well-validated but underutilized evidence-based medicine tools at the point-of-care. To date, an inability to integrate these rules into an electronic health record (EHR) has been a major limitation and we are not aware of a study demonstrating the use of CPR's in an ambulatory EHR setting. The integrated clinical prediction rule (iCPR) trial integrates two CPR's in an EHR and assesses both the usability and the effect on evidence-based practice in the primary care setting.
A multi-disciplinary design team was assembled to develop a prototype iCPR for validated streptococcal pharyngitis and bacterial pneumonia CPRs. The iCPR tool was built as an active Clinical Decision Support (CDS) tool that can be triggered by user action during typical workflow. Using the EHR CDS toolkit, the iCPR risk score calculator was linked to tailored ordered sets, documentation, and patient instructions. The team subsequently conducted two levels of 'real world' usability testing with eight providers per group. Usability data were used to refine and create a production tool. Participating primary care providers (n = 149) were randomized and intervention providers were trained in the use of the new iCPR tool. Rates of iCPR tool triggering in the intervention and control (simulated) groups are monitored and subsequent use of the various components of the iCPR tool among intervention encounters is also tracked. The primary outcome is the difference in antibiotic prescribing rates (strep and pneumonia iCPR's encounters) and chest x-rays (pneumonia iCPR only) between intervention and control providers.
Using iterative usability testing and development paired with provider training, the iCPR CDS tool leverages user-centered design principles to overcome pervasive underutilization of EBM and support evidence-based practice at the point-of-care. The ongoing trial will determine if this collaborative process will lead to higher rates of utilization and EBM guided use of antibiotics and chest x-ray's in primary care.
Trial Registration Identifier NCT01386047
PMCID: PMC3184082  PMID: 21929769
10.  The added value of user involvement during the development of a feedback system regarding physical functioning for community-dwelling elderly people 
The number of frail elderly people is increasing. Unfortunately, the number of caregivers is not increasing at the same pace, which affects older people, caregivers and healthcare systems. Because of these developments, self-management is becoming more important in healthcare. To support community-dwelling elderly people in their self-management, a system was developed that monitors their physical functioning. This system provides feedback to elderly people and their caregivers regarding physical indicators of frailty. The feedback is provided to elderly people via the screen of a mobile phone. It is important that elderly people understand the content of the feedback and are able to use the mobile phone properly. If not, it is unlikely that the system can support self-management. Many interactive health technologies that have been developed do not fulfil their promises. An important reason for this is that human and other non-technology issues are not sufficiently taken into consideration during the development process.
To collaborate with elderly people during the development and evaluation of a feedback system for community-dwelling elderly people regarding physical functioning.
An iterative user-centered design that consists of five phases was used to develop and evaluate the feedback system. These five phases were: 1) Selection of users, 2) Analysis of users and their context, 3) Identification of user needs, 4) Development of a prototype, and 5) Evaluation of the prototype. Three representatives of a target group panel for elderly people were selected in phase 1. They shared their needs and preferences during three expert group meetings that took place during the development process. This resulted in the development of a prototype which was first evaluated in a heuristic evaluation. Once adjustments were made, 11 elderly people evaluated the adjusted prototype using a think aloud procedure. They rated the usability and acceptability of the developed interface on a scale from 1 till 7 using an adapted version of the Post-Study System Usability Questionnaire (PSSUQ).
A feedback system was developed that provides feedback regarding physical indicators of frailty via a touch screen mobile phone. The interface uses colours, smiley’s, and spoken/written messages to provide feedback that is easy to understand. The heuristic evaluation revealed that there were some problems with consistency and the use of user language. The think aloud evaluation showed that the 11 elderly people were able to navigate through the interface without much difficulty despite some small problems related to the lay-out of the interface. The mean score on an adapted version of the PSSUQ was 5.90 (SD 1.09) which indicates high user satisfaction and good usability.
The involvement of end-users significantly influenced the lay-out of the interface that was developed. This resulted in an interface that was accepted by the target group. Evaluation of the prototype revealed that the usability of the interface was good. The feedback system will only succeed in supporting self-management when elderly people are able to use the interface and understand the feedback. The input of elderly people during the development process contributed to this.
PMCID: PMC3571197
user-centered design; elderly people; self management; physical functioning, usability
11.  User-centered design and the development of patient decision aids: protocol for a systematic review 
Systematic Reviews  2015;4(1):11.
Providing patient-centered care requires that patients partner in their personal health-care decisions to the full extent desired. Patient decision aids facilitate processes of shared decision-making between patients and their clinicians by presenting relevant scientific information in balanced, understandable ways, helping clarify patients’ goals, and guiding decision-making processes. Although international standards stipulate that patients and clinicians should be involved in decision aid development, little is known about how such involvement currently occurs, let alone best practices. This systematic review consisting of three interlinked subreviews seeks to describe current practices of user involvement in the development of patient decision aids, compare these to practices of user-centered design, and identify promising strategies.
A research team that includes patient and clinician representatives, decision aid developers, and systematic review method experts will guide this review according to the Cochrane Handbook and PRISMA reporting guidelines. A medical librarian will hand search key references and use a peer-reviewed search strategy to search MEDLINE, EMBASE, PubMed, Web of Science, the Cochrane Library, the ACM library, IEEE Xplore, and Google Scholar. We will identify articles across all languages and years describing the development or evaluation of a patient decision aid, or the application of user-centered design or human-centered design to tools intended for patient use. Two independent reviewers will assess article eligibility and extract data into a matrix using a structured pilot-tested form based on a conceptual framework of user-centered design. We will synthesize evidence to describe how research teams have included users in their development process and compare these practices to user-centered design methods. If data permit, we will develop a measure of the user-centeredness of development processes and identify practices that are likely to be optimal.
This systematic review will provide evidence of current practices to inform approaches for involving patients and other stakeholders in the development of patient decision aids. We anticipate that the results will help move towards the establishment of best practices for the development of patient-centered tools and, in turn, help improve the experiences of people who face difficult health decisions.
Systematic review registration
PROSPERO CRD42014013241
Electronic supplementary material
The online version of this article (doi:10.1186/2046-4053-4-11) contains supplementary material, which is available to authorized users.
PMCID: PMC4328638  PMID: 25623074
Patient decision aids; Decision support; Shared decision-making; Patient education; Counseling; User-centered design; Human-centered design; Patient partnership; Stakeholder engagement; Implementation; Knowledge translation; Patient-centered care
12.  Evaluation of an integrated graphical display to promote acute change detection in ICU patients 
The purpose of this study was to evaluate ICU nurses’ ability to detect patient change using an integrated graphical information display (IGID) versus a conventional tabular ICU patient information display (i.e. electronic chart).
Using participants from two different sites, we conducted a repeated measures simulator-based experiment to assess ICU nurses’ ability to detect abnormal patient variables using a novel IGID versus a conventional tabular information display. Patient scenarios and display presentations were fully counterbalanced.
We measured percent correct detection of abnormal patient variables, nurses’ perceived workload (NASA-TLX), and display usability ratings.
32 ICU nurses (87% female, median age of 29 years, and median ICU experience of 2.5 years) using the IGID detected more abnormal variables compared to the tabular display [F (1,119)=13.0, p < 0.05]. There was a significant main effect of site [F (1, 119)=14.2], with development site participants doing better. There were no significant differences in nurses’ perceived workload. The IGID display was rated as more usable than the conventional display, [F (1, 60)=31.7].
Overall, nurses reported more important physiological information with the novel IGID than tabular display. Moreover, the finding of site differences may reflect local influences in work practice and involvement in iterative display design methodology. Information displays developed using user-centered design should accommodate the full diversity of the intended user population across use sites.
PMCID: PMC3414670  PMID: 22534099
medical informatics; human engineering; man-machine systems; intensive care
13.  User-centred development of a monitoring and feedback tool to stimulate physical activity 
Although physical activity has many positive effects on the quality of life and prognosis, it is very difficult for people with COPD or diabetes type 2 (DM2) to be sufficiently active. Unfortunately, the long-term adherence to several care interventions that try to enhance the physical activity level of these people is generally low. Self-monitoring and feedback integrated in technology might help promoting and sustaining an active lifestyle, especially when it is part of health care services. As a prerequisite, for useful technology and a successful intervention that meets the requirements and preferences of the end users, it is important to involve the end users in the design process at an early stage.
In this contribution we report how users were involved in the design of an innovative monitoring and feedback tool to support self-management of an active lifestyle for COPD and DM2 patients. This tool will be implemented in primary care in a Self-management Support Program (SSP). Users were involved in particular to elicit:
Which feedback they need to optimally support them in their self-management.
How this feedback should be presented.
How the tool could be designed in order to be attractive, easy to use and suitable to wear on a daily basis.
An iterative user-centred approach was used to design both the tool and the SSP. Patients, care professionals and technicians were actively involved in the development process. Two patient representatives were added to the research team to help outlining the research strategy. A multidisciplinary team of technicians and computer engineers was established to help deciding on the development of the tool. Sixteen care professionals, 7 COPD and 8 DM2 patients were involved in interviews and focus groups to elicit the functional requirements of the tool.
The conceptual idea of the tool is adapted to the requirements and preferences of the end users. The user-centred design process resulted in a self-management tool that consists of three parts: an accelerometer that monitors physical activity, a smartphone that shows feedback based on this activity and tailored to personal goals, and a server that stores the data and sends the feedback messages to the patients. This server also provides a summary to practice nurses, to prepare them for consultations, telephone calls or email conversations with patients.
The involvement of end users in the development of the tool has led to new insights that increase its acceptability and usability. The tool will soon be validated and tested in pilot studies. The effects of the Self-management Support Program with the tool, embedded in primary care, will be measured in an RCT.
PMCID: PMC3571178
self management; physical activity; persuasive technology; primary care
14.  Integrating Patient-Reported Outcomes into Spine Surgical Care through Visual Dashboards: Lessons Learned from Human-Centered Design 
eGEMs  2015;3(2):1133.
The collection of patient-reported outcomes (PROs) draws attention to issues of importance to patients—physical function and quality of life. The integration of PRO data into clinical decisions and discussions with patients requires thoughtful design of user-friendly interfaces that consider user experience and present data in personalized ways to enhance patient care. Whereas most prior work on PROs focuses on capturing data from patients, little research details how to design effective user interfaces that facilitate use of this data in clinical practice. We share lessons learned from engaging health care professionals to inform design of visual dashboards, an emerging type of health information technology (HIT).
We employed human-centered design (HCD) methods to create visual displays of PROs to support patient care and quality improvement. HCD aims to optimize the design of interactive systems through iterative input from representative users who are likely to use the system in the future. Through three major steps, we engaged health care professionals in targeted, iterative design activities to inform the development of a PRO Dashboard that visually displays patient-reported pain and disability outcomes following spine surgery.
Design activities to engage health care administrators, providers, and staff guided our work from design concept to specifications for dashboard implementation. Stakeholder feedback from these health care professionals shaped user interface design features, including predefined overviews that illustrate at-a-glance trends and quarterly snapshots, granular data filters that enable users to dive into detailed PRO analytics, and user-defined views to share and reuse. Feedback also revealed important considerations for quality indicators and privacy-preserving sharing and use of PROs.
Our work illustrates a range of engagement methods guided by human-centered principles and design recommendations for optimizing PRO Dashboards for patient care and quality improvement. Engaging health care professionals as stakeholders is a critical step toward the design of user-friendly HIT that is accepted, usable, and has the potential to enhance quality of care and patient outcomes.
PMCID: PMC4431498  PMID: 25988187
Human-centered design; Human Computer Interaction (HCI); Engagement; Visualization; Health Information Technology; Data Use and Quality; Information display; Formative evaluation; Patient-reported outcomes; Learning Health System; Patient-Centered Outcomes Research (PCOR)
15.  Adoption of Clinical Decision Support in Multimorbidity: A Systematic Review 
JMIR Medical Informatics  2015;3(1):e4.
Patients with multiple conditions have complex needs and are increasing in number as populations age. This multimorbidity is one of the greatest challenges facing health care. Having more than 1 condition generates (1) interactions between pathologies, (2) duplication of tests, (3) difficulties in adhering to often conflicting clinical practice guidelines, (4) obstacles in the continuity of care, (5) confusing self-management information, and (6) medication errors. In this context, clinical decision support (CDS) systems need to be able to handle realistic complexity and minimize iatrogenic risks.
The aim of this review was to identify to what extent CDS is adopted in multimorbidity.
This review followed PRISMA guidance and adopted a multidisciplinary approach. Scopus and PubMed searches were performed by combining terms from 3 different thesauri containing synonyms for (1) multimorbidity and comorbidity, (2) polypharmacy, and (3) CDS. The relevant articles were identified by examining the titles and abstracts. The full text of selected/relevant articles was analyzed in-depth. For articles appropriate for this review, data were collected on clinical tasks, diseases, decision maker, methods, data input context, user interface considerations, and evaluation of effectiveness.
A total of 50 articles were selected for the full in-depth analysis and 20 studies were included in the final review. Medication (n=10) and clinical guidance (n=8) were the predominant clinical tasks. Four studies focused on merging concurrent clinical practice guidelines. A total of 17 articles reported their CDS systems were knowledge-based. Most articles reviewed considered patients’ clinical records (n=19), clinical practice guidelines (n=12), and clinicians’ knowledge (n=10) as contextual input data. The most frequent diseases mentioned were cardiovascular (n=9) and diabetes mellitus (n=5). In all, 12 articles mentioned generalist doctor(s) as the decision maker(s). For articles reviewed, there were no studies referring to the active involvement of the patient in the decision-making process or to patient self-management. None of the articles reviewed adopted mobile technologies. There were no rigorous evaluations of usability or effectiveness of the CDS systems reported.
This review shows that multimorbidity is underinvestigated in the informatics of supporting clinical decisions. CDS interventions that systematize clinical practice guidelines without considering the interactions of different conditions and care processes may lead to unhelpful or harmful clinical actions. To improve patient safety in multimorbidity, there is a need for more evidence about how both conditions and care processes interact. The data needed to build this evidence base exist in many electronic health record systems and are underused.
PMCID: PMC4318680  PMID: 25785897
decision support systems, management; systematic review; multiple chronic diseases; multiple pathologies; multiple medications
16.  BioCreative III interactive task: an overview 
BMC Bioinformatics  2011;12(Suppl 8):S4.
The BioCreative challenge evaluation is a community-wide effort for evaluating text mining and information extraction systems applied to the biological domain. The biocurator community, as an active user of biomedical literature, provides a diverse and engaged end user group for text mining tools. Earlier BioCreative challenges involved many text mining teams in developing basic capabilities relevant to biological curation, but they did not address the issues of system usage, insertion into the workflow and adoption by curators. Thus in BioCreative III (BC-III), the InterActive Task (IAT) was introduced to address the utility and usability of text mining tools for real-life biocuration tasks. To support the aims of the IAT in BC-III, involvement of both developers and end users was solicited, and the development of a user interface to address the tasks interactively was requested.
A User Advisory Group (UAG) actively participated in the IAT design and assessment. The task focused on gene normalization (identifying gene mentions in the article and linking these genes to standard database identifiers), gene ranking based on the overall importance of each gene mentioned in the article, and gene-oriented document retrieval (identifying full text papers relevant to a selected gene). Six systems participated and all processed and displayed the same set of articles. The articles were selected based on content known to be problematic for curation, such as ambiguity of gene names, coverage of multiple genes and species, or introduction of a new gene name. Members of the UAG curated three articles for training and assessment purposes, and each member was assigned a system to review. A questionnaire related to the interface usability and task performance (as measured by precision and recall) was answered after systems were used to curate articles. Although the limited number of articles analyzed and users involved in the IAT experiment precluded rigorous quantitative analysis of the results, a qualitative analysis provided valuable insight into some of the problems encountered by users when using the systems. The overall assessment indicates that the system usability features appealed to most users, but the system performance was suboptimal (mainly due to low accuracy in gene normalization). Some of the issues included failure of species identification and gene name ambiguity in the gene normalization task leading to an extensive list of gene identifiers to review, which, in some cases, did not contain the relevant genes. The document retrieval suffered from the same shortfalls. The UAG favored achieving high performance (measured by precision and recall), but strongly recommended the addition of features that facilitate the identification of correct gene and its identifier, such as contextual information to assist in disambiguation.
The IAT was an informative exercise that advanced the dialog between curators and developers and increased the appreciation of challenges faced by each group. A major conclusion was that the intended users should be actively involved in every phase of software development, and this will be strongly encouraged in future tasks. The IAT Task provides the first steps toward the definition of metrics and functional requirements that are necessary for designing a formal evaluation of interactive curation systems in the BioCreative IV challenge.
PMCID: PMC3269939  PMID: 22151968
17.  “Quality of prenatal and maternal care: bridging the know-do gap” (QUALMAT study): an electronic clinical decision support system for rural Sub-Saharan Africa 
Despite strong efforts to improve maternal care, its quality remains deficient in many countries of Sub-Saharan Africa as persistently high maternal mortality rates testify. The QUALMAT study seeks to improve the performance and motivation of rural health workers and ultimately quality of primary maternal health care services in three African countries Burkina Faso, Ghana, and Tanzania. One major intervention is the introduction of a computerized Clinical Decision Support System (CDSS) for rural primary health care centers to be used by health care workers of different educational levels.
A stand-alone, java-based software, able to run on any standard hardware, was developed based on assessment of the health care situation in the involved countries. The software scope was defined and the final software was programmed under consideration of test experiences. Knowledge for the decision support derived from the World Health Organization (WHO) guideline “Pregnancy, Childbirth, Postpartum and Newborn Care; A Guide for Essential Practice”.
The QUALMAT CDSS provides computerized guidance and clinical decision support for antenatal care, and care during delivery and up to 24 hours post delivery. The decision support is based on WHO guidelines and designed using three principles: (1) Guidance through routine actions in maternal and perinatal care, (2) integration of clinical data to detect situations of concern by algorithms, and (3) electronic tracking of peri- and postnatal activities. In addition, the tool facilitates patient management and is a source of training material. The implementation of the software, which is embedded in a set of interventions comprising the QUALMAT study, is subject to various research projects assessing and quantifying the impact of the CDSS on quality of care, the motivation of health care staff (users) and its health economic aspects. The software will also be assessed for its usability and acceptance, as well as for its influence on workflows in the rural setting of primary health care in the three countries involved.
The development and implementation of a CDSS in rural primary health care centres presents challenges, which may be overcome with careful planning and involvement of future users at an early stage. A tailored software with stable functionality should offer perspectives to improve maternal care in resource-poor settings.
Trial registration
PMCID: PMC3637082  PMID: 23574764
Guideline adherence; Clinical decision support systems; Medical informatics; Maternal health services; Pregnancy; Prenatal care; Perinatal care; Millennium development goal; Motivation by information technology; Rural maternal healthcare
18.  User experiences of evidence-based online resources for health professionals: User testing of The Cochrane Library 
Evidence-based decision making relies on easy access to trustworthy research results. The Cochrane Library is a key source of evidence about the effect of interventions and aims to "promote the accessibility of systematic reviews to anyone wanting to make a decision about health care". We explored how health professionals found, used and experienced The Library, looking at facets of user experience including findability, usability, usefulness, credibility, desirability and value.
We carried out 32 one-hour usability tests on participants from Norway and the UK. Participants both browsed freely and attempted to perform individually tailored tasks while "thinking aloud". Sessions were recorded and viewed in real time by researchers. Transcriptions and videos were reviewed by one researcher and one designer. Findings reported here reflect issues receiving a high degree of saturation and that we judge to be critical to the user experience of evidence-based web sites, based on principles for usability heuristics, web guidelines and evidence-based practice.
Participants had much difficulty locating both the site and its contents. Non-native English speakers were at an extra disadvantage when retrieving relevant documents despite high levels of English-language skills. Many participants displayed feelings of ineptitude, alienation and frustration. Some made serious mistakes in correctly distinguishing between different information types, for instance reviews, review protocols, and individual studies. Although most expressed a high regard for the site's credibility, some later displayed a mistrust of the independence of the information. Others were overconfident, thinking everything on The Cochrane Library site shared the same level of quality approval.
Paradoxically, The Cochrane Library, established to support easy access to research evidence, has its own problems of accessibility. Health professionals' experiences of this and other evidence-based online resources can be improved by applying existing principles for web usability, prioritizing the development of simple search functionality, emitting "researcher" jargon, consistent marking of site ownership, and clear signposting of different document types and different content quality.
PMCID: PMC2529276  PMID: 18662382
19.  The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview 
PLoS Medicine  2011;8(1):e1000387.
Aziz Sheikh and colleagues report the findings of their systematic overview that assessed the impact of eHealth solutions on the quality and safety of health care.
There is considerable international interest in exploiting the potential of digital solutions to enhance the quality and safety of health care. Implementations of transformative eHealth technologies are underway globally, often at very considerable cost. In order to assess the impact of eHealth solutions on the quality and safety of health care, and to inform policy decisions on eHealth deployments, we undertook a systematic review of systematic reviews assessing the effectiveness and consequences of various eHealth technologies on the quality and safety of care.
Methods and Findings
We developed novel search strategies, conceptual maps of health care quality, safety, and eHealth interventions, and then systematically identified, scrutinised, and synthesised the systematic review literature. Major biomedical databases were searched to identify systematic reviews published between 1997 and 2010. Related theoretical, methodological, and technical material was also reviewed. We identified 53 systematic reviews that focused on assessing the impact of eHealth interventions on the quality and/or safety of health care and 55 supplementary systematic reviews providing relevant supportive information. This systematic review literature was found to be generally of substandard quality with regards to methodology, reporting, and utility. We thematically categorised eHealth technologies into three main areas: (1) storing, managing, and transmission of data; (2) clinical decision support; and (3) facilitating care from a distance. We found that despite support from policymakers, there was relatively little empirical evidence to substantiate many of the claims made in relation to these technologies. Whether the success of those relatively few solutions identified to improve quality and safety would continue if these were deployed beyond the contexts in which they were originally developed, has yet to be established. Importantly, best practice guidelines in effective development and deployment strategies are lacking.
There is a large gap between the postulated and empirically demonstrated benefits of eHealth technologies. In addition, there is a lack of robust research on the risks of implementing these technologies and their cost-effectiveness has yet to be demonstrated, despite being frequently promoted by policymakers and “techno-enthusiasts” as if this was a given. In the light of the paucity of evidence in relation to improvements in patient outcomes, as well as the lack of evidence on their cost-effectiveness, it is vital that future eHealth technologies are evaluated against a comprehensive set of measures, ideally throughout all stages of the technology's life cycle. Such evaluation should be characterised by careful attention to socio-technical factors to maximise the likelihood of successful implementation and adoption.
Please see later in the article for the Editors' Summary
Editors' Summary
There is considerable international interest in exploiting the potential of digital health care solutions, often referred to as eHealth—the use of information and communication technologies—to enhance the quality and safety of health care. Often accompanied by large costs, any large-scale expenditure on eHealth—such as electronic health records, picture archiving and communication systems, ePrescribing, associated computerized provider order entry systems, and computerized decision support systems—has tended to be justified on the grounds that these are efficient and cost-effective means for improving health care. In 2005, the World Health Assembly passed an eHealth resolution (WHA 58.28) that acknowledged, “eHealth is the cost-effective and secure use of information and communications technologies in support of health and health-related fields, including health-care services, health surveillance, health literature, and health education, knowledge and research,” and urged member states to develop and implement eHealth technologies. Since then, implementing eHealth technologies has become a main priority for many countries. For example, England has invested at least £12.8 billion in a National Programme for Information Technology for the National Health Service, and the Obama administration in the United States has committed to a US$38 billion eHealth investment in health care.
Why Was This Study Done?
Despite the wide endorsement of and support for eHealth, the scientific basis of its benefits—which are repeatedly made and often uncritically accepted—remains to be firmly established. A robust evidence-based perspective on the advantages on eHealth could help to suggest priority areas that have the greatest potential for benefit to patients and also to inform international eHealth deliberations on costs. Therefore, in order to better inform the international community, the authors systematically reviewed the published systematic review literature on eHealth technologies and evaluated the impact of these technologies on the quality and safety of health care delivery.
What Did the Researchers Do and Find?
The researchers divided eHealth technologies into three main categories: (1) storing, managing, and transmission of data; (2) clinical decision support; and (3) facilitating care from a distance. Then, implementing methods based on those developed by the Cochrane Collaboration and the NHS Service Delivery and Organisation Programme, the researchers used detailed search strategies and maps of health care quality, safety, and eHealth interventions to identify relevant systematic reviews (and related theoretical, methodological, and technical material) published between 1997 and 2010. Using these techniques, the researchers retrieved a total of 46,349 references from which they identified 108 reviews. The 53 reviews that the researchers finally selected (and critically reviewed) provided the main evidence base for assessing the impact of eHealth technologies in the three categories selected.
In their systematic review of systematic reviews, the researchers included electronic health records and picture archiving communications systems in their evaluation of category 1, computerized provider (or physician) order entry and e-prescribing in category 2, and all clinical information systems that, when used in the context of eHealth technologies, integrate clinical and demographic patient information to support clinician decision making in category 3.
The researchers found that many of the clinical claims made about the most commonly used eHealth technologies were not substantiated by empirical evidence. The evidence base in support of eHealth technologies was weak and inconsistent and importantly, there was insubstantial evidence to support the cost-effectiveness of these technologies. For example, the researchers only found limited evidence that some of the many presumed benefits could be realized; importantly, they also found some evidence that introducing these new technologies may on occasions also generate new risks such as prescribers becoming over-reliant on clinical decision support for e-prescribing, or overestimate its functionality, resulting in decreased practitioner performance.
What Do These Findings Mean?
The researchers found that despite the wide support for eHealth technologies and the frequently made claims by policy makers when constructing business cases to raise funds for large-scale eHealth projects, there is as yet relatively little empirical evidence to substantiate many of the claims made about eHealth technologies. In addition, even for the eHealth technology tools that have proven to be successful, there is little evidence to show that such tools would continue to be successful beyond the contexts in which they were originally developed. Therefore, in light of the lack of evidence in relation to improvements in patient outcomes, as well as the lack of evidence on their cost-effectiveness, the authors say that future eHealth technologies should be evaluated against a comprehensive set of measures, ideally throughout all stages of the technology's life cycle, and include socio-technical factors to maximize the likelihood of successful implementation and adoption in a given context. Furthermore, it is equally important that eHealth projects that have already been commissioned are subject to rigorous, multidisciplinary, and independent evaluation.
Additional Information
Please access these websites via the online version of this summary at
The authors' broader study is: Car J, Black A, Anandan C, Cresswell K, Pagliari C, McKinstry B, et al. (2008) The Impact of eHealth on the Quality and Safety of Healthcare. Available at:
More information is available on the World Health Assembly eHealth resolution
The World Health Organization provides information at the Global Observatory on eHealth, as well as a global insight into eHealth developments
The European Commission provides Information on eHealth in Europe and some examples of good eHealth practice
More information is provided on NHS Connecting for Health
PMCID: PMC3022523  PMID: 21267058
20.  A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies 
Many eHealth technologies are not successful in realizing sustainable innovations in health care practices. One of the reasons for this is that the current development of eHealth technology often disregards the interdependencies between technology, human characteristics, and the socioeconomic environment, resulting in technology that has a low impact in health care practices. To overcome the hurdles with eHealth design and implementation, a new, holistic approach to the development of eHealth technologies is needed, one that takes into account the complexity of health care and the rituals and habits of patients and other stakeholders.
The aim of this viewpoint paper is to improve the uptake and impact of eHealth technologies by advocating a holistic approach toward their development and eventual integration in the health sector.
To identify the potential and limitations of current eHealth frameworks (1999–2009), we carried out a literature search in the following electronic databases: PubMed, ScienceDirect, Web of Knowledge, PiCarta, and Google Scholar. Of the 60 papers that were identified, 44 were selected for full review. We excluded those papers that did not describe hands-on guidelines or quality criteria for the design, implementation, and evaluation of eHealth technologies (28 papers). From the results retrieved, we identified 16 eHealth frameworks that matched the inclusion criteria. The outcomes were used to posit strategies and principles for a holistic approach toward the development of eHealth technologies; these principles underpin our holistic eHealth framework.
A total of 16 frameworks qualified for a final analysis, based on their theoretical backgrounds and visions on eHealth, and the strategies and conditions for the research and development of eHealth technologies. Despite their potential, the relationship between the visions on eHealth, proposed strategies, and research methods is obscure, perhaps due to a rather conceptual approach that focuses on the rationale behind the frameworks rather than on practical guidelines. In addition, the Web 2.0 technologies that call for a more stakeholder-driven approach are beyond the scope of current frameworks. To overcome these limitations, we composed a holistic framework based on a participatory development approach, persuasive design techniques, and business modeling.
To demonstrate the impact of eHealth technologies more effectively, a fresh way of thinking is required about how technology can be used to innovate health care. It also requires new concepts and instruments to develop and implement technologies in practice. The proposed framework serves as an evidence-based roadmap.
PMCID: PMC3278097  PMID: 22155738
eHealth; design; participation; implementation; evaluation; multidisciplinary approach; Health 2.0; Wiki; e-collaboration
21.  Enhancing the Informatics Evaluation Toolkit with Remote Usability Testing 
Developing functional clinical informatics products that are also usable remains a challenge. Despite evidence that usability testing should be incorporated into the lifecycle of health information technologies, rarely does this occur. Challenges include poor standards, a lack of knowledge around usability practices, and the expense involved in rigorous testing with a large number of users. Remote usability testing may be a solution for many of these challenges. Remotely testing an application can greatly enhance the number of users who can iteratively interact with a product, and it can reduce the costs associated with usability testing. A case study presents the experiences with remote usability testing when evaluating a Web site designed for health informatics knowledge dissemination. The lessons can inform others seeking to enhance their evaluation toolkits for clinical informatics products.
PMCID: PMC2815364  PMID: 20351839
22.  User perspectives on the usability of a regional health information exchange 
We assessed the usability of a health information exchange (HIE) in a densely populated metropolitan region. This grant-funded HIE had been deployed rapidly to address the imminent needs of the patient population and the need to draw wider participation from regional entities.
We conducted a cross-sectional survey of individuals given access to the HIE at participating organizations and examined some of the usability and usage factors related to the technology acceptance model.
We probed user perceptions using the Questionnaire for User Interaction Satisfaction, an author-generated Trust scale, and user characteristic questions (eg, age, weekly system usage time).
Overall, users viewed the system favorably (ratings for all usability items were greater than neutral (one-sample Wilcoxon test, p<0.0014, Bonferroni-corrected for 35 tests). System usage was regressed on usability, trust, and demographic and user characteristic factors. Three usability factors were positively predictive of system usage: overall reactions (p<0 0.01), learning (p<0.05), and system functionality (p<0.01). Although trust is an important component in collaborative relationships, we did not find that user trust of other participating healthcare entities was significantly predictive of usage. An analysis of respondents' comments revealed ways to improve the HIE.
We used a rapid deployment model to develop an HIE and found that perceptions of system usability were positive. We also found that system usage was predicted well by some aspects of usability. Results from this study suggest that a rapid development approach may serve as a viable model for developing usable HIEs serving communities with limited resources.
PMCID: PMC3168324  PMID: 21622933
Information dissemination; medical records system; regional health planning; usability; usage; biomedical informatics; evaluation; Qualitative/ethnographic field study; Improving the education and skills training of health professionals; System implementation and management issues; Social/organizational study; Collaborative technologies; Methods for integration of information from disparate sources; Demonstrating return on IT investment; distributed systems; agents; Software engineering: architecture; Supporting practice at a distance (telehealth); Data exchange; communication; and integration across care settings (inter- and intra-enterprise); Visualization of data and knowledge; policy; Legal; historical; ethical study methods; clinical informatics; biomedical informatics; pediatrics; e-prescribing; human factors
23.  Mobile Applications for Diabetes Self-Management: Status and Potential 
Advancements in smartphone technology coupled with the proliferation of data connectivity has resulted in increased interest and unprecedented growth in mobile applications for diabetes self-management. The objective of this article is to determine, in a systematic review, whether diabetes applications have been helping patients with type 1 or type 2 diabetes self-manage their condition and to identify issues necessary for large-scale adoption of such interventions.
The review covers commercial applications available on the Apple App Store (as a representative of commercially available applications) and articles published in relevant databases covering a period from January 1995 to August 2012. The review included all applications supporting any diabetes self-management task where the patient is the primary actor.
Available applications support self-management tasks such as physical exercise, insulin dosage or medication, blood glucose testing, and diet. Other support tasks considered include decision support, notification/alert, tagging of input data, and integration with social media. The review points to the potential for mobile applications to have a positive impact on diabetes self-management. Analysis indicates that application usage is associated with improved attitudes favorable to diabetes self-management. Limitations of the applications include lack of personalized feedback; usability issues, particularly the ease of data entry; and integration with patients and electronic health records.
Research into the adoption and use of user-centered and sociotechnical design principles is needed to improve usability, perceived usefulness, and, ultimately, adoption of the technology. Proliferation and efficacy of interventions involving mobile applications will benefit from a holistic approach that takes into account patients’ expectations and providers’ needs.
PMCID: PMC3692239  PMID: 23439183
glucose monitoring; mobile applications; self-care; telemedicine
24.  The Effectiveness of Mobile-Health Technologies to Improve Health Care Service Delivery Processes: A Systematic Review and Meta-Analysis 
PLoS Medicine  2013;10(1):e1001363.
Caroline Free and colleagues systematically review controlled trials of mobile technology interventions to improve health care delivery processes and show that current interventions give only modest benefits and that high-quality trials measuring clinical outcomes are needed.
Mobile health interventions could have beneficial effects on health care delivery processes. We aimed to conduct a systematic review of controlled trials of mobile technology interventions to improve health care delivery processes.
Methods and Findings
We searched for all controlled trials of mobile technology based health interventions using MEDLINE, EMBASE, PsycINFO, Global Health, Web of Science, Cochrane Library, UK NHS HTA (Jan 1990–Sept 2010). Two authors independently extracted data on allocation concealment, allocation sequence, blinding, completeness of follow-up, and measures of effect. We calculated effect estimates and we used random effects meta-analysis to give pooled estimates.
We identified 42 trials. None of the trials had low risk of bias. Seven trials of health care provider support reported 25 outcomes regarding appropriate disease management, of which 11 showed statistically significant benefits. One trial reported a statistically significant improvement in nurse/surgeon communication using mobile phones. Two trials reported statistically significant reductions in correct diagnoses using mobile technology photos compared to gold standard. The pooled effect on appointment attendance using text message (short message service or SMS) reminders versus no reminder was increased, with a relative risk (RR) of 1.06 (95% CI 1.05–1.07, I2 = 6%). The pooled effects on the number of cancelled appointments was not significantly increased RR 1.08 (95% CI 0.89–1.30). There was no difference in attendance using SMS reminders versus other reminders (RR 0.98, 95% CI 0.94–1.02, respectively). To address the limitation of the older search, we also reviewed more recent literature.
The results for health care provider support interventions on diagnosis and management outcomes are generally consistent with modest benefits. Trials using mobile technology-based photos reported reductions in correct diagnoses when compared to the gold standard. SMS appointment reminders have modest benefits and may be appropriate for implementation. High quality trials measuring clinical outcomes are needed.
Please see later in the article for the Editors' Summary
Editors’ Summary
Over the past few decades, computing and communication technologies have changed dramatically. Bulky, slow computers have been replaced by portable devices that can complete increasingly complex tasks in less and less time. Similarly, landlines have been replaced by mobile phones and other mobile communication technologies that can connect people anytime and anywhere, and that can transmit text messages (short message service; SMS), photographs, and data at the touch of a button. These advances have led to the development of mobile-health (mHealth)—the use of mobile computing and communication technologies in health care and public health. mHealth has many applications. It can be used to facilitate data collection and to encourage health-care consumers to adopt healthy lifestyles or to self-manage chronic conditions. It can also be used to improve health-care service delivery processes by targeting health-care providers or communication between these providers and their patients. So, for example, mobile technologies can be used to provide clinical management support in settings where there are no specialist clinicians, and they can be used to send patients test results and timely reminders of appointments.
Why Was This Study Done?
Many experts believe that mHealth interventions could greatly improve health-care delivery processes, particularly in resource-poor settings. The results of several controlled trials (studies that compare the outcomes of people who do or do not receive an intervention) of mHealth interventions designed to improve health-care delivery processes have been published. However, these data have not been comprehensively reviewed, and the effectiveness of this type of mHealth intervention has not been quantified. Here, the researchers rectify this situation by undertaking a systematic review and meta-analysis of controlled trials of mobile technology-based interventions designed to improve health-care service delivery processes. A systematic review is a study that uses predefined criteria to identify all the research on a given topic; a meta-analysis is a statistical approach that is used to pool the results of several independent studies.
What Did the Researchers Do and Find?
The researchers identified 42 controlled trials that investigated mobile technology-based interventions designed to improve health-care service delivery processes. None of the trials were of high quality—many had methodological problems likely to affect the accuracy of their findings—and nearly all were undertaken in high-income countries. Thirty-two of the trials tested interventions directed at health-care providers. Of these trials, seven investigated interventions providing health-care provider education, 18 investigated interventions supporting clinical diagnosis and treatment, and seven investigated interventions to facilitate communication between health-care providers. Several of the trials reported that the tested intervention led to statistically significant improvements (improvements unlikely to have happened by chance) in outcomes related to disease management. However, two trials that used mobile phones to transmit photos to off-site clinicians for diagnosis reported significant reductions in correct diagnoses compared to diagnosis by an on-site specialist. Ten of the 42 trials investigated interventions targeting communication between health-care providers and patients. Eight of these trials investigated SMS-based appointment reminders. Meta-analyses of the results of these trials indicated that using SMS appointment reminders significantly but modestly increased patient attendance compared to no reminders. However, SMS reminders were no more effective than postal or phone call reminders, and texting reminders to patients who persistently missed appointments did not significantly change the number of cancelled appointments.
What Do These Findings Mean?
These findings indicate that some mHealth interventions designed to improve health-care service delivery processes are modestly effective, but they also highlight the need for more trials of these interventions. Specifically, these findings show that although some interventions designed to provide support for health-care providers modestly improved some aspects of clinical diagnosis and management, other interventions had deleterious effects—most notably, the use of mobile technology–based photos for diagnosis. In terms of mHealth interventions targeting communication between health-care providers and patients, the finding that SMS appointment reminders have modest benefits suggests that implementation of this intervention should be considered, at least in high-income settings. However, the researchers stress that more trials are needed to robustly establish the ability of mobile technology-based interventions to improve health-care delivery processes. These trials need to be of high quality, they should be undertaken in resource-limited settings as well as in high-income countries, and, ideally, they should consider interventions that combine mHealth and conventional approaches.
Additional Information
Please access these Web sites via the online version of this summary at
A related PLOS Medicine Research Article by Free et al. investigates the effectiveness of mHealth technology-based health behavior change and disease management interventions for health-care consumers
Wikipedia has a page on mHealth (note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages)
mHealth: New horizons for health through mobile technologies is a global survey of mHealth prepared by the World Health Organization’s Global Observatory for eHealth (eHealth is health-care practice supported by electronic processes and communication)
The mHealth in Low-Resource Settings website, which is maintained by the Netherlands Royal Tropical Institute, provides information on the current use, potential, and limitations of mHealth in low-resource settings
The US National Institutes of Health Fogarty International Center provides links to resources and information about mHealth
PMCID: PMC3566926  PMID: 23458994
25.  Supporting Cancer Patients in Illness Management: Usability Evaluation of a Mobile App 
JMIR mHealth and uHealth  2014;2(3):e33.
Mobile phones and tablets currently represent a significant presence in people’s everyday lives. They enable access to different information and services independent of current place and time. Such widespread connectivity offers significant potential in different app areas including health care.
Our goal was to evaluate the usability of the Connect Mobile app. The mobile app enables mobile access to the Connect system, an online system that supports cancer patients in managing health-related issues. Along with symptom management, the system promotes better patient-provider communication, collaboration, and shared decision making. The Connect Mobile app enables access to the Connect system over both mobile phones and tablets.
The study consisted of usability tests of a high fidelity prototype with 7 cancer patients where the objectives were to identify existing design and functionality issues and to provide patients with a real look-and-feel of the mobile system. In addition, we conducted semistructured interviews to obtain participants’ feedback about app usefulness, identify the need for new system features and design requirements, and measure the acceptance of the mobile app and its features within everyday health management.
The study revealed a total of 27 design issues (13 for mobile apps and 14 for tablet apps), which were mapped to source events (ie, errors, requests for help, participants' concurrent feedback, and moderator observation). We also applied usability heuristics to identify violations of usability principles. The majority of violations were related to enabling ease of input, screen readability, and glanceability (15 issues), as well as supporting an appropriate match between systems and the real world (7 issues) and consistent mapping of system functions and interactions (4 issues). Feedback from participants also showed the cancer patients’ requirements for support systems and how these needs are influenced by different context-related factors, such as type of access terminal (eg, desktop computer, tablet, mobile phone) and phases of illness. Based on the observed results, we proposed design and functionality recommendations that can be used for the development of mobile apps for cancer patients to support their health management process.
Understanding and addressing users’ requirements is one of the main prerequisites for developing useful and effective technology-based health interventions. The results of this study outline different user requirements related to the design of the mobile patient support app for cancer patients. The results will be used in the iterative development of the Connect Mobile app and can also inform other developers and researchers in development, integration, and evaluation of mobile health apps and services that support cancer patients in managing their health-related issues.
PMCID: PMC4147703  PMID: 25119490
mobile applications; patients; cell phone; smartphone; symptom assessment; self-care; user-computer interface

Results 1-25 (1312239)