|Home | About | Journals | Submit | Contact Us | Français|
This paper describes the development and evaluation of an innovative application designed to engage children and their parents in weekly asthma self-monitoring and self-management to prompt an early response to deteriorations in chronic asthma control, and to provide their physicians with longitudinal data to assess the effectiveness of asthma therapy and prompt adjustments. The evaluation included 2 iterative usability testing cycles with 6 children with asthma and 2 parents of children with asthma to assess user performance and satisfaction with the application. Several usability problems were identified and changes were made to ensure acceptability of the application and relevance of the content. This novel application is unique compared to existing asthma tools and may shift asthma care from the current reactive, acute care model to a preventive, proactive patient-centered approach where treatment decisions are tailored to patients’ individual patterns of chronic asthma control to prevent acute exacerbations.
Asthma is the most common pediatric chronic illness with a lifetime prevalence of 13.8% in the US. In 2009, an estimated 7.1 million children under age 18 had asthma, 4.1 million had an asthma “attack” during that year, and many others had undiagnosed asthma. Asthma is also associated with a significant impact on a child’s and/or family’s quality of life, and on health care use and costs. Overall, pediatric asthma has an annual economic impact of $20.7 billion in total costs in the US. Despite well-established evidence-based interventions, chronic asthma control among children remains suboptimal. Poor control leads to frequent asthma exacerbations, poor health related quality of life (HRQOL), frequent urgent health care utilization and school absenteeism.
To improve asthma control and reduce the risk of exacerbations, the National Asthma Education and Prevention Program-Expert Panel Report 3 (NAEPP-EPR3) guidelines recommend ongoing monitoring of chronic asthma symptoms and patient’s self-management with timely adjustments of preventive therapy in order to achieve and maintain optimal control. Close monitoring of asthma control increases the patient’s awareness of chronic asthma symptoms and enhances compliance with asthma therapy. Close monitoring also helps the primary care provider (PCP) to identify patients with poor asthma control, assess the effectiveness of their treatment and adjust asthma therapy in a timely fashion. However, close monitoring of patients’ asthma control by clinicians outside clinical encounters is challenging to establish due to lack of effective tools, the fragmented nature of the current healthcare system and the inherent focus on “rescue” care rather than preventive care. Hospitals focus primarily on providing acute treatments and expediting hospital discharge but not addressing chronic care needs that could prevent a subsequent readmission. Similarly, PCPs focus mainly on intermittent (as needed) visits and often lack resources, time, financial incentive, and tools to adequately monitor their patients outside clinical encounters. In addition, children and parents often have difficulty recognizing early signs of asthma deteriorations and do not access their PCP in a timely manner. Furthermore, the complexity of a busy modern lifestyle makes it difficult to engage children and their parents in self-monitoring and self-management without outside support or tools.
Several tools have been validated for assessment of chronic asthma control, but few, other than peak flow, have been advocated and tested for ongoing self-monitoring and self-management, and studies have shown suboptimal results and significant limitations in children.[6, 7] One possible reason is that patient usability was not typically assessed in the development of existing self-monitoring and self-management tools. The objective of this paper is to describe the development and usability of an innovative, patient-centered application, the electronic Asthma Symptom Tracking and Exacerbation Reduction (e-ASTER) application. The e-ASTER application, also called eAsthma Tracker, was developed to engage children and their parents in asthma self-monitoring and self-management in order to prompt early response to deteriorations in chronic asthma control status and to support PCPs with longitudinal data to assess the effectiveness of asthma therapy and prompt adjustments.
This project took place at Primary Children’s Medical Center (PCMC), a freestanding academic children’s hospital in Salt Lake City, Utah affiliated with the Department of Pediatrics at the University of Utah, School of Medicine. PCMC has 289 licensed beds, and is owned and operated by Intermountain Healthcare, an integrated health care delivery system serving patients in Utah and portions of the 5 surrounding states.
The development of the e-ASTER application was guided by information generated from a focus group formed from a sample of children and their parents who used a paper-based version of the tool, which was developed and tested de-novo by the research team for reliability and validity. We assembled an interdisciplinary team including physicians and a nurse specialist with experience in asthma education, informaticists, a web-developer/programmer, and researchers including experts in system and process improvement to develop the initial prototype of the e-ASTER application. The team developed a list of the application requirements including needed functionalities for both children/parents and PCPs. This list was augmented with additional software requirements generated from patients and parents input during the focus group. We used a stepwise (iterative) development process and met weekly with the web-developer/programmer to review interim designs and prioritize tasks to accomplish for the following meeting. The e-ASTER application was developed for use at home by older children (≥10 years of age), with or without assistance from their parents, and by parents of younger children (< 10 years of age). The e-ASTER application comprises 4 main components: 1) a patient portal, 2) online survey questionnaire, 3) report generator system, and 4) decision support and feedback system. Other features are available to facilitate compliance and ease of use.
Participants for usability testing were selected among children 10–18 years of age and parents of children 2–9 years of age who were discharged from PCMC with a primary diagnosis of asthma, and had participated in the evaluation study of a paper-based version of the e-ASTER application. Participants included 6 children with asthma and 2 parents (both mothers) of children with asthma (n=8 participants), all English speakers, and recruited through a personalized phone call. The 6 children who participated in the usability test were accompanied by their parents. Participants received $40 cash for their time and travel expense. The study was approved by the University of Utah Institutional Review Board and the Intermountain Healthcare privacy board.
Participants were first consented and received a brief explanation about the e-ASTER application. Initials or pseudonyms were used during the usability test and in subsequent transcription and analysis to maintain confidentiality. Participants were guided to the home page of the e-ASTER application and were provided a user name, a password, and brief instruction about how to access the application.
We used a qualitative study design involving one-on-one, audio recorded interviews. The evaluation was conducted in a quiet room with computers and Internet access, and included 2 iterative usability-testing cycles to determine user performance and satisfaction with the e-ASTER application. The usability testing was led by a trained observer and moderator (JP) and was supported by a note taker with audiotaped recordings.[12, 13] Participants were asked to navigate through all of the e-ASTER modules including the main screen, instruction materials, profile screen, survey questionnaire, graphical display, and interactive components. Each session lasted about 45 minutes and participants were asked to “think aloud” as they navigated through the application. During the first cycle iteration, the trained observer/moderator asked questions regarding any issues encountered or identified by participants. To facilitate interpretation of “loud thought”, he also asked a number of direct questions at different points during navigation of the application to explore and test participants’ understanding of various aspects of the application (e.g., “What is the application asking you to do at this point?” “What do you think the different colors on the graph mean?”), and to solicit their suggestions for improving the application (e.g., “What other things, if any, would you want to see or be able to do from this page?”). The second cycle iteration was used to identify remaining issues not resolved during the first cycle. User responses and suggestions were used to drive changes to finalize the user interface of the e-ASTER application.
We conducted qualitative data analysis. The notes and the audio-recordings were used to provide a comprehensive record of each interview. We then used open coding with 2 coders (FN and JP) and grouped similar responses into categories and assigned labels capturing specific themes. Any suggested changes in either the composition of the categories or the labels were discussed among study investigators. After agreement was reached in the composition and label of each category, we associated (axial coding) categories to the central phenomena of interest related to value and use of the application, instruction materials, profile page, questionnaire, graph and other aspects of the e-ASTER application. Responses were analyzed descriptively and aligned with specific changes made to the e-ASTER application after both iterations.
Overall participants were satisfied with the e-ASTER application and believed that it can help them better manage their asthma symptoms. They were able to navigate through the application with little assistance. The most important errors included participant confusion regarding how to advance from the tutorial/instruction page to starting the application itself. Participants suggested having either a clearly labeled button like “Start using eAsthma Tracker” or having the tool automatically advance to the start screen once the tutorial ends. Several other minor performance issues were identified during the first and second iterations and changes made to the application to address these concerns are described in Table 2.
The e-ASTER application is an innovative, user-friendly and interactive web application developed to provide a focused approach to engaging children and parents in regular self-monitoring and self-management of chronic asthma symptoms. It uses a modified ACT questionnaire, adapted and validated for weekly assessment of chronic asthma control, in conjunction with a longitudinal graphic display and decision support system to prompt an early response to deteriorations of asthma control status. The e-ASTER application also provides a longitudinal depiction of patient’s asthma control over time that can assist the PCP in assessing whether a patient’s preventive asthma therapy is effective or needs adjustments. Our intervention represents a shift in asthma management from the current reactive acute care model to a preventive, proactive approach where treatment decisions (e.g. step up or step down therapy) are tailored to patients’ individual patterns of asthma chronic control to prevent acute exacerbations.
While development of self-management tools is becoming increasingly popular, little data exists regarding usability testing for asthma self-monitoring and self-management applications for children. Usability testing provided valuable input in the development of the e-ASTER application, including provision of a deeper understanding of the users’ needs and requirements for broad acceptance of the application. Usability testing uncovered several user performance issues including aspects of design format, content, navigation, layout, and feedback messages. New issues uncovered, not previously reported in the literature, were 1) how to encourage continued engagement in the application when asthma is well-controlled, and 2) parental concerns that poor scores on the e-ASTER application may reflect poorly on them, discouraging future use.
In addition to addressing other usability concerns, the novel issues identified were addressed through minor changes to the application including adding a positive feedback message when patients are doing well and information in the tutorial that a low score is not a reflection of parenting skills. We support regular use of the application, even when patients are well controlled for the following reasons: 1) it takes less than 5 minutes to complete an assessment, 2) the application puts only a minimal burden on the patients or parents, but the potential benefits are considerable, and 3) regular and long-term use of the application provides an opportunity to correlate variations (improvements and deteriorations) of asthma control overtime with seasonality, and exposure to asthma triggers, including change in weather, temperature, and spikes in environmental triggers, etc. Despite some usability issues, participants responded favorably to the e-ASTER application and found it useful and acceptable, indicating their willingness to use it in the future. They also believed that the application can help them improve their (or their child’s) asthma self-management skills and communication with their PCPs.
To reduce the risk of asthma exacerbations, asthma guidelines recommend ongoing monitoring and patient’s self-management with timely adjustments of preventive therapy to achieve and maintain optimal control. Yet existing tools developed and validated for monitoring chronic asthma control have demonstrated significant limitations when used in children. For instance, spirometry has been criticized for lack of availability, high cost, age-related difficulty in performing the maneuvers required in children and lack of feasibility for regular monitoring and self-management. The Asthma Control Questionnaire (ACQ) has been validated for weekly assessment, but classifies patients into only 2 categories (well or not-well controlled) and has less discriminative power for use in routine clinical practice, with clustering of scores at the lower end of a more limited distribution. The Asthma Control Test (ACT) is a short, easy to complete questionnaire and the most commonly used tool.[17, 18] The ACT is responsive to change and has a good distribution of scores across a wider scale for improved discrimination, but assesses asthma control over the prior 4 weeks and was not initially validated for regular scheduled use as our research team did in its paper-based format.
Peak flow (PF) is widely used for daily or weekly assessment of chronic asthma control in a fashion resembling our approach with the e-ASTER. However, while PF monitoring and charting can theoretically be very useful, there are limitations of PF monitoring that are circumvented with the e-ASTER approach, including: 1) daily PF monitoring is labor intensive for parents and children and has been associated with non compliance. 2) PF is an effort-dependent tool, limiting its effectiveness in younger children,  3) PF readings are not always reproducible over time, 4) PF can under-estimate the degree of airflow limitation as it primarily assesses airflow in larger airways, 5) the green/yellow/red zones of the PF are based on percent of personal best PF measures, but many parents and physicians use percent of predicted PF instead and a child’s personal best PF may be significantly different from the predicted PF,[22, 23] and 6) many providers forget that PFs increase with height, so that a child’s personal best/predicted PF will increase as they grow, necessitating recalculation of target zones periodically and making it difficult to use for ongoing monitoring.
Self-management is defined as the ability of a patient to assume a primary role in managing his/her condition, including: on-going monitoring of symptoms, adjusting medications and determining in a timely manner when medical attention is necessary. Barlow et al. stated that self-management is a dynamic and continuous process of self-regulation, that is, the ability to monitor one’s condition and to affect the cognitive, behavioral and emotional responses necessary to maintain a satisfactory HQOL. Online self-monitoring and self-management tools have been developed and tested to improvement management of various chronic diseases, including: diabetes,[26–30] body weight, depression, ulcerative colitis, HIV positive, alcohol use disorders, pain and symptoms in Sickle cell disease, high blood pressure, and adult asthma. Yet no online tools has been designed to address the need for children, their parents and their care providers concurrently.
The e-ASTER application offers several advantages over the existing tools. Our application uses a modified ACT test, adapted and validated for weekly assessment, and is primarily a weekly symptom-based, short questionnaire that is easy to use by older children, with or without parental assistance, or by parents of younger children. In addtion, the e-ASTER application was designed to engage not only children in self-monitoring of chronic asthma control but also their parents and care providers to support timely and optimal preventive asthma care. The e-ASTER application provides reminders and other incentives to encourage long-term use by not only patients and their families but also their care providers. Specifically, the e-ASTER provides email and text message reminders to patients and/or their parents when an assessment is overdue by more than a week. In addition, the registration process is designed so that patients are registered for the first time through his/her PCP’s office by a care manager. Enrolling patients through their PCP’s office is an additional incentive to support ongoing use as the patient’s physicians are involved in this process. Furthermore, the tool is being implemented as part of the physicians’ maintenance of certification (MOC) in pediatrics as it allows physicians to participate in quality improvement activities by supporting compliance with evidence-based asthma care including assessment of chronic asthma control and use of appropriate preventive asthma therapy. Use of the MOC is an additional incentive to encourage PCPs to use the tool and through them patient compliance will increase. Finally, the tool itself is an incentive as it is designed to improve patients’ asthma self-management skills and control.
The e-ASTER application focuses on tracking trends in chronic asthma control over time rather than on individual, intermittent assessments. The e-ASTER also addresses barriers to ongoing monitoring and self-management of asthma through the continuum of asthma care by promoting regular self-assessment and allowing continual data flow between patients, parents and PCPs to facilitate medical decision-making. Longitudinal data generated by the e-ASTER application provides a unique picture of a patient’s individual patterns of chronic asthma control that may be used to personalize asthma management. Specifically, longitudinal data and color-coded graphs may be correlated with asthma control deteriorations that can be attributed to: 1) ineffective asthma therapy, 2) patient’s non-compliance with therapy, 3) transient or sustained environmental exposure to asthma triggers, and 4) impeding risk of a severe asthma exacerbation, to help a PCP evaluate episodic changes in asthma control status over time to facilitate a personalized proactive intervention.
There is an increasing interest nationally for interventions including use of information technology (IT) tools to promote self-monitoring and self-management patients with chronic diseases. While many IT tools are developed and implemented by health care or other professionals with little or no input from end users,  development of our application involved not only a multidisciplinary team with clinical and educational expertise, as well as an understanding of asthma care processes, but also solicited input from end user representatives. We also had a list of application requirements including required functionalities for both children/parents and PCPs prior to development of the e-ASTER application. As multiple key holders (e.g. parents and care providers) are important to achieve optimal care of children with chronic diseases, there are few general principles of the e-ASTER design that could be applied to development of future self-management applications for children. These include: 1) designers of self-management applications for children with chronic diseases should incorporate functionalities that address the need for sustained used by both children and parents to ensure ongoing use of these systems, 2) stakeholders of self-management tools for children (children, parents and their care providers) should be involved early in the design so that the end product is acceptable and usable for them, 3) although we focused on asthma, our project, including use of stepwise approach, provides a model for developing patient-centered IT systems for supporting self-monitoring and self-management of chronic disease in general, especially those targeting end users of differing capacity, such as children and their parents. The development process including weekly meetings with the web-developer/programmer to review interim designs and prioritize tasks to accomplish for the following meeting is critical for success, 4) our study highlights the value of conducting usability testing prior to broad implementation of self-monitoring and self-management applications that involve children to ensure that the end product is usable and acceptable to both children and their parents, and 5) developers of self-management tools for children should also provide reassurance to parents that the goal of these tools are to support and enhance their skills in managing their child’s chronic disease and not to assess deficiencies in their parenting skills.
Our study has several limitations. First, our study results may be biased. The study was conducted in a single tertiary pediatric care center, the sample of participants for usability testing was small, and some usability problems may have been unnoticed. However, we believe that our evaluation identified most usability issues since previous studies have reported that even with a small representative sample of end users, involving as few as 5 participants, about 80% of usability issues can be identified. Second, while several usability testing methods have been proposed, there is no consensus on the best approach to use. Third, the e-ASTER application was developed to provide care providers with both longitudinal data that can be used to determine the effectiveness of asthma care and real-time alerts to promote responses that might optimize chronic asthma control and reduce the risk of severe asthma exacerbations. Although, we informally received some input from PCPs, formal usability testing of PCPs has not been conducted. Additional testing of the application with PCPs may be needed to answer usability-related questions for busy clinics where care providers have time and resource constraints. Therefore, additional changes to the e-ASTER application may be required. Finally, although usability testing uncovered some issues with the e-ASTER application, little is known about factors and characteristics that promote regular and sustained participation of children and parents in ongoing asthma self-monitoring and self-management programs. Future studies will determine generalizable factors (including barriers and facilitators) that contribute to children and/or parents’ participation in such programs.
Poor asthma control is frequently associated with patient exposure to environmental triggers including poor air quality with inhaled particulates, inhaled allergens, viral respiratory infections and even factors such as stress, physical activity and weather.[41, 42] Our long-term research plan is to link longitudinal asthma control data generated by the e-ASTER application with environmental data (including air quality, viral prevalence data, allergy data, temperature, humidity, and data from self-reported home trigger assessments) to personalize asthma care by correlating individual patterns of variation in asthma control status over time to changes in specific environmental triggers.[43, 44] We also are planning to align additional information collected by the e-ASTER application (patient compliance with controller medication, unexpected clinic visits, ED or hospital admissions for asthma) with the color-coded graph for clinicians so that it can be used by care providers to relate patients’ patterns of asthma chronic control and exacerbations with asthma outcomes. Furthermore, we are planning to add two other modalities to the e-ASTER application, including development of a mobile e-ASTER application and use of interactive voice recognition (IVR) system for patients/parents who do not have access to the Internet or a mobile phone. Finally, further studies are needed to determine patient factors associated with long-term use of the e-ASTER application and its effectiveness on chronic asthma control, specifically to determine whether use of the e-ASTER application is associated with decreased asthma exacerbations, better quality of life, and reduced acute health care utilization.
Studies of asthma self-monitoring and self-management have focused mostly on adults.[45–50] Like adults, children experience significant economic and health-related quality of life burdens from chronic illnesses. The e-ASTER application was designed to address this gap. The development involved a multidisciplinary team and end-user input, resulting in an easy to use and acceptable application for children with asthma and their parents. This novel application addresses some of the deficiencies of existing asthma monitoring tools and can shift asthma care from the current reactive acute care model to a preventive, proactive approach where treatment decisions could be tailored to patients’ individual patterns of chronic asthma control to prevent acute exacerbations.
Drs. Nkoy, Stone, Fassl and Maloney are supported by grant 1R18HS018166-01A1 (Organizational Factors Associated with Improved Inpatient Asthma Care) and grant 1R18HS018678-01A1 (Improving Post-Hospital Transitions and Ambulatory Care for Children with Asthma) from the AHRQ. Dr. Stone is also supported by Award KM1CA156723 from the National Cancer Institute (NCI). None of the sponsors participated in design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, or approval of the manuscript. Its contents are solely the responsibility of the authors and do not necessarily represent the official view of the AHRQ and NCI.