|Home | About | Journals | Submit | Contact Us | Français|
Objectives Adherence to medical regimens for children and adolescents with chronic conditions is generally below 50% and is considered the single, greatest cause of treatment failure. As the prevalence of chronic illnesses in pediatric populations increases and awareness of the negative consequences of poor adherence become clearer, the need for reliable and valid measures of adherence has grown. Methods This review evaluated empirical evidence for 18 measures utilizing three assessment methods: (a) self-report or structured interviews, (b) daily diary methods, and (c) electronic monitors. Results Ten measures met the “well-established” evidence-based (EBA) criteria. Conclusions Several recommendations for improving adherence assessment were made. In particular, consideration should be given to the use of innovative technologies that provide a window into the “real time” behaviors of patients and families. Providing written treatment plans, identifying barriers to good adherence, and examining racial and ethnic differences in attitudes, beliefs and behaviors affecting adherence were strongly recommended.
Historically, the field of psychology has made significant and lasting contributions to the development of reliable and valid measures, with recent efforts focused on measures that are directly linked to interventions and treatment outcomes (Quittner, 2000; Sechrest, McKnight, & McKnight, 1996). In fact, instrument development and assessment is one of psychology's greatest strengths. Thus, this Special Series logically extends prior reviews on evidence-based treatment to critical evaluations of assessment measures (Mash & Hunsley, 2005; Nelson-Gray, 2003). The purpose of this article is to provide an evidence-based review of measures that assess adherence to medical regimens for children and adolescents with chronic conditions, with an emphasis on their utility in research and clinical contexts.
There is little controversy about the definition of adherence. Over the past 30 years, adherence has been defined as: “The extent to which a person's behavior (in terms of taking medications, following diets, or executing lifestyle changes) coincides with medical or health advice” (Haynes, 1979; pp. 1–2). The complicating issue is determining precisely what that “medical advice” is. Once the prescription has been determined, including dose, frequency, duration (if applicable), and timing, a rate of adherence can be calculated.
The assessment and treatment of adherence has become central to improving health outcomes as the prevalence of chronic conditions increases and as medical treatments become more complex. Advances in health care, including public health initiatives, vaccines, and better treatments for previously fatal conditions, such as leukemia and cystic fibrosis (CF) are partly responsible for this increase. Further, environmental pollutants and lifestyle issues (e.g., obesity) have led to dramatic increases in diseases, such as asthma and type 2 diabetes (DeAngelis & Zylke, 2006). Prevalence rates based on two large survey studies have reported similar estimates of children with chronic conditions—12.8% or approximately 9.3 million children (Newacheck & Halfon, 2000; van Dyck, Kogan, McPherson, Weissman, & Newacheck, 2004). In this context, adherence to medical treatments becomes a central issue in these children's and families’ lives: “an endless cycle of medications, treatments, procedures, and medical visits” (Anderson & Collier, 1994; p. 394).
Rates of adherence for children with chronic illnesses vary depending on the disease, complexity of the regimen, and measures that are used (DiMatteo, Giordani, Lepper, & Croghan, 2002; Quittner, Espelage, Ievers-Landis, & Drotar, 2000b; Rapoff, 1999). However, there is consensus across studies that rates of adherence are generally below 50% (Dunbar-Jacob & Mortimer-Stephens, 2001; La Greca & Bearman, 2003; Rapoff, 1999). Adherence is often better for simple behaviors, such as pill taking, but is substantially lower for more complex behaviors, such as dietary modifications, glucose monitoring, and nebulized medications (DiMatteo, 2004; Modi et al., 2006; Quittner et al., 2000a; Wysocki et al., 2000).
The consequences of poor adherence are extremely serious. Failure to take medications as prescribed can result in drug resistance, drug reactions, increased morbidity and mortality, and reduced quality of life (Dew et al., 2001; Kelly & Kalichman, 2002). Nonadherence has been estimated to compromise the health outcomes of pediatric treatments by an average of 33% and by as much as 71% (DiMatteo et al., 2002). Poor adherence also affects health care provider behavior, potentially leading to increased dosages or discontinuation of medication believed to be ineffective (DiMatteo et al., 2002). Data from clinical trials evaluating new treatments and effective doses can also be compromised by inconsistent or poor adherence to the treatment (Christensen, 2004). Finally, an estimate of health care dollars wasted due to poor adherence is approximately 300 billion dollars annually (DiMatteo, 2004). In sum, poor adherence has been cited as the single greatest cause of treatment failure.
There are a number of unique challenges faced by researchers and clinicians seeking to assess adherence behaviors. These challenges include, but are not limited to: (a) use of different time scales from 24 h recall to global self-reports over 1–3 months, making it difficult to combine data across measures; (b) advances in medical treatments which are progressing more rapidly than our ability to develop reliable and valid instruments; (c) data obtained from multiple respondents (e.g., patient, parent) without certainty about which respondent will provide the most accurate data; (d) determination of the prescription; and (e) rapidly changing treatments for patients.
One of the most difficult problems, which has largely been ignored, is determining exactly what the physician has prescribed so that a calculation of adherence is possible (Modi et al., 2006; Quittner et al., 2000b). Often in complex chronic conditions, such as diabetes, HIV-AIDS and CF, several treatments and dietary modifications must be adhered to each day. Extensive reviews of patient charts in pediatric pulmonary clinics, for example, indicate that little reliable or consistent information can be gathered from these charts on the treatment regimen (Quittner et al., 2000a).
Further complicating this process are the rapidly changing treatments for these patients, which may require changes to the regimen (e.g., intensive insulin regimens for diabetes; use of insulin pump) and greater patient responsibility (e.g., home intravenous antibiotics for CF). If physicians do not provide a written treatment plan to children and families, it can be very difficult to assess whether dosages, duration of treatment, or changes to the daily regimen were understood by the patient and family (Modi & Quittner, 2006a). Although this may seem like a basic step in assessing and improving adherence, surprisingly, few written treatment plans are provided to families, even for the most complex medical regimens [see the Prescribed Treatment Plan; (Modi & Quittner, 2006a; Modi et al., 2006) for an exception].
As described in the Introductory article (Cohen et al., in press), members of Division 54 formed review groups in eight content domains. This group of reviewers comprised the American Psychological Association (APA) Division 54 Evidence-Based Assessment (EBA) Task Force. First members of the Task Force were asked to generate a list of existing measures in their respective domains. The Adherence Workgroup identified measures of adherence that included self-report questionnaires and interviews, daily diaries, and electronic monitors. In 2003, Division 54 posted these measures on the listserve to assess their rate of endorsement; however, too few Division 54 members were using these measures to use this as the selection criterion. Instead, given that these measures were relatively new (most developed in the last 10 years), we reviewed all of the measures generated by the Adherence Workgroup for which we could find sufficient published psychometric data or information from the author/s. In addition, we conducted a literature search and added a few measures suggested by members of the listserve.
There are several different methods of assessing adherence behaviors: (a) self-report questionnaires and structured interviews, (b) diary measures, (c) electronic monitors, (d) prescription refill histories, and (e) biochemical assays [see (Drotar, 2000; Quittner et al., 2000b; Rapoff, 1999) for reviews]. We did not include biological assays because their measurement properties seemed beyond the scope of this review and we did not include pharmacy refill indices because they are often designed and accessed through pharmacy databases, which are not currently standardized. Each method has different strengths and weaknesses in terms of reliability, validity, and cost which were included in this review. A total of 18 measures were included (see Appendix A).
We used the EBA criteria that have been endorsed by APA (Chambless & Ollendick, 2001) to review the adherence measures. The three evidence-based categories are as follows: (a) well-established assessment (e.g., at least two research teams have published sufficient information evaluating the measure and establishing its strong psychometric properties); (b) approaching well-established assessment (e.g., measures have been published in at least two articles indicating reasonable or vague psychometric properties); and (c) promising assessment (e.g., measures have been published in at least one peer-reviewed article indicating reasonable or vague psychometric properties). Similar to the criteria for evidence-based interventions in the literature on psychological treatments, a measure must have been utilized by more than one investigative team to meet the criteria for “well-established.”
The use of patient or parent self-report is the most common method of assessing adherence. Both self-report measures and structured interviews have been developed to measure adherence for a variety of pediatric chronic conditions, including asthma, diabetes, HIV-AIDS, and CF. Self-report measures have a number of strengths. They are inexpensive, comprehensive, and available for multiple informants (patients, parents, healthcare providers). Structured interviews also allow for follow-up questions which can provide important information about patient perceptions of the regimen and specific barriers. However, weaknesses include the tendency to overestimate adherence, problems with accurate recall, assessment of global perceptions rather than frequencies of behavior (e.g., “Did you eat more calories today?” vs. “How many snacks did you eat today?”), and the difficulty of using them with younger children (8 years and below). Finally, for complex regimens, treatments may be shared or supervised by several caregivers (e.g., parents, grandparents, school staff), from whom the data must also be collected.
Eleven self-report or structured interview measures of adherence were reviewed across six chronic conditions: diabetes, transplantation, CF, asthma, HIV-AIDS, and spina bifida. Several of these measures were designed to be completed by both parents and adolescents. The largest number of self-report measures focused on diabetes (three of the nine), including the Self-Care Adherence Interview (SCAI) (Hanson et al., 1989), Self-Care Inventory (SCI) (La Greca, Swales, Klemp, & Madigan, 1988), and the Diabetes Regimen Adherence Questionnaire (DRAQ) (Brownlee-Duffeck et al., 1987) (Table I). In addition, the Disease Management Interview is being adapted for diabetes (Johnson & Quittner, 2001) but has not appeared in a peer-reviewed publication. All of these measures reported good psychometric properties in terms of either internal consistency or stability coefficients, and most had validity data that demonstrated a significant association between adherence and better glycemic control. The SCI and DRAQ adherence scores also converged with other measures—either the 24 h recall diary (Johnson, Silverstein, Rosenbloom, Carter, & Cunningham, 1986) or an assessment of health beliefs and social problem-solving skills (Bond, Aiken, & Somerville, 1992; Thomas, Peterson, & Goldstein, 1997). Both of the SCI and DRAQ measures were categorized as “well-established” and the SCAI was categorized as “approaching well-established.”
Two measures of adherence for patients who have undergone a solid organ transplant were reviewed. One measure, the Behavioral Affective and Somatic Experiences-Compliance Scale (BASES) (Phipps, Hinds, Channell, & Bell, 1994) reported good internal consistency coefficients and reasonable agreement between parents and nurses, but did not report any validity data. Two other empirical articles using the BASES have been published, but they were written by the same investigatory group and thus, this measure fell into the “approaching well-established” category. The other measure, the Self-Regulation Medication Adherence Battery (SRMAAB) (Tucker et al., 2001) did not report any psychometric information, but found convergence between these adherence scales and cyclosporine levels. Furthermore, this measure was sensitive to the cultural differences between African-American and Caucasian transplant patients, with unique predictors of adherence identified for each group. Thus, this measure was categorized as “promising.”
Two measures of adherence for CF were reviewed, the Disease Management Interview-CF (DMI-CF), originally titled the Treatment Adherence Questionnaire (Quittner et al., 2000b) and the Treatment Adherence Regimen Scale [TARS; (DeLambo, Ievers-Landis, Drotar, & Quittner, 2004)]. These measures have both parent and adolescent versions. Strong psychometric characteristics have been reported for the DMI-CF, including test–retest reliability, parent-adolescent agreement, and associations between adherence and knowledge of the treatment regimen. The DMI-CF has appeared in two independent peer-reviewed publications and thus is considered a “well-established” measure. The TARS was also found to have good psychometric properties for a majority of scales; however, it has only been utilized in one published study and is therefore considered “promising.”
Two measures of adherence for children with asthma were reviewed: the Disease Management Interview-Asthma (DMI-Asthma) (Modi & Quittner, 2006a) and the Family Asthma Management System (FAMSS) (Klinnert, McQuaid, & Gavin, 1997). The DMI-Asthma interview was developed for children with asthma ages 10 and older and their parents. This is a newly developed measure and little psychometric information is available. Good parent–child agreement was demonstrated and children's self-reported adherence was correlated with the number of barriers to adherence they reported. The DMI-Asthma was categorized as a “promising” measure. The FAMSS is a semi-structured interview conducted with both children with asthma ages 11–17 and their parents. Psychometric data on internal consistency, interrater agreement, and convergent validity with child self-efficacy, knowledge, and electronic data are all strong. Note that training and reliability for coders must be established prior to use. Information on the FAMSS has been published by two research teams and thus, is a “well-established” measure.
One measure of adherence was found for pediatric HIV-AIDS, the Pediatric AIDS Clinical Trials Group (PACTG): Adherence Modules (Farley et al., 2003). The PACTG has two modules that assess adherence which are administered by interview. No psychometric data have been published to date; however, the measure has been used in three published studies, with one reporting significant associations between adherence and virological response and two finding no association. Further psychometric work is needed on this measure, placing it in the “promising” category.
One measure of adherence for children with spina bifida was reviewed, the Parent Report of Medical Adherence in Spina Bifida Scale (PROMASB) (Holmbeck et al., 1998). This is a parent self-report measure which has shown generally good internal consistency, good agreement between mothers and fathers, and high interrater agreement. Validity data have not yet been published. This measure is classified as “approaching well-established.”
Diary measures can take several forms, including written logs, hand-held computers (PDAs) with time-sensitive prompts, and phone diaries completed directly with the respondent. Prior studies indicate that compliance with written logs is extremely poor and they are often completed just prior to their return, rather than on a daily basis (Johnson, 1993). Given that patients have difficulty adhering to their medical regimens, it is not surprising that they also have difficulty tracking their behaviors on diary cards. In contrast, modern diary methods, such as ecological momentary assessment [EMA; (Csikszentmihalyi & Larson, 1987)] and day reconstruction methods [DRM; (Kahneman, Krueger, Schkade, Schwarz, & Stone, 2004)] which assess daily activity patterns, hold considerable promise for measuring adherence because they obviate the most serious weaknesses of questionnaire/interview data—problems with memory and recall (Johnson, 1995; Modi & Quittner, 2006b; Quittner et al., 2000b).
The measurement of daily activities via phone or PDA has several advantages over self-report methods. First, data is collected in real-time or within a short, 24 hr period so that actual behaviors, rather than global estimates are obtained. Second, because data is collected in real-time, there is less memory decay and less potential for social desirability biases to affect reports. For example as adolescents are tracked through a 24 hr period by phone, it is difficult for them to insert an activity that did not occur; thus, the absence of time devoted to medical treatments indicates poor adherence. Furthermore, Modi and Quittner (2006b) reported that data from the Daily Phone Diary more closely converged with objective electronic monitors than self-report questionnaires. Diary methods also facilitate identification of barriers to adherence because of the temporal nature of the data. Finally, phone diaries can be unobtrusive measures of adherence because all activities over the previous 24 hr are recorded and patients and families are typically unaware that treatment activities are the focus of the assessment (Modi & Quittner, 2006b; Modi et al., 2006; Wiener, Riekert, Ryder, & Wood, 2004).
The disadvantages of phone diary methods include the greater time required by the patient, family and interviewer to complete the diary. In addition, scheduling phone calls can be difficult, some families do not have access to a phone, and they are not designed for use with children under the age of 8. Phone diaries also require research assistants who are trained and able to interview adolescents and parents comfortably over the phone. Finally, diaries produce extensive and often complex data sets that require more sophisticated analytic procedures (Larson & Delespaul 1992; Modi & Quittner, 2006b).
Two cued-recall diary measures have been developed to assess adherence, one for children with diabetes and one which has been applied to a variety of chronic illnesses (e.g., CF, asthma, HIV-AIDS) (Table II). The 24 hr Recall (Johnson et al., 1986) measure is a phone-based diary that has been used for two decades to assess adherence in children and adolescents with diabetes. Strong stability coefficients, good parent–child agreement, and associations between adherence and glycemic control have been reported. A recent article applying this measure to HIV-AIDS (Marhefka, Tepper, Farley, Sleasman, & Mellins, 2006) was published by a second, independent group, making it a “well-established” measure.
The second cued-recall measure, the Daily Phone Diary (the DPD) (Quittner & Opipari, 1994), was developed initially to assess adherence behaviors (as well as family activities) in children and adolescents with CF. The DPD measure is available for parents of children and adolescents with CF and adolescents with CF. Good stability over a 3-week period, high levels of interrater agreement, and strong convergence between the DPD and electronic monitors have been reported. The DPD was recently adapted for parents of children with asthma (Modi & Quittner, 2006b) and HIV-AIDS (Wiener et al., 2004), with modest-to-strong convergence found between diary-measured adherence and the MDILog (an electronic monitor for metered-dose inhalers), and significant associations found between DPD-reported adherence and viral loads in children with HIV. Published articles are from two independent research groups making this a “well-established” measure.
Technological advances in microprocessors have led to the development of automated measures of adherence. These monitors are now available to record and store information on the date and time of tablet or liquid medication removal from standard vials, removal of pills from blister packages, actuation of metered-dose inhalers, blood glucose test results, and patient diary notations on adherence or other clinical events, such as pain levels. These monitors can store information in real-time from several months to 3 years and can be downloaded into data files for analysis. This is one of the most exciting developments in adherence measurement, with some even calling electronic monitors the “new gold standard” (Cramer, 1995; Rapoff, 1999).
As with other measures of adherence, electronic monitors have advantages and disadvantages. In terms of advantages, electronic monitors provide a continuous and long-term measure of medication adherence in real-time, which is not available with any other measure. Monitors can also reveal a spectrum of adherence problems, including: (a) underdosing (the most common dosing error); (b) overdosing (which can contribute to toxic effects); (c) delayed dosing (dosing that exceeds recommended intervals, reducing therapeutic coverage); (d) drug “holidays” (omitting doses for several days without authorization); and (e) “white-coat” adherence or giving the appearance of adequate adherence by “dumping” medications or taking them consistently several days before clinic visits [see (Rapoff, 1999; Riekert & Rand, 2002) for reviews].
Despite these advantages, electronic monitors are a relatively new phenomenon and have several disadvantages. First, although the monitor records precisely when a pill bottle is opened, what they measure is “presumptive” dosing—an assumption that patients ingest what they dispense. However, this is a problem with all of the methods of measurement we have reviewed—only biochemical assays are able to confirm, through blood levels, that the drug has actually been taken. Monitors can also underestimate adherence if patients take out several doses at once to carry with them when they are away from home or to load pill reminder boxes.
Electronic monitors, like any mechanical device, can malfunction. They may record events that did not occur, fail to record events that did occur, or simply stop working due to battery failures. Missing data due to device failure has ranged from 0% to 24% (Riekert & Rand, 2002). Although mechanical failures are typically associated with the first or second prototypes of a device, malfunctions can also occur because of patient behavior (e.g., taking the device apart, cleaning it). The feasibility and clinical utility of monitors is also problematic because of the relatively high costs of purchasing the monitors, communicators, and proprietary software. For example, MEMS caps cost ~$130 per cap and families often request three or more caps. Practical issues, such as portability and improper fit between the medication and device may also reduce their utility. Unfortunately, development of monitoring devices has lagged behind development of new medications. For example, nebulized medications have been recently reformulated as dry powder. Finally, in order to download data from the monitors, they have to be retrieved and in some cases, patients have lost the monitors or have not returned them.
Traditional reliability estimates, such as internal consistency and test–retest reliability, are less relevant for electronic monitors than other methods of assessment. Accuracy is critically important and bench studies in which monitors are triggered by investigators at predetermined times and compared to written records of device actuations are critical for ensuring that monitors are functioning as designed. Periodic calibration and testing is also important when using these devices in an on-going study or clinical trial.
Electronic monitors have been considered more accurate than other measures because they are “objective” and typically reveal lower rates of adherence when compared to patient or parent self-reports, physician estimates, and pharmacy refill records (Rapoff, 1999; Riekert & Rand, 2002). This claim of greater accuracy rests on the assumption that electronic monitors are the “gold standard” of adherence, an issue which has been hotly debated (DiMatteo, 2004; Riekert & Rand, 2002). Although we have thoroughly reviewed electronic monitoring measures, it is difficult to categorize them using the EBA criteria. Thus, we have placed them all in the “well-established” category if they have been shown to be accurate and converge with at least one other “well-established” measure (Table III).
The MEMS bottle cap records the date and time of pill bottle opening. MEMS caps have been used to measure medication-taking in pediatric patients with HIV-AIDS, CF, β-thalassaemia, and tuberculosis (TB) (Blowey et al., 1997; Farley et al., 2003; Modi et al., 2006; Olivieri et al., Starr et al., 1999). It has been shown to be highly accurate in benchmarking studies. Convergent and predictive validity have also been established. The MEMS demonstrated good convergence with pharmacy refill and daily diary data for children with CF (Modi et al., 2006), as well as urine assays for patients with TB (Starr et al., 1999). MEMS data also predicted viral load in children with HIV-AIDS (Farley et al., 2003).
For patients with asthma and CF, several medications are administered via MDI, including β-agonists and steroids. Older electronic monitors, such as the Doser CT, recorded the date and number of puffs taken via the MDI. Accuracy of the Doser in bench studies was 94.3%; however, this device does not record the time of each actuation and data cannot be downloaded to the computer. Failure rate in one study was significant (21%) (Bender et al., 2000) and because the Doser uses an older technology, it is no longer available. The Nebulizer Chronolog is also an older electronic monitoring device that has been used to record asthma medications. Three studies of pediatric patients with asthma were located that used the Nebulizer Chronolog, which all showed that adherence was considerably lower as measured by the Chronolog than patient reports (Chemlik & Doughty, 1994; Gibson et al., 1995; Milgrom et al., 1996). For example, Bender and colleagues (1998) reported that complete use of β-agonists was 30% by patient report versus 12.7% with the Chronolog. Interestingly, inflated patient reports of adherence were correlated with lower parent education and less affective responsiveness in the family. Both the Doser and Nebulizer Chronolog are only briefly described in this review to provide relevant history regarding the use of electronic monitors; however, they were not included in the Appendices because they are no longer used for research or clinical purposes.
The newer MDILog was benchmarked in 2001 in a comparison of three MDILog devices. This device can identify errors in administration of the medication with an impressive rate of accuracy: actuation accuracy was 97–100%, inhalation accuracy was 86–95%, and late inhalations and multiple actuation accuracy was 97–99% (Apter et al., 2001). Importantly for children and adolescents, no artifactual recordings were observed when MDILogs were carried in bookbags over 3 days (Apter et al., 2001). The newest version of the MDILog is more accurate and reliable than previous versions. Rates of adherence on the MDILog correlated with other measures of adherence, such as the Daily Phone Diary for children with CF and asthma (Modi et al., 2006).
The Halolite™ is an adaptive aerosol delivery (AAD™) system that only releases aerosol medication when inhalation is detected. This shortens the treatment time for nebulized medications by 40%. The Halolite™ records the date, time, and duration of each nebulized treatment. In addition, any form of nebulized medication can be used in the Halolite™ (e.g., inhaled tobramycin, dornase alpha, bronchodilators), which is an advantage over newer nebulizers that can only be used with specific medications. Data from two studies on the Halolite™ suggest that it converges well with the Daily Phone Diary (r's =.88–.94) (Modi & Quittner, 2006b) and provides more accurate estimates of “true compliance” compared to traditional nebulizer monitors (Convway et al., 2002).
Eighteen measures of adherence for pediatric chronic conditions were reviewed. A significant strength of this area is the diversity of methods that have been developed to measure these complex behaviors, including self-report/structured interviews, daily diary procedures, and electronic monitoring. This systematic review provided an interesting perspective on the current state of pediatric adherence research and suggested several important directions for the future. First, there were surprisingly few self-report and interview measures in the published literature. Most likely there are other self-report and interview measures that are being used, but have not yet accrued sufficient psychometric data for publication. Only 11 measures were found, with four meeting the criteria for “well-established.” Three measures were in the “approaching well-established” and four were in the “promising” category. Second, several major pediatric chronic conditions, such as juvenile rheumatoid arthritis (JRA), sickle cell anemia, and epilepsy were not covered by any of the existing self-report or interview measures. One reason for this may be the skepticism with which self-report measures are viewed, given their consistent over-reporting of adherence behaviors (Quittner et al., 2000b; Rapoff, 1999). Examination of the psychometric evidence supporting these measures in Table I indicates good agreement between parents and children, but overestimates of adherence in comparison to diary and electronic monitoring.
What are the alternatives to traditional self-report or interview-based measures of adherence? One possibility is to use daily diary procedures which are, in fact, a form of self-report. However, this method of self-report is structured to reduce the memory and social desirability biases associated with paper–pencil measures. Both of the diary measures included in this review were rated as “well-established” and are being applied to a variety of chronic illness populations. Evidence of their convergence with other, more “objective” electronic monitors provides strong support for their validity in comparison to self-report questionnaires and interviews.
This shift toward “real time” assessment and the development of measures that capture daily activities and functioning can be seen in other areas of psychology, in which ecological momentary assessment (also called experience sampling) and day reconstruction methods are being applied (Csikszentmihalyi & Larson, 1987; Kahneman et al., 2004). Experience sampling methods have used electronic pagers to elicit affect and arousal during daily activities (Csikszentmihalyi & LeFevre, 1989). Schiffman and colleagues (2002) have used hand-held computers (PDAs) to record the use of cigarettes and antecedents of smoking behaviors, and preprogrammed wristwatches have been used to assess mood, pain and disability in patients with arthritis (Stone, Broderick, Porter, & Kaell, 1997). These methods often provide rich, sequential information about behavior and affect, which can increase our understanding of the processes that influence complex behaviors, such as adherence (Modi & Quittner, 2006b).
As electronic microchip technologies have advanced, their utility for measuring adherence has improved. Earlier devices that recorded MDI adherence were simplistic and provided minimal information. In contrast, newer generations and prototypes of the MDILog are more reliable and have added important features to assess the techniques patients use when taking their medication (e.g., shaking before inhaling). These devices, however, still assume that when the device is actuated, the medication is being taken by the patient. Electronic monitors have a number of other limitations, including malfunctions, battery failures, complex software that must be learned, and a high cost. Although they do provide more reliable and accurate data than other measures when they are working properly, managing these monitors requires technical expertise and a significant commitment of time and resources.
Although some consider electronic monitors the “gold standard” against which all other measures of adherence should be compared, data presented in this review do not support that position. Studies that used electronic monitors reported significant amounts of missing data, numerous device failures, the loss of monitors, and problems downloading data. Importantly, they are not available for all components of treatment (e.g., dietary alterations). It is also clear that most electronic monitors are not currently feasible for clinical use (blood glucose monitors are the exception), and thus their utility is limited to the research context. Improved technology and reduced cost will be required before electronic monitors can become the “gold standard.”
This review indicates that we have made substantial progress in the last decade in developing reliable and valid measures of adherence for pediatric populations. The methods of assessment in this area are among the most innovative in our field, and we now have an opportunity to “triangulate” two or more methods to obtain a more accurate assessment of adherence behaviors. This review also indicates that this area is in its infancy compared to assessment tools in other child specialties (e.g., academic achievement, child behavior problems), with additional research needed. The following is a list of recommendations that may help guide future instrument development and research on the assessment of adherence:
(a) Our review indicates that electronic monitors have not yet achieved the status of the “gold standard, and thus we suggest that studies of adherence include at least two methods of assessment (e.g., diary plus electronic monitoring). This increases the complexity of the analyses (see next recommendation), but will enable researchers to examine the extent of convergence across two different methods and potentially triangulate the data to obtain a more reliable result. This may not be feasible in clinical settings, in which both time and technical expertise may be lacking. However, for some chronic conditions, such as diabetes, blood glucose monitors are routinely downloaded as part of the clinic visit. As other types of monitors become more user-friendly and less expensive, they could also become part of standard care [e.g., e-Flow rapid® (Pari GmbH, München, Germany), for nebulized medications], facilitating a discussion of adherence with the patient and family. Unfortunately, in many subspecialty clinics, a “don't ask, don't tell” policy prevails and members of the health care team are often reluctant to ask patients and family members whether they are able to successfully manage their treatments.
(b) One of the methodological challenges of measuring adherence is the variation in rates of adherence related assessment method. As mentioned above, we recommend that a minimum of two measures be used—which increases the complexity of the data analysis. If rates of adherence differ (which is likely across self-report, diary, and electronic methods), which rate of adherence should be reported? How should data from different measures be integrated for analysis and interpretation? One suggestion is to use the electronic data (if available) to “correct” self-reported data from the child and family, allowing for self-report measurement error, and bias (Jasti, Siega-Riz, Cogswell, & Hartzema, 2006). For example, the regression coefficient for self-reported adherence from a simple linear regression on MEMS data can be applied to self-report. If the regression coefficient is.60 based on electronic monitoring data and self-reported adherence is 95%, the correction factor would yield an adherence rate of 57%. Another suggestion is to utilize newer modeling techniques, such as hierarchical linear modeling, which allows data from several measures to be combined to form an underlying construct (frequency of taking oral medications), with measurement error from each source defined and reduced in the modeling process (McCullagh & Nelder, 1989).
(c) The identification of barriers to adherence is a new and promising direction for measurement development. It is clear from studies of adherence over the past 20 years that patients and families find it difficult to comply with their treatments. However, the reasons for poor adherence are less well-understood and have only recently been measured (Modi & Quittner, 2006a). Three instruments assessing barriers to adherence were identified during this review, but were not included because they measure factors that impede adherence, rather than measuring adherence rates (Logan, Zelikovsky, Labay, & Spergel, 2003; Modi & Quittner, 2006a; Riekert & Drotar, 2002). We strongly recommend that studies of adherence include an assessment of the barriers that patients and families encounter. Identifying individual and family-level reasons for poor adherence will be critical as the field moves toward the development and evaluation of interventions to improve adherence behaviors.
(d) Another area that should be addressed is whether patients and families have the appropriate knowledge and skills to perform the treatments correctly. Although knowledge of the disease per se has not been strongly associated with rates of adherence, it is likely that treatment-specific knowledge (e.g., when and how to perform various treatments) is associated with adherence. For example, research in CF shows that parents and children have a number of misconceptions about when and how to do their treatments (e.g., taking enzymes before or after meals) which affect adherence and the efficacy of the treatment (Henley & Hill, 1990; Modi & Quittner, 2006a). Having the skills to perform the treatment is also an important issue which is rarely addressed. For example, most patients with asthma do not know how to use a metered dose inhaler correctly (Burkhart, Rayens, & Bowman, 2005). Although they are not direct measures of adherence, measures of knowledge and skills can provide critical information for an adherence intervention. If children are spending the time to take their medications, we need to make sure they have the knowledge and skills to gain the maximum benefit. This is even more important as children get older and begin to assume responsibility for their treatments, or encounter changes in their treatments with advancing age (e.g., nebulizer face mask vs. mouthpiece).
(e) In conjunction with the development of skills tests, we strongly recommend that comprehensive treatment plans be developed and utilized in clinics treating children with chronic conditions. These “treatment plans” are necessary for two reasons: First, medical regimens typically include multiple components (e.g., medications plus dietary alterations) with different dosages, time schedules and durations. This makes it extremely difficult for families to be adherent without having something in writing they can refer to. Second, medical charts often do not have complete information on treatment recommendations (e.g., duration and frequency of treatment) or on recent changes in the prescription, and thus it is difficult to determine what the family understands and is adhering to (i.e., the denominator in the adherence equation). As more hospitals and clinics move to electronic charts, this may provide an opportunity to record and provide written treatment plans to families.
(f) There is little data on how racial and ethnic differences in attitudes, beliefs and behaviors affect adherence to chronic therapies. Often studies of pediatric adherence do not have samples that are sufficiently large and representative to permit analyses of subgroups according to race and ethnicity; however, validation studies often do include minority samples. Data on these differences have been reported for African-American versus Non-Hispanic White families of children receiving kidney transplants (Tucker et al., 2002) and African-American, Hispanic, and Non-Hispanic White children and teens with CF (Quittner, Schechter, Rasouliyan, Pasta, & Wagener, 2006). However, more research is needed on this important topic. Tucker and colleagues (2002) presented a “cultural difference” rather than “deficit” model to identify culture-specific factors that are associated with treatment adherence. This model could be useful in a variety of pediatric chronic conditions and would likely have implications for intervention as well as assessment.
(g) We recommend that more refined EBA classifications be developed to better capture the measurement characteristics of adherence behaviors. Given that multiple methods are used in this area, it is not always possible to evaluate them with traditional psychometric parameters. Furthermore, finer distinctions between the categories in terms of reliability and validity would be beneficial (Mash & Hunsley, 2005).
In sum, improving adherence in children and adolescents with chronic illnesses is going to be the key to better healthcare delivery and health outcomes over the next 20 years. In order to address the complex issues related to adherence behaviors, reliable, ecologically valid, and empirically supported measures of adherence will be required. This systematic review highlights a number of well-established measures and methods that can be used for pediatric populations; however, further measurement development is necessary to advance the field.
Conflicts of interest: None declared.
Hanson, C. L., Cigrang, J. A., Harris, M. A., Carle, D. L., Relyea, G., & Burghen, G. A. (1989). Coping styles in youths with insulin-dependent diabetes mellitus. Journal of Consulting and Clinical Psychology, 57(5), 644–651.
The SCAI is a 15-item semi-structured interview for youth and their parents assessing adherence to the diabetes regimen.
The SCAI can be requested from Dr Cindy L. Hanson at University of Central Florida, Department of Psychology, Orlando, FL32816, USA.
La Greca, A. M., Swales, T., Klemp, S., & Madigan, S. (1988). Self care behaviors among adolescents with diabetes (Abstract). In Proceedings of the Ninth Annual Sessions of the Society of Behavioral Medicine (p. A42). Rockville, MD: Society of Behavioral Medicine.
The SCI is a measure of self-reported adherence to the diabetes regimen over a 1-month period of time.
The SCI was developed by Dr Annette La Greca and can be obtained from her at ude.imaim@acergala.
Brownlee-Duffeck, M., Peterson, L., Simonds. J. F., Goldstein. D., Kilo, C., & Hoette, S. (1987). The role of health beliefs in the regimen adherence and metabolic control of adolescents and adults with diabetes mellitus. Journal of Consulting and Clinical Psychology, 55, 139–144.
The DRAQ is a retrospective self-report measure of diabetes adherence.
The DRAQ was developed by Dr Martha Brownlee-Duffeck and can be obtained from her at vog.AV.dem@kceffuD-eelnworB.ahtraM.
Phipps, S., Hinds, P. S., Channell, S., & Bell, G. L. (1994). Measurement of behavioral, affective, and somatic responses to pediatric bone marrow transplantation: Development of the BASES scale. Journal of Pediatric Oncology Nursing, 11(3) 109–117.
The BASES measure was designed to assess the behavioral, affective, and somatic outcomes in the acute phase of bone marrow transplantation and includes a “Compliance” scale.
The behavioral affective and somatic experiences scale was developed by Dr Sean Phipps and can be obtained from him at email@example.com.
Tucker, C. M., Petersen, S., Herman, K. C., Fennell, R. S., Bowling, B., Pedersen, T., & Josmik, J. R. (2001). Self-regulation predictors of medication adherence among ethnically different pediatric patients with renal transplants. Journal of Pediatric Psychology, 26, 455–464.
The SRMAAB is a verbally administered adherence questionnaire for pediatric renal transplant patients, which assesses adherence motivation, perceived control of medication adherence, and perceived caregiver support of medication adherence.
Dr. Carolyn M. Tucker, Department of Psychology, P.O. Box 112250 University of Florida, Gainesville, FL 32611, USA. E-mail: ude.lfu@rekcutmc
Quittner, A. L., Espelage, D. L., Ievers-Landis, C. E., & Drotar, D. (2000b). Measuring adherence to medical treatments in childhood chronic illness: Considering multiple methods and sources of information. Journal of Clinical Psychology in Medical Settings, 7, 41–54.
The DMI-CF is an interview-based self-report measure of treatment adherence for patients with CF. It is used for children of 10 years of age and older and their parents. This has been adapted to children and parents of children with asthma.
DeLambo, K. E., Ievers-Landis, C. E., Drotar, D., & Quittner, A. L. (2004).
Association of observed family relationship quality and problem-solving skills with treatment adherence in older children and adolescents with cystic fibrosis. Journal of Pediatric Psychology, 29(5), 343–353.
The TARS is a self-report measure of treatment adherence for patients with CF and their parents.
The TARS is available in the original citation listed above.
Klinnert, M. D., McQuaid, E. L., & Gavin, L. A. (1997). Assessing the family asthma management system, Journal of Asthma, 34(1), 77–88.
The FAMSS is a semi-structured interview for caregivers (school-aged children can be included as an additional informant) designed to assess the management of children's asthma within the context of the family.
The FAMSS was originally developed by Dr Mary D. Klinnert. Correspondence should be addressed to Mary Klinnert, PhD, National Jewish Center for Immunology and Respiratory Medicine, 1400 Jackson St, Denver, CO 80206, USA.
Modi, A. C., & Quittner, A. L. (2006). Barriers to treatment adherence for children with cystic fibrosis and asthma: What gets in the way? Journal of Pediatric Psychology, 31(8), 846–858.
The DMI-Asthma is an interview-based self-report measure of treatment adherence for patients with asthma. It is used for children of 10 years of age and older and their parents.
The DMI-Asthma is available from Dr Avani Modi at firstname.lastname@example.org.
Van Dyke, R. B., Lee, S., Johnson, G. M., Wiznia, A., Mohan, K., Stanley, K., et al. (2002). Reported adherence as a determinant of response to highly active antiretroviral therapy (HAART) in children who have human immunodeficiency virus infection. Pediatrics, 109 (4), e61.
The PACTG is an interview-administered measure that is conducted with primary caregivers and it consists of two adherence modules. Module One assesses adherence to the HIV HAART regimen and Module Two assesses barriers to adherence.
The PACTG is available at www.fstrf.org/qol/peds/pedadhere.html
Holmbeck, G. N., Belvedere, M. C., Christensen, M., Czerwinski, A. M., Hommeyer, J. S., Johnson, S. Z., et al. (1998). Assessment of adherence with multiple informants in pre-adolescents with spina bifida: Initial development of a multidimensional, multitask parent-report questionnaire. Journal of Personality Assessment, 70(3), 427–440.
The PROMASB is a Likert scale parent-report measure of adherence for spina bifida and includes treatment aspects such as catheterization, bowel care, skin care, medication, and ambulation.
The PROMASB can be obtained in the appendix of the original citation.
Johnson, S. B, Silverstein, J., Rosenbloom, A., Carter, R., and Cunningham, W. (1986). Assessing daily management in childhood diabetes. Health Psychology, 5(6), 545–564.
The 24 hr recall interview was designed to assess adherence behaviors in childhood diabetes. Respondents were asked to recall the previous day's events and within interview, 13 measures of adherence are obtained; injection regularity, injection interval, injection–meal timing, regularity of injection–meal timing, calories consumed, percentage of calories from fat, percentage of calories from carbohydrates, concentrated sweets, eating frequency, exercise frequency and duration, exercise type, and glucose testing frequency.
The diabetes management interview was developed by Dr Suzanne Bennett Johnson and can be obtained upon request (E-mail: email@example.com).
Quittner, A. L., & Opipari, L. C. (1994). Differential treatment of siblings: interview and diary analyses comparing two family contexts. Child Development, 65(3), 800–814.
The DPD uses a cued recall procedure to track parents and/or older children/adolescents through their activities over the past 24 hr. For all activities lasting 5 min or longer, participants report the type of activity, duration, and who was present. Each activity, including medications and treatments, is recorded by the interviewer on a computer screen with clock hands which rotate through a 24 hr clock, a set of activities, companions, and a rating scale for mood.
The DPD was developed by Dr. Alexandra L. Quittner and can be obtained upon request (E-mail: ude.imaim@renttiuqa).
The MEMS cap is an electronic monitor embedded within a cap that fits on pill vial containers and records and stores information on the date and time the vial was opened.
APREX (division of AARDEX), 2849B Whipple Road, Union City, CA 94587. Phone: 877-227-3391; Fax: 510-476-1946; Website: www.aardex.ch.
This is an electronic device that attaches to a metered dose inhaler (MDI) and records the date and time of inhaled medication use, including whether the inhaler was shaken(to mix the medication in the canister), dispensed, and the aerosolized medication was inhaled.
Medtrac Technologies, Inc. A division of Westmed, 6950 W. Jefferson Ave. Suite 210, Lakewood, CO 80235, USA. Phone: 800-724-2328; Fax: 303-985-1014; Website: www.westmedinc.com.
The Halolite™ is an electronic device that records the date, time, and length of nebulized treatments (e.g., bronchodilators, inhaled antibiotics).
AAD devices are available from Profile Therapeutics, UK.