|Home | About | Journals | Submit | Contact Us | Français|
This study evaluates the validity, inter-rater reliability, and stability over 3 months of a semi-structured telephone interview measuring adherence to stimulant treatment, the Stimulant Adherence Measure, against the Medication Event Monitoring System (MEMS®).
Clinic-referred children (N=22, age 11.85 ± 2.1 yrs) using psychostimulants for DSM-IV attention-deficit/hyperactivity disorder (ADHD) were eligible. Families used a MEMS® device for the primary stimulant medication. Children and parents participated in a semi-structured telephone interview, the Stimulant Adherence Measure, for 3 consecutive months. Parent reports for previous 7 days and 28 days and child report for previous 7 days of medication use were compared to MEMS® report. Inter-rater reliability and interview order were also examined.
Nineteen children and parents completed (86%). Agreement between MEMS® and parent report for previous 7 days at months 1, 2 and 3 (ICC=0.829, p<0.001; ICC=0.663, p<0.05; ICC=0.878, p<0.001 respectively) and for 28 days at months 1, 2 and 3 (ICC=0.793, p<0.001; ICC=0.907, p< 0.001; ICC=0.806, p<0.001 respectively) was good to excellent. Agreement between MEMS® and child report for 7 days at months 1, 2 and 3 (ICC=0.773, p<0.001, ICC=0.542, p<0.05, ICC=0.606, p<0.05 respectively) was good. Inter-rater reliability was excellent (ICC=0.956, p<0.001). There was no interview order effect for parents (F=1.771, p>0.05) or children (F=1.621, p>0.05).
The Stimulant Adherence Measure provides a valid and reliable method for determining stimulant medication use by children with ADHD.
Analyser, sur une période de trois mois, la validité, la fiabilité inter-juges et la cohérence d’une méthode de mesure du respect du traitement par stimulants basée sur une entrevue téléphonique semi-structurée, et la comparer au système MEMS® de surveillance de la prise de médication.
Vingt-deux enfants (n = 22 âgés de 11,85 ans ± 2,1 ans) traités aux psychostimulants pour un TDAH tel que défini dans le DSM-IV ont été référés par une clinique. Le bouchon du flacon du stimulant était équipé d’un microprocesseur MEMS®. Les enfants et les parents ont participé à une entrevue téléphonique semi-structurée destinée à mesurer le respect du traitement pendant trois mois consécutifs. Les auteurs ont comparé le rapport des parents sur les sept jours avant le traitement et sur les 28 jours de traitement ainsi que le rapport de l’enfant sur les sept jours avant le traitement à celui produit par le microprocesseur MEMS®. Ils ont également analysé la fiabilité inter-juges et l’ordre d’entrevue.
Dix-neuf enfants et parents (86 %) ont terminé l’étude. La concordance entre le rapport MEMS® et le rapport des parents sur les sept jours avant le traitement les premier, deuxième et troisième mois (ICC=0,829, p<0,001; ICC=0,663, p<0,05; ICC=0,878, p<0,001 respectivement), et sur les 28 jours de traitement les premier, deuxième et troisième mois (ICC=0,793, p<0,001; ICC=0,907, p<0,001; ICC=0,806, p<0,001 respectivement) allait de bonne à excellente. La concordance entre le rapport MEMS® et celui de l’enfant sur les sept jours les premier, deuxième et troisième mois (ICC=0,773, p<0,001, ICC=0,542, p<0,05, ICC=0,606, p<0,05 respectivement) était bonne. La fiabilité inter-juges était excellente (ICC=0,956, p<0,001). L’ordre dans lequel l’entrevue avait eu lieu n’a eu d’effet ni sur les parents (F=1,771, p>0,05) ni sur les enfants (F=1,621, p>0,05).
La méthode de mesure du respect du traitement par stimulants basée sur une entrevue téléphonique semi-structurée est fiable, et permet de suivre la prise de médication par les enfants souffrant de TDAH.
Affecting 4.8% of children (Waddell, Offord, Shepherd, Hua & McEwan, 2002), Attention-Deficit/Hyperactivity Disorder (ADHD) is characterized by excessive inattention, hyperactivity, and impulsivity. Controlled trials suggest that stimulant medication is effective at alleviating symptoms of ADHD over the short-term, (Goldman, Genel, Bezman, & Slanetz, 1998; Greenhill, Pliszka, Dulcan et al., 2002), and that the majority of children respond to properly titrated stimulants (Goldman et al., 1998). Long-term follow-up studies, however, indicate that children with ADHD struggle with academic and social function when they grow older, even when their treatment history documents stimulant prescriptions (Hechtman, 2006).
Examining whether children actually use their medication as prescribed might provide a reasonable explanation for the disparity between short and long-term outcomes. A recent systematic review of randomized control trials of stimulant medication argued that assessing medication adherence is necessary in order to maintain methodological rigour when evaluating outcomes of long-term stimulant treatment (Schachar, Jadad, Gauld et al., 2002). Despite the importance of assessing adherence, the process is often confounded by methodological measurement problems (Hechtman, 2006; Brown, Borden, & Clingerman, 1985), including, but not limited to, the paucity of validated, easy-to-use measurement tools.
While objective measures are available to assess medication adherence, there are limits to their use. Biologic measures can examine the concentration of medication in the blood or urine, but are expensive and time consuming for researchers and can be invasive and uncomfortable for children (Osterberg & Blaschke, 2005). Indirect tools such as pill counts and electronic monitoring devices can give a good estimate of the number of doses missed, but cannot provide reasons for medication non-adherence and are generally difficult to use when measuring adherence to multiple medications. Though prior research has suggested that parent and child reports are not as trustworthy as these objective measures, due to a tendency to over report adherence (Brown, et al., 1985; Frey & Naar-King, 2001), parent and child reports have the potential to supplement objective measures by providing further details on why medication is not being taken.
In order to determine whether structured interviews with parents and with children can provide accurate reports of medication use, we developed the Stimulant Adherence Measure, a short telephone interview that can be used to quickly and non-invasively measure adherence in children with ADHD. The Stimulant Adherence Measure is loosely based on the Treatment Monitoring Questionnaire, an earlier parent-report instrument used to measure stimulant adherence on an annual basis (Charach, Ickowicz, & Schachar, 2004; Schachar, Tannock, Cunningham, & Corkum, 1997; Thiruchelvam, Charach, & Schachar, 2001) and makes an initial indirect inquiry about medication use followed by more specific probes. The aim of the study was to measure the tool’s inter-rater reliability and validity against an electronic medication monitoring device, the Medication Event Monitoring System (MEMS® (Aardex, 2005, 2006)).
Twenty-two children, aged 8–15, followed in a child psychiatry medication monitoring outpatient clinic and their parents participated. Children were included if they had a DSM-IV diagnosis of ADHD (American Psychiatric Association, 1994), were using stimulants, and were able to participate in three telephone interviews. Children were excluded if they had an IQ < 80, a medical illness contraindicating stimulant use, or had recently experienced trauma. The internal Research Ethics Board approved the study prior to initiation and written consent (parents) and assent (children) was obtained by the study coordinator without the family’s clinician present.
Participants were given a MEMS® (Aardex, 2005, 2006) device at enrollment to be used for the child’s primary ADHD stimulant medication. Families were contacted by telephone by a trained clinical interviewer who had no clinical responsibility for the participants once a month for three months. During each interview, children and their parents were interviewed separately by the same interviewer; children were asked to describe their medication use over the week prior to the interview date, and parents were asked to describe their child’s medication use over the week and month prior to the interview date. At the completion of each interview, the interviewer recorded the number and percentage of pills taken over the previous 7 days as described by parent and by child, and over the previous 28 days as described by parent. The percentage of pills taken was calculated by dividing the informant’s report of pills taken by the parent’s report of number of pills prescribed. Regardless of parent’s report of their child’s clinician’s treatment plan, it was assumed stimulants were to be used daily. For example, where the routine was to use pills for five school days per week the documented percentage was 71%, irrespective of whether the prescribing physician recommended that medication not be used on weekends.
Telephone interviews were tape recorded and listened to by a second rater who independently determined the number and percentage of pills taken over the previous 7 and 28 day periods. The estimates from the two raters were summed and averaged to determine the informant’s estimate of medication use.
The Stimulant Adherence Measure is a semi-structured telephone interview designed to elicit a detailed description of medication use from parents and children that takes 5 to 15 minutes to complete. The interview script begins by asking indirect questions related to treatment usage, including perceived benefits and any concerns about the medication. The informant is then asked details about medication types and dosing schedules used. Following this is both a general and detailed inquiry about adverse effects and an inquiry about whether the parent or child has considered stopping the medication. Finally, the informant is asked to estimate how many pills the child did not take over the past 7 days and to indicate reasons for missing pills. Any discrepancies between parent’s report and the child’s report were clarified by speaking with the parent and child together and asking them to discuss their different impressions of how medication was used. For the statistical analyses, discrepancies between parent and child reports remained unchanged. This was repeated for the 28 day estimate with parents as the only informants. Initially, children were also asked about their medication use over the past 28 days. However, the first three children were unable to answer the question, and therefore it was removed from the interview script. The interview script contains a subsection for participants who started and then temporarily stopped using medication and for participants who discontinued medication. The interview scripts can be obtained at http://www.sickkids.ca/psychiatry/.
This device recorded the date and time the pill container was opened using an electronic computer chip in the cap. Data was downloaded directly into a computer program. This tool reliably measures outpatient medication use in clinical and research settings (Schwed et al., 1999).
All analyses were completed using SPSS 15.0.
Participant report and MEMS® report of adherence were analysed using a two-way mixed-model Intraclass Correlation (ICC), consistence index type. Nine such correlations were performed comparing informant report to MEMS® data from the identical time period. Three compared parents’ weekly report to the MEMS® data from the same time frame, 3 compared parents’ monthly report to MEMS® data, and 3 compared the child’s weekly report to the MEMS® data.
To evaluate agreement among raters, two-way-mixed-model ICCs (consistence index type) were performed. The first contrasts rater 1’s estimate of percent adherence with rater 2’s estimate of percent adherence when the interviews covered 7 days. The second does the same over 28 days. To evaluate the stability of the measure over time, two two-way repeated measures ANOVAs were performed. The first measures the stability of parent estimates by comparing two within-subject factors: order of interview (number 1, 2, or 3) by method of report (MEMS® at 7 and 28 days, parent at 7 and 28 days). The second measures the stability of child estimates and compared the same two within- subject factors, order of interview and method of report, but used only two levels (MEMS® at 7 days and child at 7 days) in the report factor.
Nineteen of 22 families (86%) completed the study, of which 84% were boys, aged 11.9 ± 2.1 years. Three children, all boys and aged 8.5 ± 0.6 years, withdrew. One discontinued medication due to adverse effects and subsequently withdrew from the study; the other two did so due to logistical difficulties with the protocol. One family did not use the MEMS® for the first two interviews and was excluded from those analyses. See Table 1 for a description of the sample. Some participants stopped medications for school holidays and then restarted; the data was gathered whether or not children were currently using medication. There was excellent agreement between raters’ estimates of weekly parent report (ICC=0.946, p<0.001), raters’ estimates of monthly parent report (ICC=0.956, p<0.001), and raters’ estimates of weekly child report (ICC=0.927, p<0.001). Agreement between parent report and MEMS® was good to excellent between parent report and MEMS® for both weekly and monthly interviews (Table 2). Agreement between child report and MEMS® was very good for the week prior to interview 1 (Table 3).
While it may appear that the accuracy of the child interview lessens over time (Table 3), there was no statistically significant interaction between method of report and interview order for either parents (F=1.771, p>0.05) or children (F=1.621, p>0.05).
The current results demonstrate that a semi-structured telephone interview, the Stimulant Adherence Measure, measures medication adherence in children with ADHD comparably to objective measures such as the electronic MEMS®. Though prior research has suggested that parent and child reports are inferior to objective data collection methods (Hack & Chow, 2001), our research suggests that semi-structured subjective measures can provide similar information to that gathered from objective measures, recognizing that in clinical practice the use of such measures is rarely feasible (Stephenson, Rowe, Hayne, Macharia, & Leon, 1993). The Stimulant Adherence Measure interviews can be used at monthly intervals for at least three consecutive months to document stimulant adherence in children with ADHD in a manner comparable to an electronic medication monitoring device. The measure appears to be more accurate for parent ratings of stimulant adherence than for child ratings.
The child and parent interviews of the Stimulant Adherence Measure were designed as an easy-to-use clinical tool for monitoring child medication adherence over an extended length of time. The tone of the interview scripts is one of collaboration and exploration so that the child and parent are offered opportunities to describe what is happening in a non-judgmental fashion. Because it was designed to be used in children up to age 15 years, there were no study requirements for the parent to watch the child take the pills. Such close parental monitoring would have been developmentally inappropriate for the older children in the study and, in a clinical tool, unsustainable over an extended period of time. It is important to note that the interviewers did not simply ask informants about the number of pills ingested. Rather, following years of experience using the precursor interview measure, (Charach et al., 2004; Schachar et al., 1997; Thiruchelvam et al., 2001) we hypothesized that initial indirect inquiry followed by specific questions about medication use would result in parents and children offering a reliable and accurate picture of recent medication-taking behaviour. In particular, asking questions related to the benefits, side effects, and dosing patterns of medication use can facilitate reporting of non-adherence (Osterberg & Blaschke, 2005). Since study participants exhibited a range of medication usage patterns, the measure can be used for those with diverse medication-taking behaviour.
This study is limited by its small sample size and the fact that it was administered to clinic-referred children rather than a community-based sample. Due to the small sample size, we were unable to determine whether co-morbid conditions, such as Conduct or Oppositional Defiant Disorder, adversely affected the accuracy of child or parent report. It is certainly plausible that a child or parent could have been untruthful about the use of medication, although there was little social desirability incentive to do so, as the interview script communicated acceptance of potential choices to discontinue medication at the beginning of the interviews. To guard further against social desirability bias, the person interviewing the family shared no information from the interviews with the treating clinicians. There may also have been a slight tendency for parents to over-report medication use about monthly intervals compared to weekly intervals, although the accuracy of parent reports remained very good. Children were able to report medication use over the prior week but not the prior month. Finally, the mean age for the three children that discontinued the study was almost three years younger than those children that completed. Though further investigation should determine whether this measure is effective in younger children, it should be noted that poor adherence to medication is a particular concern during adolescence (Wolraich et al., 2005). The Stimulant Adherence Measure can reliably monitor stimulant adherence for early adolescent children potentially at high risk of poor adherence.
Despite the limitations of this research, it does provide preliminary evidence that the Stimulant Adherence Measure is a valid and reliable method for determining stimulant medication use by children with ADHD, providing information similar to that of electronic monitoring devices. There is little evidence on stimulant adherence amongst children with ADHD (Hack & Chow, 2001), and the evidence that exists tends to be inconsistent (Gau et al., 2006). This interview tool may help gather further information on the area. The measure could be used both to research adherence directly, and to provide a method for addressing adherence as a potential confound in other clinical research (Schachar et al., 2002). Clinical practice guidelines recommend consistent monitoring of stimulant medication adherence to enhance long term treatment effectiveness (Greenhill et al., 2002; Perrin et al., 2001). These telephone interviews, designed for use as a clinical tool to monitor stimulant adherence at regular intervals, could be helpful for such monitoring. The Stimulant Adherence Measure can be accessed at http://www.sickkids.ca/psychiatry/.
This research was supported by the Hospital for Sick Children Department of Psychiatry Endowment Fund. Dr. Charach is the recipient of a New Investigator Fellowship from the Ontario Mental Health Foundation