Consistent with previous research, this study documents that rates of adherence to AAP-recommended practice behaviors for ADHD care among community pediatricians are low and remain low without intervention. Although pediatricians in the control group did demonstrate improvement over time, likely because they knew that their performance was being monitored (ie, the Hawthorne effect14
), results of the cluster-randomized trial demonstrated that pediatricians in the intervention group showed significantly improved rates of many AAP-recommended ADHD care practice behaviors, including collection of parent and teacher rating scales for assessment of children with ADHD, use of DSM-IV criteria, and use of teacher rating scales to monitor treatment responses, compared with rates observed among pediatricians assigned to the control group. In addition to being effective for most care outcomes, the intervention was well accepted by pediatricians, who largely expressed high levels of satisfaction with the intervention model and would recommend it to other pediatricians. The intervention model was designed to be able to be disseminated widely, by combining quality improvement methods,15
and an innovative, Internet-based interface.
This is the first randomized trial to demonstrate the efficacy of a quality improvement intervention in improving community-based pediatricians' ADHD assessment and treatment practice behaviors. One of the primary focuses of the intervention is its ability to improve physicians' ability to collect teacher ratings during assessment and treatment. Teacher ratings typically are difficult to collect in pediatric settings, because of logistic problems with distributing forms and then obtaining completed ratings from teachers. The ADHD Internet portal seemed to alleviate typical barriers by allowing physicians to distribute and collect follow-up rating scales from teachers directly. When physicians registered patients on the Internet portal, there were high rates of return of teacher rating scales during assessment (83%) and treatment monitoring (55%). Moreover, ratings were returned quickly during assessment (median time to return: 3 days). Comparable rates of rating scale completion and turnaround times were observed for parent ratings.
An unexpected benefit of our intervention was the tendency for physicians in the intervention group to rely on themselves, rather than outside referrals, for documentation of DSM-IV ADHD criteria for their patients. Although all physicians in this study relied on outside referral sources to document ADHD criteria at baseline, physicians conducted the majority of ADHD evaluations for their patients after the intervention. One apparent effect of the intervention might have been to provide physicians with the confidence and tools to be able to conduct ADHD assessments without having to rely on other health care professionals. Given the high prevalence rates of ADHD and the shortage of pediatric mental health care professionals to conduct assessments,18
empowering pediatricians to conduct these initial assessments, particularly straightforward cases of ADHD, likely expedites the assessment and treatment process for families and reduces strain on the mental health system, thereby allowing children with more severe or complex presentations to obtain specialist care.
Another unexpected finding was the discrepancy between the baseline rates of ADHD care practice behaviors in the present study and existing literature findings.3,4,6
For example, pediatricians reported collecting rating scales from parents and/or teachers as part of their ADHD evaluations 67% to 87% of the time.3,19
In previous investigations of this intervention model using practice-selected charts, we reported higher baseline levels of practice behaviors.5,11
The present study's use of randomly selected patient charts, as opposed to physician self-reports or practice-selected patient charts, produced dramatically lower rates of evidence-based practice behaviors (ie, 14%–20% used teacher rating scales during assessments). The discrepancies between these different data collection modalities are considerable and should be considered in interpretation of the results of studies that use self-reports or reviews of nonrandomly selected charts.
Although significant intervention-related improvements in ADHD care quality were observed, there continued to be significant room for improvement in the quality of care at the end of the 6-month randomized controlled trial. For example, pediatricians in the intervention group still used parent and teacher rating scales to monitor treatment responses <50% of the time. The low rates of evidence-based ADHD treatment practices have repeatedly been found to be resistant to reliable implementation.5,7
Additional intervention components (eg, community collaborations20
or assignment of case managers21–23
) might possibly help parents and teachers to improve adherence to prescribed practices.
The finding that the intervention was not able to promote higher levels of adherence to AAP practice guidelines represents a potential weakness of our intervention model. One explanation for the continuing low rates of adherence after intervention implementation is that the 6-month intervention period did not allow enough time for change. Indeed, data from the 15-month naturalistic follow-up assessment suggested that some practice behaviors, particularly those related to ADHD assessment, continued to improve beyond the 6-month time point. Changing practice behaviors is a process that occurs slowly.19
It may be that full adoption of the intervention components takes time or that many cycles of tests of change are required before adoption is fully solidified.15
Another explanation for physicians' nonuniform implementation of the intervention components may be that appropriate incentives were not in place to promote reliable usage. Although earning American Board of Pediatrics Maintenance of Certification Performance in Practice credits may incentivize pediatricians to enroll and to participate initially, because this requirement is time-limited and involves only a subset of patients, it does not necessarily incentivize uniform implementation. Physicians might have perceived that the time and effort toward implementation of the prescribed intervention methods were uncompensated or undercompensated in the current reimbursement system and therefore they decided not to implement the methods with all of their patients. Practices were encouraged to bill for collection of rating scales, and many did so successfully. Even with this additional billing, however, physicians likely remained undercompensated for their time and effort in administering high-quality care. Incentives such as pay for performance24
and appropriate reimbursement for e-mail and telephone communications25
likely are necessary to promote uniform adoption and widespread dissemination of quality improvement interventions such as that described in this study.
The current study has some limitations. First, intervention efficacy was established only to 6 months after baseline. Although chart reviews were collected to 15 months after baseline and suggested continued improvement for some outcomes, without comparable data from a control group we are not able to assess the longer-term sustainability of intervention effects. Second, it was impossible to keep chart reviewers blinded to treatment condition because of the need to query the Internet portal at intervention practices for patient care information (ie, rating scale completion). Therefore, data were susceptible to rater bias.
Although this study demonstrates intervention effectiveness with respect to the quality of ADHD care, additional studies are needed to elucidate the intervention adoption process and the full breadth of intervention effects. For example, it is likely that barriers at the patient level (eg, poverty and parents' drug use21,26
) and physician level (eg, computer access in patient rooms and attitudes regarding change27
) prevent uniform adoption of this study's intervention. Furthermore, the Internet-based portal presented additional technological challenges to parents in the form of the need for Internet access21
and to physicians in terms of adding another electronic patient medical record. Identifying and addressing these barriers may facilitate consistent implementation and widespread dissemination of this intervention. Although there is some evidence that high-quality ADHD care is associated with improved patient outcomes,7,28,29
additional research is necessary to establish that adoption of this intervention would lead to better patient outcomes and patient satisfaction. Finally, future research must address the cost-effectiveness of this intervention by comparing the potential benefits of this intervention, with respect to clinical utility to pediatricians and potentially improved patient outcomes, with the cost of implementing this intervention (ie, physician/staff time, technical support, and server hosting).