PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Pediatrics. Author manuscript; available in PMC 2006 July 25.
Published in final edited form as:
PMCID: PMC1519417
NIHMSID: NIHMS11140

Implementing the American Academy of Pediatrics Attention-Deficit/Hyperactivity Disorder Diagnostic Guidelines in Primary Care Settings

Laurel K. Leslie, MD,*|| Jill Weckerly, PhD,*§ Dena Plemmons, PhD,* John Landsverk, PhD,* and Sarita Eastman, MD#

Abstract

Objectives

To evaluate the feasibility of the San Diego Attention-Deficit/Hyperactivity Disorder Project (SANDAP) protocol, a pediatric community-initiated quality improvement effort to foster implementation of the American Academy of Pediatrics (AAP) attention-deficit/hyperactivity disorder (ADHD) diagnostic guidelines, and to identify any additional barriers to providing evidence-based ADHD evaluative care.

Methods

Seven research-naïve primary care offices in the San Diego area were recruited to participate. Offices were trained in the SANDAP protocol, which included 1) physician education, 2) a standardized assessment packet for parents and teachers, 3) an ADHD coordinator to assist in collection and collation of the assessment packet components, 4) educational materials for clinicians, parents, and teachers, in the form of handouts and a website, and 5) flowcharts delineating local paths for referral to medical subspecialists, mental health practitioners, and school-based professionals. The assessment packet included the parent and teacher versions of the Vanderbilt ADHD Diagnostic Rating Scales. In this study, we chose a conservative interpretation of the AAP ADHD guidelines for diagnosing ADHD, requiring that a child met criteria for ADHD on both the parent and teacher rating scales. A mixed-method analytic strategy was used to address feasibility and barriers, including quantitative surveys with parents and teachers and qualitative debriefing sessions conducted an average of 3 times per year with pediatricians and office staff members.

Results

Between December 2000 and April 2003, 159 children were consecutively enrolled for evaluation of school and/or behavioral problems. Clinically, only 44% of the children met criteria for ADHD on both the parent and teacher scales, and 73.5% of those children were categorized as having the combined subtype. More than 40% of the subjects demonstrated discrepant results on the Vanderbilt scales, with only the parent or teacher endorsing sufficient symptoms to meet the criteria of the Diagnostic and Statistical Manual of Mental Disorders, 4th ed. Other mental health and learning problems were common in the sample; 58.5% of subjects met screening criteria for oppositional defiant disorder/conduct disorder, 32.7% met screening criteria for anxiety/depression, and approximately one-third had an active individualized education program in place or had received an individualized education program in the past. On evaluation, the SANDAP protocol was acceptable and feasible for all stakeholders. However, additional barriers to implementing the AAP ADHD guidelines were identified, including 1) limited information in the guidelines regarding the use of specific ADHD rating scales, the evaluation and treatment of children with discrepant and/or negative results, and the indications for psychologic evaluation of learning problems, 2) families’ need for education regarding ADHD and support, 3) characteristics of physical health and mental health plans that limited care for children with ADHD, and 4) limited knowledge and use of potential community resources.

Conclusions

Our results indicate that children presenting for evaluation of possible ADHD in primary care offices have complex clinical characteristics. Providers need mechanisms for implementing the ADHD diagnostic guidelines that address the physician education and delivery system design aspects of care that were developed in the SANDAP protocol. Additional barriers were also identified. Careful attention to these factors will be necessary to ensure the sustained provision of quality care for children with ADHD in primary care settings.

Keywords: pediatrics, primary care, attention-deficit/hyperactivity disorder, guidelines
ABBREVIATIONS: DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, 4th ed.; AAP, American Academy of Pediatrics; ADHD, attention-deficit/hyperactivity disorder; NICHQ, National Initiative for Children’s Healthcare Quality; PCP, primary care provider; SANDAP, San Diego ADHD Project; AC, ADHD coordinator; ODD, oppositional defiant disorder; CD, conduct disorder

Attention-deficit/hyperactivity disorder (ADHD) is one of the most prevalent mental health problems of childhood and adolescence, affecting an estimated 3% to 5% of school-aged children in subspecialty mental health settings1 and 4% to 12% of children in community settings.2 ADHD can significantly affect children’s functioning and predispose children to psychiatric and social pathologic conditions in later life.3 Although youths with ADHD constitute up to 50% of the child psychiatry clinic population,4 at least 50% of patients with ADHD are not treated for ADHD by a mental health specialist.57 This is not surprising, given the high prevalence of ADHD, families’ perceptions of the role of primary care providers (PCPs), and the insufficient numbers of pediatric mental health professionals.8,9 These factors, however, suggest that pediatricians will be called on to treat large proportions of children and adolescents with ADHD and thus will need to be skilled in the diagnosis and treatment of ADHD.

Although controversies exist regarding the diagnosis of ADHD, the diagnostic criteria have been extensively studied and promoted in the past decade. In 1994, the American Psychiatric Association used an expert consensus technique to revise the Diagnostic and Statistical Manual of Mental Disorders, 4th ed. (DSM-IV), diagnostic criteria for ADHD.1 The Agency for Health Care Research and Quality recently published an evidence-based report on the identification of ADHD2 and recommended strict adherence to the current diagnostic criteria. In DSM-IV, the diagnosis of ADHD requires 1) a sufficient number of 18 behavioral symptoms related to ADHD, persisting for at least 6 months (core A criteria), 2) symptom onset before 7 years of age (core B criteria), 3) clear evidence of functional impairment in at least 2 different settings (core C and D criteria), and 4) no other mental, medical, or environmental explanation for the symptoms (core E criteria).

These research efforts, in combination with national concern regarding the increased use of stimulants in the past decade, prompted the American Academy of Pediatrics (AAP) to publish guidelines for the diagnosis of ADHD in primary care settings in 2000.10 Specifically, the guidelines recommend that PCPs 1) initiate evaluations for ADHD among school-aged children who present with behavioral or school problems, 2) use DSM-IV criteria to make a diagnosis of ADHD, 3) gather detailed information from both parents and teachers regarding the DSM-IV criteria, 4) assess coexisting mental health and learning problems, and 5) order other diagnostic tests only as indicated on the basis of history or physical findings.

Continuing medical education and practice patterns research suggests that many PCPs face significant barriers in preparing to adopt the published guidelines. First, many PCPs may have limited knowledge regarding the diagnosis of ADHD. It was not until 1997 that the Pediatric Residency Review Committee required a formal 1-month rotation in behavioral and developmental pediatrics; no rotation in psychiatry is currently required (E. Coss, personal communication, 2001). This lack of formalized training means that PCPs exhibit variations in their knowledge of ADHD. Consequently, they may be less likely to diagnose ADHD than are mental health specialists1113; alternatively, they may overdiagnose ADHD.14 Second, many pediatricians do not use DSM-IV criteria to diagnose ADHD. A recent study of >400 clinicians found that <40% used the DSM-IV criteria to diagnose ADHD and <37% used behavior-rating scales to assess patients.15 Third, data indicate difficulty in collecting information from a second source (such as the school), as recommended in the AAP guidelines.1619 Fourth, PCPs may be less likely to recognize coexisting mental health conditions, given their lack of formal training.14 Fifth, lack of adequate time to make a diagnosis and discuss treatment options is a barrier. PCPs have contended that comprehensive assessments for ADHD can be time-consuming and require significant staff time to collect and collate information from ≥2 sources. Published protocols, generated primarily from referral centers, suggest that the time necessary for an initial evaluation for ADHD ranges from 2.5 hours to 8 hours.20 Even a streamlined approach, such as that recommended by Block,21 calls for two 30- to 45-minute visits. These extended times contrast sharply with the 7- to 20-minute pediatric visits common in the San Diego area (G. Nathan, personal communication, 2000).

The research presented here capitalized on the presence of the San Diego ADHD Project (SANDAP), a naturally occurring quality improvement effort initiated in the practicing pediatric community in San Diego County to implement the AAP ADHD diagnostic guidelines in primary care. The research presented addressed the following 3 questions. 1) What are the clinical characteristics of children presenting for an evaluation for ADHD in the participating primary care offices, and how do these characteristics compare with those of children from other clinical and community samples? 2) Is the SANDAP protocol feasible and acceptable to doctors, office staff members, parents, and teachers? 3) Are there any additional barriers to implementing the AAP guidelines?

METHODS

Development of the SANDAP Protocol

The publication of the AAP diagnostic guidelines spurred interest on the part of several San Diego-based pediatric groups and managed behavioral health organizations in developing mechanisms for managing ADHD in primary care settings. In response, San Diego Children’s Hospital, under the guidance of Dr Paul Kurtin, Vice President for Clinical Innovations, convened a panel of local pediatricians and ADHD experts to address the implementation of the guidelines in primary care settings. The panel first held 3 focus groups (n = 3–7 in each group) with pediatrician leaders from different health plans, to identify perceived barriers to implementation of the AAP guidelines on ADHD. Five barriers to implementing the AAP guidelines were identified in the focus groups, ie, 1) limited knowledge regarding the management of ADHD and its possible coexisting conditions, 2) lack of PCP time to interview children and families and contact school personnel, 3) lack of office staff time to assist PCPs, 4) lack of reimbursement for time and effort, and 5) fragmented referral systems for children with coexisting learning problems and/or mental health conditions. These barriers were not surprising and mirrored barriers identified by primary care internists with respect to the management of depression in primary care settings.2227 It should be noted that the time and reimbursement concerns were not insignificant; a report published concurrently by the California Medical Association stated that >70% of primary care offices were operating at a fiscal deficit, given reimbursement rates under managed care.28

On the basis of this information, a group of pediatricians, psychologists, and psychiatrists affiliated with the Department of Pediatrics, University of California, San Diego, and San Diego Children’s Hospital and Health Center initiated the SANDAP to address these issues. The SANDAP protocol was crafted with 5 basic components, ie, 1) PCP education, 2) a standardized assessment packet, with complete history forms for both parents and teachers and ADHD rating scales (see below), 3) ancillary personnel to assist in the collection and collation of the assessment packet components, 4) informational handouts and a website (www.sandiegoadhd.org) for clinicians, parents, and teachers, and 5) flowcharts with guidelines regarding local pathways for referral to mental health practitioners, pediatric subspecialists, and school professionals.

Two components of the SANDAP deserve additional description, namely, the use of a standardized assessment packet and the use of ancillary personnel. We chose to use a questionnaire-based assessment packet to solicit historical information before the initial visit for ADHD assessment, to provide PCPs with information efficiently. Questionnaires have a long history of use in pediatrics and have been described as an efficient mechanism for gathering standardized background information, improving the comprehensiveness of medical histories, assisting in the diagnostic process, and providing baseline data for follow-up visits.29 Questionnaires may also be a more comfortable way for parents to share mental health and social concerns, compared with direct interviews.30,31 Central to the SANDAP was the ADHD coordinator (AC), who ensured that families and teachers completed the initial assessment packet and returned it to the office of the PCP (Fig 1). The use of ancillary personnel has been suggested in the literature as a mechanism for improving care for children with chronic conditions32 and was perceived as an efficient mechanism for addressing PCP and office staff member concerns regarding time and reimbursement.

Fig 1
SANDAP flowchart.

After receipt of San Diego Children’s Hospital institutional review board approval, the protocol was pilot-tested by 2 pediatricians in their practices for 4 months. One pediatrician served middle to upper-middle class clientele; the other was located in a community health center. Neither client group found the instruments troublesome. Components of the assessment packet were revised on the basis of feedback from the 2 pediatricians, as well as pilot families and teachers.

Sample Participants

After pilot testing of the protocol and appropriate revision, 7 offices in the San Diego area were selected to participate in the research study. The 7 offices were chosen from 3 different health care networks in the San Diego area, to capture variability in child and family sociodemographic features and in the organization of health care delivery systems. None of the offices had previously participated in practice-based research, allowing evaluation of the feasibility of the ADHD quality improvement initiative in research-naïve settings. The 7 offices included 4 private offices and 3 community clinics. Three of the private offices were part of an independent practice association, and the majority of children were privately insured. The practices were operated as group practices containing independent practitioners. Mental health services for their clients were primarily provided by privately managed behavioral health organizations. The fourth private practice office was affiliated with an integrated delivery system, treated children with private insurance, and provided mental health services within the system. The 3 community health clinics cared primarily for children with public insurance or no insurance. Mental health services were brokered through a single behavioral health organization.

A total of 16 pediatricians from the 7 offices participated in the study. The average age range of the pediatricians was 41 to 50 years, and more than one-half were female. The pediatricians had been practicing for 11 to 15 years. The numbers of children and adolescents examined in an average day were 20 to 39; however, several pediatricians (25%) reported treating as many as 40 to 60 per day. Before participating in the study, >33% referred all children with possible ADHD for an ADHD evaluation. Four other pediatricians in the offices were not included in the study because they joined the practices after the initial training in the SANDAP protocol had occurred.

Procedures

All 7 practices were asked to consecutively enroll children after the initial physician education sessions. Inclusion criteria for child enrollees were 1) age of 5 to 17 years (children 4 years of age could be enrolled if in elementary school), 2) previously not diagnosed with ADHD or not treated for ADHD in the past 6 months by the current PCP, 3) child and parent willing to participate in the protocol, as determined during receipt of informed consent, and 4) English or Spanish speaking. The exclusion of families whose primary language was not English or Spanish was necessary because of the large numbers of Asian and African dialects spoken in the San Diego area.

Office staff members gave assessment packets to any child identified as having any attentional/hyperactive problems and/or school problems, during telephone intake or an office visit. Office staff members then faxed the child’s name and contact information to the AC at the central SANDAP office. Assessment data were collected by the AC and returned to the PCP. Office staff members then confirmed that the family had a follow-up appointment scheduled with the PCP for evaluation.

As approved by the Children’s Hospital institutional review board, the AC approached families regarding participation in the research study after the assessment packet had been returned to the pediatric office. If a family expressed interest, then the AC gave the family’s identifying material to a trained research assistant, who obtained consent for the child and family to participate in the study. Data from the assessment packet were then made available to the research staff, and a trained research assistant conducted a semistructured telephone interview with the family. If a family chose not to participate, then their materials were sent by the AC to the participating pediatrician but were not released to the research staff.

Data were collected for enrolled children from December 2000 through April 2003. Methods to promote PCP and office staff member adherence to the protocol during this time included 1) ongoing, periodic, academic detailing with PCPs and staff members regarding the SANDAP protocol,33,34 2) inclusion of PCPs in the development of solutions,35 and 3) administrative interventions, specifically, support by the AC.25,3537 Academic detailing took place during regularly scheduled weekly staff meetings for the community clinics and during breakfast or lunch meetings for the private offices.

Measures

Approach

This research effort used a mixed-method approach. Quantitative data were collected in the assessment packet and through parent and teacher surveys after completion of the packet. We also used qualitative methods for this research. Qualitative research methods have been described as valuable in “illuminating the experience and interpretation of events by actors with widely differing stakes and roles.”38(p A variety of qualitative research techniques are available for use in health services research; the current study used open-ended interviewing.39 Open-ended interviews were conducted with PCPs and office staff members in debriefing sessions held throughout the study period. In addition, the parent surveys included open-ended questions. Specific measures are detailed below.

Assessment Packet

Detailed birth, medical, developmental, behavioral, family, school, and social histories were collected through the standardized assessment, which contained questions in both a dichotomous (yes/no) format and a Likert-scale format. These forms are available in the public domain at www.sandiegoadhd.org.

The packet also included the Vanderbilt ADHD Diagnostic Rating Scales (parent and teacher versions) to quantify ADHD symptoms (α values ranging from .80 to .95).40 The Vanderbilt scales use a 4-point Likert rating, in which a respondent notes whether 18 specific behavior symptoms of ADHD occur rarely, sometimes, often, or very often. A child meets behavioral criteria for the inattentive subtype if ≥6 of the 9 inattentive items are scored as often or very often by the respondent, for the hyperactive/impulsive subtype if ≥6 of the 9 hyperactive/impulsive items are scored, and for the combined subtype if ≥6 of each of the 2 sets of items are scored. The Vanderbilt scales also address functional impairment with respect to relationships with family and peers (parent version), classroom behavior (teacher version), and academic performance (parent and teacher versions). In addition, the Vanderbilt scales contain items to screen for common coexisting mental health conditions (oppositional defiant disorder [ODD], conduct disorder [CD], anxiety, and depression).

For these analyses, we implemented a conservative framework for the establishment of an ADHD diagnosis, in which both a respondent from home and a respondent from school needed to endorse ≥6 of the 9 behavioral symptom criteria in 1 or both of the 2 symptom clusters (inattention or hyperactivity/impulsivity) to meet behavioral criteria for that ADHD subtype. In addition, at least 1 functional impairment item needed to be endorsed by the respondent, as specified in the Vanderbilt scale scoring directions. We designated a diagnosis of ADHD-combined subtype if both the parent and teacher endorsed symptoms required for this subtype or if the parent and teacher endorsed different subtypes.

Debriefing Sessions

SANDAP staff members conducted debriefing sessions with PCPs and staff members regarding the SANDAP protocol approximately 3 times per year. These debriefing sessions were designed as semistructured interviews with available PCPs and staff members. Interviews elicited details of personal experiences with the SANDAP protocols, including distribution of the assessment packets in the office, formatting of the packet materials for ease of review after return to the office, use of the AC, diagnostic and treatment management issues that arose (see below), and any identified barriers to implementation of the AAP guidelines in primary care settings. Debriefing sessions were held during routine staff meetings or at lunch. The exigencies of practice meant that PCPs and staff members were often called out of these meetings or were not able to attend consistently, particularly if the offices did not have routine meetings.

Parent Interviews

After return of the assessment packet, parents were interviewed by telephone regarding acceptability of the packet, use of the AC, time needed for completion of the packet, ease of obtaining teacher information requested by the packet, and time to follow-up assessment with the child’s PCP after completion of the packet. Questions were modified from the face-validated Parent Consumer Satisfaction Scale.41 Each interview ended with the open-ended question, “If your health plan were to start a program to help children with behavioral and attentional difficulties and their families, what would you want it to provide?”

Teacher Survey

The teacher survey was faxed to teachers after receipt of the teacher portion of the packet. Questions investigated the time needed to complete the packet, use of the AC, and satisfaction with the packet. Satisfaction questions were modeled after the Parent Consumer Satisfaction Scale.41

Analyses

The primary objectives of the research presented here were to answer the following questions. 1) What are the clinical characteristics of children presenting for evaluation for possible ADHD? 2) Is the SANDAP protocol feasible and acceptable to stakeholders? 3) Are there any additional barriers to implementation of the AAP guidelines?

Accomplishment of the first objective relied primarily on descriptive analyses of quantitative data, conducted by using SPSS 9.0 software (SPSS, Inc, Chicago, IL). Statistical methods used included t tests and χ2 tests. Qualitative data were collected to address the remaining research questions and consisted of narratives from the debriefing sessions conducted with PCPs and office staff members and open-ended questions in the follow-up parent interviews. At least 2 members of the research team reviewed data from each of these sources, to identify preliminary themes regarding the management of ADHD in primary care settings. Preliminary themes were brought back to the entire research team and compared for the different sources. We thus returned repeatedly to the primary data in an iterative process, refining themes and comparing information from different sources. Finalized themes were then categorized by the research team within a published conceptual framework for child health care (see Discussion).

RESULTS

Child Characteristics

Two hundred thirty-five children were initially referred to the SANDAP AC from the practices. Of that group, 16 children were ineligible for the study because of their ages (<4 years) or because they were already receiving psychotropic medication. Ten families moved immediately after their initial referrals and were not included in the study. Seven parents refused to allow their children to participate in the study. An additional 43 did not return packets, and those children were not included in the current study. This left a remaining sample size of 159 children.

The mean age of the sample was 8.2 years (median: 7.0 years; SD: 2.7 years). Seventy-five percent of the children were male. With respect to race/ethnicity, 68.6% were white American, 3.1% were African American, 20.1% were Hispanic American, and 8.2% were reported as other (primarily biracial).

The results regarding parent- and teacher-reported ADHD symptoms are presented in Table 1. Using the restrictive diagnostic parameters described above, 11.5% of children had symptoms that met the criteria for the ADHD-inattentive subtype, 0.0% for the ADHD-hyperactive subtype, and 31.8% for the combined subtype. Interestingly, 40.8% of children had discrepant responses from parents and teachers, with only the parent or teacher endorsing sufficient numbers of symptoms of ADHD to meet the DSM-IV criteria. Another 15.9% of children did not have patterns of symptoms that met the criteria for ADHD on either the parent or teacher forms.

TABLE 1
Proportions of Children Presenting With Common Comorbidities, According to ADHD Subtype

With respect to coexisting conditions, Table 1 presents results for several commonly coexisting conditions screened for on the Vanderbilt scales, including ODD/CD and anxiety/depression. Of the total SANDAP sample, 58.5% of the children met the screening criteria for ODD/CD on either the parent or teacher Vanderbilt forms. We were unable to differentiate the children who met the screening criteria for ODD on the Vanderbilt scales from those who met the criteria for CD, because the Vanderbilt scales combine ODD and CD symptoms on the teacher form. A smaller proportion (32.7%) met the screening criteria for anxiety/depression, according to either parent or teacher reports. The assessment instrument also screened for suicidality; on a 4-point Likert scale, 11.4% of parents and/or teachers responded that the child had commented often or very often that “I wish I were dead.” In terms of school issues, 98.0% of the total sample had school problems endorsed by the parent and/or teacher. Another 32.1% received special education services or had participated in an individualized education program in the past.

Feasibility of the SANDAP Process

Feasibility of the protocol was measured from an administrative perspective and from the perspective of stakeholders (PCPs, office staff members, families, and teachers). Administratively, the cost of the packet (in terms of paper copies) was $1.50. The cost of the AC’s time included an average of 41 minutes to score each packet (median: 35 minutes; SD: 16 minutes) and an average of 20 minutes of tracking time (median: 15 minutes; SD: 18 minutes) per child, to contact parents and/or teachers to prompt them to return their portions of the packet. Referrals during the research period averaged 0.43 referrals per physician per month (range: 0–3 referrals per physician per month), with peak months being October through December and February through June. These findings suggest that a trained, full-time AC could complete ~35 assessment packets per week.

Feasibility from the perspective of PCPs and staff members was assessed in the semistructured debriefing sessions. PCPs and office staff members reported that the protocol was not difficult to implement in the office setting. Office staff members noted that it was important to designate specific staff members to communicate with the AC, to ensure consistent care. Office staff members highlighted their critical role in identifying families to receive a packet, providing referral information to the AC, letting the AC know when the supplies of packets and educational handouts were getting low, and ensuring that children received follow-up appointments when the packet was completed. PCPs stated that there was a definite need for a designated staff person such as the AC to collect and collate information from parents and teachers and that the identification packet substantially saved PCP and office staff time and provided a more complete understanding of patients.

Parents commented that it required an average of 59.2 minutes to complete the packet (median: 40.0 minutes; SD: 55.3 minutes), and 81.7% thought that the forms were “easy/very easy” to complete. Almost 90% of parents considered it worth their effort to complete the packet. In open-ended comments, parents expressed satisfaction with the amount of material collected in the packet; 1 parent commented that the doctor was “taking seriously” her child’s condition.

Teacher surveys were completed for 116 children (75%). Teachers found that it required an average of 48 minutes to complete the packet (median: 45 minutes; SD: 34.2 minutes), and 60.9% thought that the forms were “easy/very easy” to complete. More than 95% considered it worth their time and effort to complete the packet.

Additional Barriers

Additional barriers were elicited from physicians and office staff members during the quarterly debriefing sessions. Common themes included 1) limited information in the AAP guidelines regarding the diagnostic process, 2) challenges of “opening a Pandora’s box” in the diagnostic process, with respect to coexisting conditions, 3) difficulty implementing treatment in the primary care setting, 4) issues inherent in collaboration with schools, 5) lack of time, and 6) problems accessing mental health services, such as behavioral therapy, particularly for offices not in an integrated delivery system.

The first theme addressed limited information regarding the diagnostic process in the AAP guidelines. PCPs requested more specific details regarding 1) the strengths and weaknesses of available ADHD rating scales, 2) assessment and management of coexisting conditions, and 3) indications for a psychoeducational evaluation. PCPs also requested guidance regarding diagnostic and management strategies for the 41% of the sample with discrepant results between parents and teachers.

The second theme involved varying interest displayed by the PCP participants regarding psychosocial issues and personal willingness to open a Pandora’s box. The presence of coexisting mental health and/or educational conditions and the lack of clarity concerning referral pathways for evaluation and treatment of these conditions compounded the initial reluctance of some PCPs to care for children with ADHD. Some adopted a utilitarian approach to screening for coexisting mental health and learning problems, stating, “I don’t want something identified if I don’t have the resources to do something about it,” thus highlighting the importance of clear referral paths for PCPs caring for children with ADHD.

The third theme for some of the PCPs was difficulty in providing treatment in the primary care setting. For some PCPs, this was a function of limited knowledge regarding medication treatment (specifically, titration of stimulant medications). Others cited productivity pressures that subtly reinforced the use of fewer follow-up visits. In addition, others described inequities in workloads for providers who wrote stimulant prescriptions, because California requires the use of triplicate prescriptions for stimulant medications.

The fourth theme concerned the difficulties inherent in working collaboratively with schools regarding both diagnosis and treatment. With respect to the diagnostic process, determining when and how to request from the school evaluations of possible learning disabilities was particularly challenging. Many PCPs voiced discomfort regarding establishing a diagnosis of ADHD without a complete psychoeducational evaluation but were also concerned about overwhelming school staff. With respect to treatment, PCPs discussed problems with obtaining information from schools regarding the effects of treatment.

The fifth theme addressed the difficulties of evaluating a child for ADHD in an acute care setting modeled on 7- to 15-minute visits. PCPs discussed less obvious time requirements, in addition to making the diagnosis and providing medication, including a family’s need for time to process a diagnosis and their requests for information and education regarding available treatments. PCPs were, for the most part, unaware of community groups that might be able to partner with offices to provide such information to families. PCPs varied in their opinions regarding extending the time for visits; some developed strategies for managing longer visits by scheduling families at the end of the day after “regular hours,” and others split a 1-hour visit into several shorter consecutive visits. Others were unable to make these types of changes within the structure and reimbursement system of their health care facilities.

The last theme addressed the fragmentation between health and mental health care systems in the San Diego region and the lack of clarity regarding referral paths. PCPs stressed that the “carving out” of mental health services, whereby mental health services were contracted to a separate fiscal institution, negatively affected their ability to access quality mental health services from a provider they knew and trusted. Several referred patients to neurology services instead of mental health services because of the ease of utilization review within the physical health plan, compared with accessing carved-out mental health services. One commented on the impersonal quality of a toll-free line for both providers and clients to use to access services. PCPs also wanted clear information regarding who in the community provided behavioral treatment. These issues of fragmentation were salient in both the community clinics and the independent practice association offices but not in the office in the integrated delivery system; the integrated delivery system had a defined panel of mental health providers and had even developed a program for parents of children with ADHD that was being coordinated by social workers.

Themes from parents paralleled many of the themes generated by the offices. Open-ended parental comments from the telephone interviews indicated the need for 1) more information regarding ADHD, the results of the child’s evaluation, medication and why it might be prescribed, and other available treatments (52%), 2) counseling (35.8%) or support groups (25.8%) that could help family members better cope with the disorder, 3) child-focused programming to help the child cope with ADHD (17.0%), and 4) help in working with the child’s school to access help with behavioral strategies, homework planning, and medication (13.2%).

DISCUSSION

Overall Findings

This article provides a detailed examination of the attempts of 7 primary care offices to implement the AAP ADHD diagnostic guidelines. Our first question focused on the clinical characteristics of children who presented for evaluation at the 7 participating primary care offices. In brief, the 159 children who presented to these offices were similar in many ways to cohorts with ADHD that were previously discussed in the literature. The majority of children in the sample were in elementary school at the time of presentation for evaluation. More than 75% were male, mirroring national trends with respect to ADHD prevalence (male/female ratios of 9:1 in clinical samples and 4:1 in epidemiologic samples).4 The sample included children from different racial/ethnic groups, with the majority being white American.

Clinically, 44% of the children met stringent criteria for ADHD, with the majority meeting criteria for the combined subtype and a smaller proportion meeting criteria for the inattentive subtype. The finding of limited numbers of children meeting criteria for the hyperactive subtype is consistent with results reported by Wolraich et al42 regarding the prevalence of different subtypes of DSM-IV-determined ADHD among school-aged children. Importantly, more than one-half the children did not meet our conservative criteria for ADHD, with the majority demonstrating discrepant results between home and school respondents. The rate of discrepant results in this study is not surprising and is comparable to the rates reported by the Multimodal Treatment Study of the ADHD Collaborative Research Group43 and observed in an earlier study on children being evaluated for ADHD.44

The question of how clinicians should interpret discrepant results remains unresolved. Some might argue that a child with discrepant results should be diagnosed as having “situational” ADHD (eg, symptoms present in only 1 setting), which would increase prevalence rates for ADHD in this study to 84.1%. However, debate continues in the literature regarding whether situational ADHD exists and, if it does, whether it is a distinct disorder, compared with “pervasive” ADHD.4548 Some clinicians would argue that discrepant results are an important indicator of a potential alternative or coexisting problem and that the diagnosis of situational ADHD should not be readily given. The guidelines, as well as the recently published book on ADHD by the AAP,49 do not address the issue of situational ADHD. The controversies in the literature and the comments of pediatricians participating in this study suggest that an evidence-based review of this topic is needed.

We examined rates of symptoms of common coexisting conditions by using the Vanderbilt scales. Of our sample of children, 58.5% met screening criteria for ODD/CD and 32.7% met screening criteria for anxiety/depression. Because of the scoring procedures of the Vanderbilt scales, we were not able to compare our rates directly with rates published in the literature. However, our rates are generally within the ranges reported. Wolraich et al42 determined that, for children with ADHD in a school-aged general population sample, rates of coexisting conditions were 26.5% for ODD, 9.6% for CD, and 20.8% for anxiety/depression. Rates in a recently published Agency for Health Care Policy and Research meta-analysis2 were 35.2% for ODD, 25.7% for CD, 25.8% for anxiety, and 18.2% for depression. Our proportion of children receiving special education services was comparable to values reported by August and Garfinkel.45

These findings regarding the characteristics of children examined in primary care offices have several important implications. Our results confirm that children presenting for ADHD evaluations in primary care offices have high rates of mental health conditions and/or learning problems. PCPs need mechanisms for evaluating children for these problems or easy access to referral services. These findings also raise an important issue not discussed in the AAP guidelines, ie, that the majority of children presenting with behavioral and/or school problems do not meet stringent criteria for ADHD and require additional evaluation. Therefore, PCPs need strategies for evaluating and treating children with negative and/or discrepant results. Strategies must take into account the PCP’s level of comfort in dealing with behavioral and developmental problems and the availability of additional mental health and educational resources in the community.

Regarding the logistics of the SANDAP protocol, we found that the protocol was acceptable and feasible for all stakeholders in primary care offices. In addition, the number of referrals per pediatrician was relatively small, and SANDAP provided an efficient mechanism for collating information from different offices and health plans. However, the AC position was grant-funded under the SANDAP and was not self-sustaining after completion of this quality improvement initiative, except in the community clinics that obtained ongoing community grant funding.

Additional barriers to implementing the AAP ADHD guidelines were identified by PCPs, office staff members, and families and can be examined within the context of the chronic care model, which was developed by Wagner et al.50 and then modified for pediatric populations by the National Initiative for Children’s Healthcare Quality (NICHQ) (Fig 2) (C. Homer, personal communication, 2001). Table 2 presents the components of the NICHQ Care Model for Child Health, barriers identified by PCPs, office staff members, and families, and potential solutions. Each of these areas is discussed below.

Fig 2
NICHQ Care Model for Child Health.
TABLE 2
ADHD Care Within the Context of the NICHQ Care Model for Child Health

Clinical Implications

Decision Support

Decision support tools, such as educational support for physicians, remain critical for PCPs, given the variability of training with respect to behavioral-developmental pediatrics. These tools must be available in multiple modalities, including traditional continuing medical education programs, case reviews conducted by experts in the field on periodic bases, and self-instructional programs in paper-, audio-, video-, or interactive computer-based formats. The content of these tools must be specific, addressing practical procedural issues in implementing the AAP guidelines. Topics must include the DSM-IV criteria for ADHD, available ADHD rating scales and their advantages and disadvantages, how to handle negative and/or discrepant results on ADHD rating scales, and mechanisms for evaluating children for common coexisting mental health conditions and learning disabilities.

Delivery System Design

The research literature is replete with examples of how education alone does not change practitioner behavior but serves a preparatory function of educating providers regarding the need to change.5155 Processes for structured diagnostic and follow-up care have been found to be critical in reengineering systems for the treatment of mental health disorders in adult primary care settings.23 Implementing the AAP ADHD guidelines in primary care settings thus necessitates careful attention to delivery system design, with a specific focus on what tools will be used to evaluate a child for ADHD and the mechanisms by which an office will implement the evaluation process. Several tools for delivery system design planning are already available for use by PCPs. In 2002, the AAP and NICHQ jointly released a toolkit for the management of ADHD in primary care settings, which includes the Vanderbilt rating scales and is available via the Internet (www.nichq.org). The AAP also released an Internet-based learning tool, which was developed with assistance from NICHQ and other national experts. A module of the Education in Quality Improvement for Pediatric Practice program, this tool aims to aid physicians in establishing an effective delivery system for ADHD care in their particular office settings (www.eqipp.org).

The role of a care coordinator or case manager as part of delivery system design deserves additional discussion. In recent years, a variety of allied health care professionals, including child psychologists, social workers, nurse practitioners, and physician assistants, have partnered with primary care pediatricians in the care of children. These professionals have partnered successfully with PCPs in several models for management of mental health conditions in pediatric and internal medicine settings.23,56 They hold great promise for improving the diagnosis and treatment of children with ADHD, by providing critical assistance in the diagnostic process, medication titration, coordination of behavioral modification programs, and education of parents and children regarding ADHD and available community resources.

With respect to the use of affiliated professionals, 2 findings from our study bear repeating. First, the cost of affiliated personnel was not insubstantial and was not self-sustaining without grant funding. Many of the cost-containment strategies proposed in the past decade in the commercial and public insurance sectors have limited available funds for coordinating positions. The separation of mental health care from physical health care also discourages creative funding mechanisms that integrate primary care and mental health settings. Second, our study noted a relatively low number of referrals per physician per month. This suggests that it may be inefficient and too costly for single offices to fund an ADHD care coordinator or case manager. Solutions include having ADHD-specific case managers who are shared among offices or health plans and having professionals in office settings who can serve these functions for multiple chronic disorders.

Treatment of children with ADHD often requires consultation with both mental health and school professionals. The creative use of information systems may help improve communication among health, mental health, and school professionals.

Family Education and Support and Community Resources

The NICHQ Care Model also highlights the importance of family education and support. Our findings indicate that physicians, office staff members, and families all thought that parents needed access to up-to-date information, as well as opportunities to process the information accessed. Creative strategies worth investigating might include the use of handouts and/or websites, such as the materials developed for the SANDAP program. Group visits, which were initially suggested by Dodds et al57 for well-child visits, might potentially address parental needs for education and support. Additional strategies include educational programs cosponsored with mental health professionals or community organizations such as Children and Adults with ADD or the Learning Disabilities Association, in which PCPs partner with community resources (the fourth component in the NICHQ model).

Health System Design and Clinical Information Systems

The fifth component of the NICHQ Care Model, health system design, may be one of the most important components to address and yet the most difficult to implement. Changes in health system design by definition fall outside the realm of individual clinical practitioners or offices and require significant policy and advocacy efforts at the local, state, and national levels. However, without attention to the health care delivery system, individual physicians may not be able to implement the guidelines in their office settings. For example, as part of the SANDAP, we explored the possibilities of sustainable funding for parent education sessions, a triage clinic model, and Internet-accessible packets and standardized forms with several local health plans and managed behavioral health organizations. Unfortunately, in autumn of 2002, several commercial physical health managed care plans discontinued ADHD as a covered benefit in their contracts; consequently, efforts to develop these strategies lost momentum as PCPs reverted to referring children with possible ADHD to behavioral health organizations.

The past decade has seen an increase in administrative and financial incentives and barriers imparted by third-party payers that affect clinician treatment plans, similar to the example given above. Effects in pediatrics include capitation, larger panel sizes, constraints on referrals, and disincentives to treating children with chronic needs.58 These programmatic changes in the health care system threaten the ability to develop multifaceted longitudinal approaches to improving care by PCPs for children with mental health problems. These transformations also affect the incorporation of allied health care professionals within primary care settings.59

Several critical issues must be addressed by health care plans, insurance companies, and professional organizations that advocate regarding the mental health needs of children. First, some consensus must be reached regarding the role of PCPs in the diagnosis and psychopharmacologic treatment of ADHD among children. If PCPs are treating these children, then health plans need to ensure that PCPs are appropriately trained and reimbursed to provide these services. It is important to consider the availability of informal subspecialist consultative services, as well as ready access to clinicians trained in behavioral modification. If PCPs do not care for these children, then sufficient numbers of mental health professionals trained in the evidence-based diagnosis and treatment of ADHD must be available for families. Second, creative, efficient, and effective mechanisms for organizing behavioral health care services that span primary care and specialty services must be developed. Third, we need to reconsider the allocation and management of fiscal resources dedicated to mental health conditions.60 Decisions may differ depending on the type of health care system and the characteristics of the geographic location, including available provider supply and the number of inhabitants per square mile.61 Evaluation of the effectiveness of these health system design changes would require the collection of sophisticated utilization data and charges by using clinical information systems (the sixth component of the NICHQ Care Model).

Limitations

This study was exploratory in nature, and its limitations are related to that feature. The study used a convenience sample of offices, with consecutive sampling of children, to test the feasibility of an intervention and did not use a randomized design to test the intervention. The research presented here did not explore child, family, or teacher characteristics that might affect implementation of the guidelines. We were also limited in the numbers of participating pediatricians and were unable to examine physician characteristics that might affect implementation of the guidelines in primary care settings, such as attitudes62 and/or interest in mental health.63 However, the use of a limited number of practices within 3 networks was an effective strategy for a feasibility study and allowed preliminary exploration of provider and practice characteristics associated with the acceptability and ease of implementation of the SANDAP protocol. The use of both quantitative and qualitative methods allowed the generation of more extensive data than would have been possible with a single method.

CONCLUSIONS

To our knowledge, this study reports the first attempt to implement the AAP ADHD diagnostic guidelines in research-naïve primary care offices. Our results suggest that additional steps must be taken to allow implementation of the AAP guidelines in primary care setting. These findings support the conclusions of 3 recent publications, ie, Blueprint for Change: Research on Child and Adolescent Mental Health,64 Report of the Surgeon General’s Conference on Children’s Mental Health: A National Action Agenda,65 and the review by Hoagwood et al.66 Those authors stated that efficacy studies are often deployed into real-world service settings without taking into account the exigencies of clinic-based or community-based care and that the implementation of evidence-based practice in these settings may fail unless such factors are addressed in the long-term design of health care delivery. Careful attention to factors in addition to provider education and the distribution of tools will be necessary to ensure the sustained provision of quality care for children with ADHD in primary care settings.

Acknowledgments

This research was supported by National Institute of Mental Health grants K08-MH59672 and MH050313.

We thank the physicians, office staff members, and families in the 7 offices from Children’s Primary Care Medical Group, Scripps Pediatrics, and Neighborhood Healthcare for their time and effort. Without their participation, this research could not have been attempted or completed. We also thank Paul Kurtin, MD, Vice President for Clinical Innovations, and Patricia Richardson, MA, of the Children’s Hospital Center for Child Health Outcomes, for their commitment to bringing quality improvement efforts to both inpatient and outpatient pediatric issues. In addition, we thank members of the SANDAP clinical and research staff, including Amy McDaniel, Kim Stallone, Shellane Calma, William Gagner, MS, Amanda Bargallo, Christina Huang, Lucia Gonzalez, Mairav Cohen-Zion, Gabriela Gonzalea, and Phong Long.

References

1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders4th ed. Washington, DC: American Psychiatric Association; 1994.
2. Green M, Wong M, Atkins D, Taylor J, Feinleib M. Diagnosis of Attention-Deficit/Hyperactivity DisorderRockville, MD: Agency for Health Care Policy and Research; 1999. Technical review 3 (prepared by Technical Resources International under contract 290-94-2024). Agency for Health Care Policy and Research publication 99-0050.
3. Dulcan M. Practice parameters for the assessment and treatment of children, adolescents, and adults with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry. 1997;36:85–121. [PubMed]
4. Cantwell DP. Attention deficit disorder: A review of the past 10 years. J Am Acad Child Adolesc Psychiatry. 1996;35:978–987. [PubMed]
5. Hoagwood K, Kelleher KJ, Feil M, Comer DM. Treatment services for children with ADHD: a national perspective. J Am Acad Child Adolesc Psychiatry. 2000;39:198–206. [PubMed]
6. Hoagwood K. A national perspective on treatments and services for children with attention deficit hyperactivity disorder. In: Proceedings of the NIH Consensus Conference on Diagnosis and Treatment of ADHDBethesda, MD: National Institutes of Health; 1998:211–219.
7. Kazdin AE. Psychosocial treatments for conduct disorder in children and adolescents. In: Nathan P, Gorman J, eds. A Guide to Treatments That Work2nd ed. New York, NY: Oxford University Press; 2002:57–85.
8. Wolraich ML. Current assessment and treatment practices. In: Proceedings of the NIH Consensus Conference on Diagnosis and Treatment of ADHDBethesda, MD: National Institutes of Health; 1998:221–225.
9. Bureau of Health Professions. Shortage designation. Available at: http://bhpr.hrsa.gov/shortageAccessed May 6, 2004.
10. American Academy of Pediatrics. Clinical practice guideline: diagnosis and evaluation of the child with attention-deficit/hyperactivity disorder. Pediatrics. 2000;105:1158–1170. [PubMed]
11. Costello EJ, Angold A, Burns BJ, et al. The Great Smoky Mountains Study of Youth: Goals, design, methods, and the prevalence of DSM-III-R disorders. Arch Gen Psychiatry. 1996;53:1129–1136. [PubMed]
12. Wolraich ML, Lambert EW, Bickman L, Simmons T, Doffing MA, Worley KA. Assessing the impact of parent and teacher agreement on diagnosing attention-deficit hyperactivity disorder. J Dev Behav Pediatr. 2004;25:41–47. [PubMed]
13. Foley HA, Carlton CO, Howell RJ. The relationship of attention deficit hyperactivity disorder and conduct disorder to juvenile delinquency: legal implications. Bull Am Acad Psychiatry Law. 1996;24:333–345. [PubMed]
14. National Institutes of Health. Diagnosis and treatment of attention deficit/hyperactivity disorder (ADHD) NIH Consens Statement. 1998;16:1–37. [PubMed]
15. Wasserman R, Kelleher K, Bocian ABA, et al. Identification of attentional and hyperactivity problems in primary care: a report from Pediatric Research in Office Settings and the Ambulatory Sentinel Practice Network. Pediatrics. 1999;103(3). Available at: www.pediatrics.org/cgi/content/full/103/3/e38. [PubMed]
16. Jerome L, Gordon M, Hustler P. A comparison of American and Canadian teachers’ knowledge and attitudes towards attention deficit hyperactivity disorder (ADHD) Can J Psychiatry. 1994;39:563–567. [PubMed]
17. Homer CJ. The care model for child health. In: Improving Care for Children With Attention Deficit Hyperactivity DisorderDurham, NC: National Initiative for Children’s Healthcare Quality; 2001:29–32.
18. Wolraich ML, Lindgren S, Stromquist A, Milich R, Davis C, Watson D. Stimulant medication use by primary care physicians in the treatment of attention deficit hyperactivity disorder. Pediatrics. 1990;86:95–101. [PubMed]
19. Wolraich ML, Hannah J. Community resources and partnerships. In: Improving Care for Children With Attention Deficit Hyperactivity Disorder.Durham, NC: National Initiative for Children’s Healthcare Quality; 2001:67–81.
20. Robin AL. ADHD in Adolescents: Diagnosis and TreatmentNew York, NY: Guilford Press; 1998.
21. Block SL. Attention-deficit disorder: a paradigm for psychotropic medication intervention in pediatrics. Pediatr Clin North Am. 1998;45:1053–1083. [PubMed]
22. Olson AL, Kemper KJ, Kelleher KJ, Hammond CS, Zuckerman BS, Dietrich AJ. Primary care pediatricians’ roles and perceived responsibilities in the identification and management of maternal depression. Pediatrics. 2002;110:1169–1176. [PubMed]
23. Oxman TE, Dietrich AJ, Williams JW, Kroenke K. A three-component model for reengineering systems for the treatment of depression in primary care. Psychosomatics. 2002;43:441–450. [PubMed]
24. Rubenstein LV, Jackson-Triche M, Unutzer J, et al. Evidence-based care for depression in managed primary care practices. Health Aff (Millwood) 1999;18:89–105. [PubMed]
25. Wells KB. The design of partners in care: evaluating the cost-effectiveness of improving care for depression in primary care. Soc Psychiatry Psychiatr Epidemiol. 1999;34:20–29. [PubMed]
26. Wells KB, Schoenbaum M, Unützer J, Lagomasino IT, Rubenstein LV. Quality of care for primary care patients with depression in managed care. Arch Fam Med. 1999;8:529–536. [PubMed]
27. Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care: a randomized controlled trial. JAMA. 2000;283:212–220. [PubMed]
28. California Medical Association. The Coming Medical Group Failure Epidemic: Access to Medical Care for Millions of Californians Is at RiskSan Francisco, CA: California Medical Association; 1999.
29. Eisert DC, Sturner RA, Mabe PA. Questionnaires in behavioral pediatrics: guidelines for selection and use. J Dev Behav Pediatr. 1991;12:42–50. [PubMed]
30. Cassidy LJ, Jellinek MS. Approaches to recognition and management of childhood psychiatric disorders in pediatric primary care. Pediatr Clin North Am. 1998;45:1037–1052. [PubMed]
31. Stancin T, Palermo TM. A review of behavioral screening practices in pediatric settings: do they pass the test? J Dev Behav Pediatr. 1997;18:183–194. [PubMed]
32. Perrin EC. Presidential address: the promise of collaborative care. J Dev Behav Pediatr. 1999;20:57–62. [PubMed]
33. Peterson KA, Vinicor F. Strategies to improve diabetes care delivery. J Fam Pract. 1998;47(suppl):S55–S62. [PubMed]
34. Soumerai SB, McLaughlin TJ, Gurwitz JH, et al. Effect of local medical opinion leaders on quality of care for acute myocardial infarction: a randomized controlled trial. JAMA. 1998;279:1358–1363. [PubMed]
35. Greco PJ, Eisenberg JM. Changing physicians’ practices. N Engl J Med. 1993;329:1271–1273. [PubMed]
36. Katon W, Robinson P, Von Korff M, et al. A multifaceted intervention to improve treatment of depression in primary care. Arch Gen Psychiatry. 1996;53:924–932. [PubMed]
37. Katon W, Von Korff M, Lin E, et al. Collaborative management to achieve treatment guidelines: impact on depression in primary care. JAMA. 1995;273:1026–1031. [PubMed]
38. Sofaer S. Qualitative methods: what are they and why use them? Health Serv Res. 1999;34:1101–1118. [PMC free article] [PubMed]
39. Ware NC, Tugenberg T, Dickey B, McHorney CA. An ethnographic study of the meaning of continuity of care in mental health services. Psychiatr Serv. 1999;50:395–400. [PubMed]
40. Wolraich ML, Feurer ID, Hannah JN, Baumgaertel A, Pinnock TY. Obtaining systematic teacher reports of disruptive behavior disorders utilizing DSM-IV. J Abnorm Child Psychol. 1998;26:141–152. [PubMed]
41. Johnston C, Fine S. Methods of evaluating methylphenidate in children with attention deficit hyperactivity disorder: acceptability, satisfaction, and compliance. J Pediatr Psychol. 1993;18:717–730. [PubMed]
42. Wolraich ML, Hannah JN, Baumgaertel A, Feurer ID. Examination of DSM-IV criteria for attention deficit/hyperactivity disorder in a countywide sample. J Dev Behav Pediatr. 1998;19:162–168. [PubMed]
43. Swanson J, Lerner M, March J, Gresham F. Assessment and intervention for attention-deficit/hyperactivity disorder in the schools. Pediatr Clin North Am. 1999;46:993–1009. [PubMed]
44. Mulhern S, Dworkin PH, Bernstein B. Do parental concerns predict a diagnosis of attention-deficit hyperactivity disorder? J Dev Behav Pediatr. 1994;15:348–352. [PubMed]
45. August GJ, Garfinkel BD. Behavioral and cognitive subtypes of ADHD. J Am Acad Child Adolesc Psychiatry. 1989;28:739–748. [PubMed]
46. Rapoport JL, Donnelly M, Zametkin A, Carrougher J. “Situational hyperactivity” in a U.S. clinical setting” J Child Psychol Psychiatry. 1986;27:639–646. [PubMed]
47. Mannuzza S, Klein RG, Moulton JL., III Young adult outcome of children with “situational” hyperactivity: a prospective, controlled follow-up study. J Abnorm Child Psychol. 2002;30:191–198. [PubMed]
48. Costello EJ, Loeber R, Stouthamer-Loeber M. Pervasive and situational hyperactivity–confounding effect of informant: a research note. J Child Psychol Psychiatry. 1991;32:367–376. [PubMed]
49. Reiff MI, Tippins S, eds. ADHD: A Complete and Authoritative GuideElk Grove Village, IL: American Academy of Pediatrics; 2004.
50. Wagner EH, Glasgow RE, Davis C, et al. Quality improvement in chronic illness care: a collaborative approach. Jt Comm J Qual Improv. 2001;27:63–80. [PubMed]
51. Addis ME, Waltz J. Implicit and untested assumptions about the role of psychotherapy treatment manuals in evidence-based mental health practice. Clin Psychol Sci Pract. 2002;9:421–424.
52. Davis DA, Taylor-Vaisey A. Translating guidelines into practice: a systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. Can Med Assoc J. 1997;157:408–416. [PMC free article] [PubMed]
53. Davis D, Thomson O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282:867–874. [PubMed]
54. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. 1995;274:700–705. [PubMed]
55. VandeCreek L, Knapp S, Brace K. Mandatory continuing education for licensed psychologists: its rationale and current implementation. Prof Psychol. 1990;21:135–140.
56. Riekert KA, Stancin T, Palermo TM, Drotar D. A psychological behavioral screening service: use, feasibility, and impact in a primary care setting. J Pediatr Psychol. 1999;24:405–414. [PubMed]
57. Dodds M, Nicholson L, Muse B, III, Osborn LM. Group health supervision visits more effective than individual visits in delivering health care information. Pediatrics. 1993;91:668–670. [PubMed]
58. Jellinek M, Nurcombe B. Two wrongs don’t make a right: managed care, mental health, and the marketplace. JAMA. 1993;270:1737–1739. [PubMed]
59. Drotar D. Coming of age: critical challenges to the future development of pediatric psychology. J Pediatr Psychol. 1991;16:1–11. [PubMed]
60. Fisher L, Ransom DC. Developing a strategy for managing behavioral health care within the context of primary care. Arch Fam Med. 1997;6:324–333. [PubMed]
61. Kelleher K, Long N. Barriers and new directions in mental health services research in the primary care setting. J Clin Child Psychol. 1994;23:133–142.
62. Link B, Levav I, Cohen A. The primary medical care practitioner’s attitudes toward psychiatry: an Israeli study. Soc Sci Med. 1982;16:1413–1420. [PubMed]
63. Haggerty R. Behavioral pediatrics: can it be taught? Can it be practiced? Pediatr Clin North Am. 1982;29:391–398. [PubMed]
64. National Advisory Mental Health Council Workgroup on Child and Adolescent Mental Health Intervention Development and Deployment. Blueprint for Change: Research on Child and Adolescent Mental HealthWashington, DC: National Institute of Mental Health; 2001.
65. US Public Health Service. Report of the Surgeon General’s Conference on Children’s Mental Health: A National Action AgendaWashington, DC: Department of Health and Human Services; 2000.
66. Hoagwood K, Burns BJ, Kiser L, Ringeisen H, Schoenwald SK. Evidence-based practice in child and adolescent mental health services. Psychiatr Serv. 2001;52:1179–1189. [PubMed]