Overlooked and Underserved: “Action Signs” for Identifying Children With Unmet Mental Health Needs

doi:10.1542/peds.2009-0367

Pediatrics. 2011 November; 128(5): 970–979.

doi: 10.1542/peds.2009-0367

PMCID: PMC3387881

Overlooked and Underserved: “Action Signs” for Identifying Children With Unmet Mental Health Needs

Peter S. Jensen, MD,^a^b Eliot Goldman, PhD,^c David Offord, MD,^d Elizabeth J. Costello, PhD,^e Robert Friedman, PhD,^f Barbara Huff,^g Maura Crowe, BA,^c Lawrence Amsel, MD,^c Kathryn Bennett, PhD,^d Hector Bird, MD,^c Rand Conger, PhD,^h Prudence Fisher, PhD,^c Kimberly Hoagwood, PhD,ⁱ Ronald C. Kessler, PhD,^j and Robert Roberts, PhD^k

^aDepartment of Psychiatry and Psychology, Mayo Clinic, Rochester, New York;

^bReach Institute, New York, New York;

^cDepartment of Psychiatry and

ⁱDivision of Services Research, Columbia University/New York State Psychiatric Institute, New York, New York;

^dDepartment of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada;

^eDepartment of Psychiatry and Behavioral Sciences, Duke University, Durham, North Carolina;

^fFlorida Mental Health Institute, University of South Florida, Tampa, Florida;

^gFederation of Families for Children's Mental Health, Alexandria, Virginia;

^hInstitute for Social and Behavioral Research, University of California, Davis, California;

^jDepartment of Health Care Policy, Harvard Medical School, Boston, Massachusetts; and

^kDivision of Health Promotion and Behavioral Science, School of Public Health, University of Texas, Houston, Texas

Corresponding author.

Address correspondence to Peter S. Jensen, MD, Department of Psychiatry and Psychology, Mayo Clinic, 200 First St SW, Rochester, MN 55905. E-mail: jensen.peter/at/mayo.edu

Accepted August 8, 2011.

Abstract

OBJECTIVE:

The US Surgeon General has called for new approaches to close the mental health services gap for the large proportion of US children with significant mental health needs who have not received evaluation or services within the previous 6 to 12 months. In response, investigators sought to develop brief, easily understood, scientifically derived “warning signs” to help parents, teachers, and the lay public to more easily recognize children with unmet mental health needs and bring these children to health care providers' attention for evaluation and possible services.

METHOD:

Analyses of epidemiologic data sets from >6000 children and parents were conducted to (1) determine the frequency of common but severely impairing symptom profiles, (2) examine symptom profile frequencies according to age and gender, (3) evaluate positive predictive values of symptom profiles relative to Diagnostic and Statistical Manual of Mental Disorders diagnoses, and (4) examine whether children with 1 or more symptom profiles receive mental health services.

RESULTS:

Symptom-profile frequencies ranged from 0.5% to 2.0%, and 8% of the children had 1 or more symptom profile. Profiles generated moderate-to-high positive predictive values (52.7%–75.4%) for impairing psychiatric diagnoses, but fewer than 25% of children with 1 or more profiles had received services in the previous 6 months.

CONCLUSIONS:

Scientifically robust symptom profiles that reflect severe but largely untreated mental health problems were identified. Used as “action signs,” these profiles might help increase public awareness about children's mental health needs, facilitate communication and referral for specific children in need of evaluation, and narrow the child mental health services gap.

Keywords: children, adolescents, mental health, services use, underserved, unmet need

Despite the presence of well-established diagnostic criteria for children's mental disorders for >20 years,^1–3 parents and teachers often fail to recognize the presence of possible mental health problems in children. Estimates by the US Surgeon General⁴ have revealed that most children with mental disorders do not receive treatment,^5,6 even with well-recognized conditions such as attention-deficit/hyperactivity disorder (ADHD).

Recognition of children's mental health needs depends on the awareness and actions of key adults.^5,6 However, failure to recognize children's mental health needs is not only a problem among laypersons but also among education, welfare, juvenile justice, and health care professionals.^7,8 Underidentification is of particular concern in schools^9–13 and primary care,^4,14–20 where virtually all children are seen and where identification should be feasible. However, only 1 in 4 children with a mental disorder is identified by the primary care doctor.²¹ In fact, the best predictor of a primary care clinician (PCC) identifying a child's mental health problem is whether parents draw the PCC's attention to the issues rather than any PCC-initiated procedure.²² Evidence suggests that PCCs are more comfortable in identifying and managing attention disorders, whereas identification, treatment, and referral for other disorders lag.²³

Communication is complicated by difficulties distinguishing symptomatic from normal behaviors. Symptom lists are often used to help parents and others identify children with mental health needs, but such lists might not be of assistance if the parent, teacher, or physician reasons (correctly) that “every child has some of these symptoms some of the time.” In addition, the time and administrative burdens needed to screen for mental health problems within primary care practices seem to be major deterrents to their use.^23–25 Attempts to remedy this situation have involved computer-assisted²⁵ and symptom-specific²² screening procedures, but these methods also involve time or expense and might be biased toward a single area of functioning (eg, depression screeners).

Better ways to communicate about children's mental health needs must be developed, in terms easily understood by persons of different backgrounds, cultures, and education levels. Ideally, these simpler communication methods should apply the “lessons learned” from studies of decision-making and our understanding of the limitations of human capacities to make complex judgments under uncertainty (eg, to refer or not refer a child for evaluation).^26,27 For example, many cognitive operations are required for parents to (1) determine which, if any, symptoms their child might have from a long list, (2) evaluate the significance of any symptoms in terms of what the parents know about the child, (3) ponder whether the problems are severe enough to bring them to their doctor's attention, and (4) actually schedule a visit. By way of contrast, the cognitive present/not-present determinations of a single well-described criterion, such as a single symptom profile or “warning sign,” greatly simplify the cognitive operations, might reduce uncertainty in decision-making, and facilitate parents and others to obtain a health care evaluation when a child is in distress.

The idea of creating warning signs for health problems is not new. In 1971, President Nixon declared a “war on cancer” with enactment of the National Cancer Act.²⁸ Seven warning signs were developed as a communication tool for early intervention.^26,27 These warning signs were designed to be easily understood so that those with a potential cancer warning sign would realize that a checkup is required. Any single warning sign was meant to trigger an action (ie, seeing one's doctor for an evaluation).

Invoking the need to develop a similar strategy for child mental health, the US Surgeon General^29–31 called for researchers to find more effective means for public communications about the types of children's behaviors that warrant professional attention. This need was further underscored by the President's New Freedom Commission on Mental Health.^32–35

To meet the Surgeon General's challenge, federal officials determined that brief, common-sense descriptions of child mental health problems might be useful if they (1) validly characterize children with mental health problems, (2) are cast in language readily understood by the public, and (3) are accepted by the public as credible indicators of a child's need for a health evaluation. The value of such descriptions would then be for their use as communication/education tools and public messages (ie, as “warning” or “action” signs) to educate and mobilize the public to identify and refer appropriate children for health care evaluations.

Thus, officials from the National Institute of Mental Health and the Center for Mental Health Services supported a contract to empirically determine if certain behavior/symptom profiles might be developed as action signs for public communication and educational objectives. Under federal guidance, we established a steering committee (SC) (that consisted of child mental health epidemiologists, parent/advocacy representatives, and policy experts [see author list]), the goal of which was to develop these warning signs. The SC met regularly (by telephone and in person) to define the goals and methods of the project. The following project objectives were set.

To identify brief, easily understood symptom profiles of children with significant behavioral health problems from extant epidemiologic data sets. Because of well-known variations in mental disorder prevalence based on the level of impairment required to meet diagnostic criteria, and given the SC's wish to avoid identifying children without significant problems as having a warning sign (ie, false-positives), the SC focused on symptom profiles for severely ill children for whom a medical evaluation would be deemed medically necessary and noncontroversial, even to those who are skeptical.
To determine if these descriptors characterize children with common yet severe behavioral problems and who are not receiving any health care for these problems (eg, unmet need). This second objective was critical, because if the descriptors only identified children who were already receiving health care services, they would not serve the purpose of identifying children with severe yet unrecognized problems. SC members set the goal that the action signs should identify ~5% to 7% of the community population,³⁵ erring on the side of low sensitivity and high specificity.
To ensure that the descriptors map onto recognized, impairing psychiatric diagnoses with positive predictive values of ≥50%.
To translate the profiles into common-sense language that can be readily understood by the public and ultimate users.
To work with major national professional and advocacy organizations to officially endorse/adopt the warning signs for further dissemination.

Here we present the methods and findings from these 5 goals of our warning-sign development. Please note that, from this point forward, we refer to the warning signs as “action signs” on the basis of recommendations of the SC and multiple focus groups, which indicated that the latter term would be more useful and less likely to be stigmatizing.

METHOD

To identify data sets for which the necessary analyses could be performed, the SC comprehensively reviewed the English-language research literature on epidemiologic studies of childhood mental illness (for children aged 7–17 years), services use, and unmet need by using the Medline and PsycINFO search engines. SC members consulted with the National Institute of Mental Health to find any relevant, federally funded epidemiologic studies not yet in the literature. Ultimately, all federally funded national and regional epidemiologic studies^36–49 were identified, and their results were reviewed. After review of eligible data sets, the SC determined that the final data sets must (1) have used well-validated measures in assessing the range of established psychiatric disorders, (2) have ascertained specific symptoms, symptom patterns, and final diagnoses, (3) provide information about the children's receipt of health care services, (4) have taken place in North America, (5) have been conducted in 1990 or later, and (6) have been drawn from a large, representative community sample.

Data sets from 4 studies were determined to be appropriate and available for analysis: Methods for the Epidemiology of Mental Disorders in Children and Adolescents (MECA)³⁶; Iowa-Georgia Rural Minority Study (Iowa)³⁷; Depression and Anxiety in Minority Youth and Primary Care (Texas)³⁸; and Antisocial Behaviors in US and Island Puerto Rican Youth (Bronx, NY).^39,40 Detailed descriptions of all data sets and their study procedures are available on request. Each data set was representative of the given population within the sampling frame. The sample sizes ranged from nearly 1000 to >4000 children, and child ages ranged between 7 and 17 years. In addition to the necessary characteristics we described earlier, these data sets were chosen because the same diagnostic assessment instrument (the Diagnostic Interview for Children and Adolescents [DISC] versions 2.3 and 4.0^41,42) was used, which facilitated the same analyses across all 4 data sets and enabled us to ultimately combine the data sets for summary analyses.

The SC proposed symptom profiles based on face-valid medical-necessity criteria (ie, that the profile includes a common reason for a child to be seen in clinical settings; that the profile is characterized by substantial impairment or risk for harm; that in most communities the symptom profile would constitute credible reasons among the lay public for seeking a health care evaluation and/or services; and that failing to get an evaluation could put the child at risk for further harm). During SC deliberations a list of symptom profiles quickly emerged; these profiles changed only slightly despite extensive vetting and input from teacher, parent, youth, and PCC focus groups over the ensuing 12 months. All warning-sign constructs included impairment, which entailed 1 or more of the following: (1) extreme emotions that interfere with daily life (eg, severe depression, anger/rage); (2) behaviors that put the child or others at risk for physical harm (eg, suicidal plans/attempts, aggression, purging, or laxative abuse); or (3) extreme inattention or hyperactive behaviors that caused school failure or physical danger. Although some of these problems map loosely onto specific diagnoses, there are many exceptions (eg, the child who is suicidal might actually have substance abuse or panic disorder; the withdrawn child might have a substance use disorder, depression, or schizophrenia). The decision to use symptom/problem profiles rather than diagnoses was based on pragmatic communication and decision-making principles: (1) use common-sense language and constructs familiar to parents, teachers, and youth (peers); (2) avoid jargon or terms that might stigmatize (eg, mental, hyper, schizo, psycho); and (3) keep it simple (ie, avoid symptom profiles that would require people to ponder long Diagnostic and Statistical Manual of Mental Disorders [DSM]^1–3 symptom lists before concluding that a given child has a “problem”).

After input from multiple focus groups, 4 additional constructs were added, namely for repeated use of illegal drugs, extreme problems with attention, mood swings, and drastic personality changes. The latter 2 indicators could not be gleaned from the DISC interviews, because questions pertaining to those constructs are not asked in the DISC.

The DISC was examined to find question items that, if answered positively, would meet the criteria for the proposed initial symptom descriptions. For example, the SC operationalized the depression symptom profile with 3 criteria: (1) depressive symptoms; (2) lasting a minimum of 2 weeks; and (3) resulting in substantial impairment in ≥2 settings (home functioning, school functioning, peer relationships, and/or intrapsychic distress). All of these variables are assessed by the DISC, which makes it possible to define the construct. A complete listing of DISC interview questions that map onto symptom profiles are available from the authors.

To assess impairment, the DISC specifically asks at the end of each section (ADHD, anxiety, depression, etc) whether the symptoms endorsed by the interviewee cause impairment at home, school, or with peers or cause the child severe internal distress. Each impairment item is coded on a 3-point scale (0, none; 1, some; 2, a lot). The SC required that a child have a total impairment score of ≥2, which could mean that the child had “a lot” of impairment at school² but none at home or elsewhere or at least some impairment in ≥2 domains (1 + 1). Likewise, at the end of each section the DISC asks whether the child has seen any mental health, educational, or health care provider in the previous 6 months because of the endorsed problems. Thus, we could determine how many children had a specific symptom profile, whether it caused substantial impairment, whether that symptom profile met full criteria for a DSM-IV diagnosis,³ and whether the child had received any services for the problem(s). If not, by definition that child fell into the mental health services gap definition of having “unmet need.” Frequency analyses according to age and gender were calculated for all indicators and diagnoses.

Parallel analyses were then performed to determine if the findings could be replicated in other data sets. Thus, the Great Smoky Mountains Study⁴⁹ met all of the same criteria as the other 4 data sets except it used the Child and Adolescent Psychiatric Assessment (rather than the DISC) as the diagnostic instrument in this sample of >1000 children. Similar analyses were performed on this sample to determine if findings were specific to the DISC versus more applicable generally. As a final step, after the initial definition and empirical testing of the symptom profiles, the SC convened multiple geographically and ethnically diverse focus groups to (1) solicit feedback about the action signs, (2) ascertain their credibility with parents, teachers, youth, and PCCs, (3) identify any missing action signs, and (4) craft optimal language for laypersons' understanding, use, and eventual dissemination. A description of this focus-group feedback process is available from the authors (P.S.J., unpublished data). The action-sign descriptors initially identified by the SC, as well as the final action signs' wording after the focus groups' input, are shown in Table 1.

TABLE 1

Final List of Action/Warning Signs

RESULTS

Because our ultimate objective was to develop action signs that identify underserved children with significant mental health problems, our first analyses necessarily examined the extent to which the data sets contained substantial numbers of children who met criteria for specific DSM-IV³ diagnoses and who were not receiving services. Disorder-specific prevalence rates and the corresponding proportion of those who did not receive service (shown in parentheses) were as follows, parent versus youth reports, respectively: major depressive disorder, 1.9% (74.5%) vs 1.8% (70.1%); conduct disorder, 1.2% (69.8%) vs 2.2% (88.8%); eating disorder (bulimia or anorexia nervosa), 0.5% (85.7%) vs 0.4% (60.0%); ADHD, 5.4% (63.6%) vs 2.2% (63.1%); substance use disorder, 0.1% (80.%) vs 0.8% (76.5%); and any anxiety disorder, 7.3% (82.5%) vs 8.6% (93.3%). Tables in which disorder rates are compared according to age and gender, with significance values and confidence intervals, are available on request.

However, the action signs were not intended to identify specific diagnoses per se but, rather, were meant to help non–mental health professionals recognize underserved children with significant mental health needs regardless of specific diagnosis to ensure that such children are brought to clinical attention. To avoid action-sign definitions that would identify mildly troubled children and/or those with no clinical disorder whatsoever, the SC stipulated that action signs should be associated with substantial impairment, such as that required for any DSM diagnosis. However, a child might exhibit an action sign and have a quite different diagnosis, which makes it difficult to determine if any specific action sign is linked to impairment. To address this problem, we computed an overall “any-diagnosis” category to assess how well the presence of 1 or more action signs mapped on to the any-diagnosis criterion (and, by extension, clinically significant impairment). Any-diagnosis prevalence rates for children are listed in Table 2, separated according to age and gender groups. As seen here, regardless of whether diagnoses were determined by child or parent reports, ~15% met criteria for 1 or more DSM-IV disorders,³ which is consistent with the results of most previous studies^36–49 but slightly higher than those of a recent epidemiologic study.⁵²

TABLE 2

Number and Proportion of Children With Any Diagnosis According to the DISC

Table 3 indicates the number and proportion of children with specific action signs, as well as those who met the any-action-sign criterion (ie, ≥1), which yielded <10%, parents or youth. Thus, as intended, the indicators identified a smaller proportion of children (presumably those more severely affected) than the larger proportion that met DSM-IV diagnostic criteria. Also seen here is that high levels of unmet need were found among children with 1 or more action signs that ranged from 60% for depression to 88.9% for aggression (specific findings of levels of unmet need for each action sign are available on request).

TABLE 3

Action-Sign Frequency According to Age, Gender, and Informant

Major differences in frequencies can be seen between parent and youth reports in Table 3. One extreme example is the difference in recognition of youths' suicidal behavior by youth and parents: using the youth report as the gold standard, parents are only aware of approximately one-third of all youth with suicidal plans and one-sixth of youth with actual attempts. Parents are also less aware of eating-behavior problems (which might be expected given the secrecy of the disorder), repeated drug use, and severe aggression or carrying a weapon. These differences in the informant-described prevalence of the action signs are generally consistent with results from previous studies.^50,51

Table 4 shows the extent to which subjects who met the any-action-sign criterion also meet the any-diagnosis criterion. These data reveal moderate-to-high positive predictive values (52.7% youth report, 74.8% parent report) of action signs to DSM-IV diagnoses, which indicates that children with 1 or more action signs have clinically significant, impairing problems.

TABLE 4

Positive Predictive Value of Having 1 or More Action Signs (Any Action Sign) Versus Any DSM Diagnosis

In terms of unmet need, Table 5 lists the number and proportion of children in services with any diagnosis (≥1) or any action sign (≥1). For children who met the any-diagnosis criterion, most were not receiving services in the previous 6 months (75.2%–86.9%). Likewise, despite the likely greater severity of the action-sign profiles, 73.8% (parent DISC) and 83.2% (youth DISC) of children who met any action-sign criteria received no health care assessment or services during the previous 6 months.

TABLE 5

Unmet Needs: Number and Percentage of Children Who Did not Receive Services, According to Any Diagnosis or Any Action Sign

The parallel analyses with the Child and Adolescent Psychiatric Assessment instrument reinforced these findings. In this sample, for which a different diagnostic instrument was used, approximately two-thirds of the children (63.6%) with an indicator had not seen anyone for evaluation or services before the interview period. Also similar to our findings earlier, analyses that compared the indicators with actual Child and Adolescent Psychiatric Assessment diagnoses revealed moderate-to-high positive predictive values (range: 30.6%–100%; average: 45.8%) (tables are available on request). The final action signs and toolkit prepared for the US Department of Health and Human Services Center for Mental Health Services is available online at www.thereachinstitute.org/files/documents/action-signs-toolkit-final.pdf.

DISCUSSION

According to these analyses, the overall prevalence of mental health disorders among children and adolescents is ~15%, whether by parent or youth report (note that not all diagnoses were included in these samples [eg, oppositional defiant disorder, Tourette syndrome, autism, etc], so rates including these conditions would likely be higher, perhaps at the generally acknowledged levels of 20%).²⁸ In contrast, the prevalence of more severe indicators ranged from 6.7% for parent reports to 9.9% for youth reports, which yields rates that more closely approximate the efforts by the Center for Mental Health Services to identify the prevalence of “severe emotional disorder.” However, the high levels of children with significant behavioral or mental health problems who were not receiving any services in the previous 6 months are alarming, particularly because the action signs were intended to identify those with an undisputed (face-valid) medical necessity for evaluation and possible services.

These statistics reveal a sobering problem: among children with severe types of problems of greatest concern to all focus groups, most are not being seen by any type of provider, neither health care nor educational. These findings, sadly, are not new. Even for the ADHD indicator, a presumably overdiagnosed disorder, of the 1% of children who had hyperactivity so severe that it put them at physical risk or resulted in school failure, only one-third had received an assessment or had been in care in the previous 6 months. Most dramatically, for action signs related to purported abnormal behaviors such as aggression, children are almost never in services, as shown by our data. Thus, among children who are viewed by stakeholders as having severe types of behavioral health problems, lack of services was extremely high, which is an important new but uniform finding across all 4 of the methodologically rigorous data sets.

The credibility and face validity of the proposed action signs was apparent in the analyses of action signs according to diagnosis (Table 5). On the whole, the action signs mapped onto specific DSM diagnoses 52% to 75% (positive predictive value) of the time and onto the reality that the overwhelming majority of these children do not receive any health care services for their problems. The replication of analyses with the Great Smoky Mountains Study revealed the validity of our findings independent of the DISC instrument.

By way of caution, it should be noted that, by design, the action signs have low sensitivity to most diagnoses, which is easily seen when one views Table 3 and the prevalence rates for any disorder (15.2% and 15.2%, parent and child report, respectively) and compares them to the lower rates for any action sign in Table 4 (6.7% and 9.9%, parent and child report, respectively). Thus, the sensitivities of the any-action-sign metric against the any-disorder variable is low (0.31 and 0.34, parent and child, respectively), whereas the specificities are high (0.98 and 0.95, respectively). As a consequence, if one were to use the action signs as some type of screener, two-thirds of the children with DSM disorders would be missed. On the other hand, the action signs might be useful for educational and broad public service messaging, because they were deliberately designed to avoid risk for harm by overidentifying children or alarming parents. It is important to note that despite their low sensitivities, 70% to 80% of the children the action signs do identify have not received relevant services in the previous 6 months, which indicates that the action signs do seem to effectively identify severely ill children, most of whom have unmet need.

Overall, the action signs identified approximately equal numbers of boys and girls, although specific action-sign rates varied according to gender (eg, substantially greater rates for the anxiety-problem action signs in 7- to 11-year-old girls than boys or the twofold-greater rates of extreme hyperactivity in 12- to 17-year-old boys than girls). It should be noted that the prevalence of action signs seemed to increase with age (from 6% to 8% in the younger age groups to 8% to 12% in older children and adolescents). The difference might reflect a real variation in rates of severe, action-sign–type problems in children versus adolescents, or it might signify a need to identify profiles that better identify younger children. Unfortunately, among children younger than 7 years, we are unaware of any representative data in the United States that could be used to develop and test action signs in this age group, so we could not address this potential problem in this report.

Clinical Implications

This project has broad clinical, service, and policy applications. Scientifically valid, easily understood, and readily accessible action signs might facilitate children's pathways to referral and receipt of needed services and warrant further study and application if we are to close the mental health services gap for children with severe emotional and behavioral difficulties.^53,54 Although new, effective treatments are increasingly available, such treatments will not reach children in need without informed parents and well-educated primary care providers, teachers, and others who are armed with efficient and easily applied means of identifying which children need these services.

Limitations

Although the action signs might serve as effective educational and communication tools for facilitating awareness among parents, teachers, youth, and health care professionals about children's mental health needs, additional studies are needed to ensure that parents and teachers understand them; to determine if they “take action” in appropriate fashion when a child manifests an action sign; and that children and families are bridged effectively over the mental health services gap. Additional studies should also conduct independent evaluations of impairment associated with the various action signs to firmly establish whether they do, in fact, identify more severely impaired children.

We noted that children and adolescents might have other problems and/or significant impairment in functioning not adequately defined by these action signs. For example, should “cutting” be included in some future action signs? Likewise, epidemiologic data sets in which symptoms in very young children were assessed were not available for our analysis, which resulted in significant gaps in our understanding about appropriate action signs in this younger age group. In addition, the full spectrum of diagnoses (autism, Tourette syndrome, enuresis/encopresis, milder cases of ADHD, oppositional defiant disorder, etc) were not included or captured by the current action signs, in part because the DISC does not address conditions such as autism and in part because the SC chose to focus on common and severely impairing conditions.

Finally, we noted that a number of important variables (such as ethnicity, family income) were not available from all 4 data sets, so we could not determine if these variables were related to different levels of action signs. Further analyses of other data sets with a broader array of potential moderating variables could yield additional information about how such factors might affect the distribution of action signs across different population groups. Available evidence suggests that these variables can exert significant effects, particularly in relation to access to and utilization of care and, by extension, unmet needs for services.^54,55

Despite these caveats, the action signs reported here do seem to characterize children across major gender and age groups who have relatively common yet credible behavioral health problems that, more often than not, are not being addressed. Applying action signs to a variety of settings could have a significant impact on early identification and access to services for many underidentified children. However, more research is needed to ensure the ultimate usefulness and broad-scale applicability of the action signs and to determine if they can serve as effective tools for educating the public to better recognize and respond to children's needs for behavioral health services. If appropriately coupled with other efforts to actually improve our diagnostic and treatment services for these children via guidelines and toolkits, substantial progress in children's behavior health outcomes might soon be possible.

The action signs and an associated toolkit are available at www.thereachinstitute.org/files/documents/action-signs-toolkit-final.pdf. The final federal report is available from the corresponding author on request.

ACKNOWLEDGMENTS

This article and the associated Action Signs toolkit was developed under a contract with the Substance Abuse and Mental Health Services Administration/US Department of Health and Human Services through the American Institutes for Research prime contract No. 280-2003-00042 and Task 14, a Center for Mental Health Services/National Institute of Mental Health Child Mental Health Indicators Project.

We gratefully acknowledge Ms Wei Li, MS, for completing the statistical analyses for this report.

The views, policies, and opinions expressed are those of the authors and do not necessarily reflect those of the Substance Abuse and Mental Health Services Administration or the US Department of Health and Human Services.

FINANCIAL DISCLOSURE: Dr Jensen has been a consultant for Shire US, Inc, Janssen-Ortho, and the Neuroscience Education Institute; and Dr Kessler has been a consultant for AstraZeneca, Analysis Group, Bristol-Myers Squibb, Cerner-Galt Associates, Eli Lilly & Company, GlaxoSmithKline Inc, HealthCore Inc, Health Dialog, Integrated Benefits Institute, John Snow Inc, Kaiser Permanente, Matria Inc, Mensante, Merck & Co, Inc, Ortho-McNeil Janssen Scientific Affairs, Pfizer Inc, Primary Care Network, Research Triangle Institute, Sanofi-Aventis Groupe, Shire US Inc, SRA International, Inc, Takeda Global Research & Development, Transcept Pharmaceuticals Inc, and Wyeth-Ayerst, has served on advisory boards for Appliance Computing II, Eli Lilly & Company, Mindsite, Ortho-McNeil Janssen Scientific Affairs, and Wyeth-Ayerst, and has had research support for his epidemiologic studies from Analysis Group Inc, Bristol-Myers Squibb, Eli Lilly & Company, EPI-Q, GlaxoSmithKline, Johnson & Johnson Pharmaceuticals, Ortho-McNeil Janssen Scientific Affairs, Pfizer Inc, Sanofi-Aventis Groupe, and Shire US, Inc. The other authors have indicated they have no financial relationships relevant to this article to disclose.


ADHD	attention-deficit/hyperactivity disorder

PCC	primary care clinician

SC	steering committee

DISC	Diagnostic Interview Schedule for Children

DSM	Diagnostic and Statistical Manual of Mental Disorders

REFERENCES

1. American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders. 3rd ed Washington, DC: American Psychiatric Association; 1980.

2. American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders. 3rd ed, Revised Washington, DC: American Psychiatric Association; 1987.

3. American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders. 4th ed Washington DC: American Psychiatric Association; 1994.

4. US Public Health Service Mental Health: A Report of the Surgeon General. Rockville, MD: US Department of Health and Human Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental Health; 1999.

5. Burns BJ, Costello EJ, Angold A, et al. Children's mental health service use across service sectors. Health Aff (Millwood). 1995;14(3):147–159. [PubMed]

6. Leaf PJ, Alegria M, Cohen P, et al. Mental health service use in the community and schools: results from the four-community MECA study—Methods for the Epidemiology of Child and Adolescent Mental Disorders study. J Am Acad Child Adolesc Psychiatry. 1996;35(7):889–897. [PubMed]

7. Jensen PS, Kettle L, Roper MT, et al. Are stimulants overprescribed? Treatment of ADHD in four U.S. communities. J Am Acad Child Adolesc Psychiatry. 1999;38(7):797–804. [PubMed]

8. Jensen PS. Current concepts and controversies in the diagnosis and treatment of attention deficit hyperactivity disorder. Curr Psychiatry Rep. 2000;2(2):102–109. [PubMed]

9. Pescosolido B. Beyond rational choice: the social dynamics of how people seek help. Am J Soc. 1992;97(4):1096–1138.

10. Briggs-Gowan MJ, Horwitz SM, Schwab-Stone ME, Leventhal JM, Leaf PJ. Mental health in pediatric settings: distribution of disorders and factors related to service use. J Am Acad Child Adolesc Psychiatry. 2000;39(7):841–849. [PubMed]

11. Garland AF, Hough RL, Landsverk JA, Brown SA. Multi-sector of systems of care for youth with mental health needs. Child Serv. 2001;4(3):123–140.

12. Wildman BG, Kizilbash AH, Smucker WD. Physicians' attention to parents' concerns about the psychosocial functioning of their children. Arch Fam Med. 1999;8(5):440–444. [PubMed]

13. Pescosolido BA, Jensen PS, Martin JK, Perry BL, Olafsdottir S, Fettes D. Public knowledge and assessment of child mental health problems: findings from the National Stigma Study-Children. J Am Acad Child Adolesc Psychiatry. 2008;47(3):339–349. [PubMed]

14. Costello EJ, Costello AJ, Edelbrock C, et al. Psychiatric disorders in pediatric primary care: prevalence and risk factors. Arch Gen Psychiatry. 1988;45(12):1107–1116. [PubMed]

15. Costello EJ, Edelbrock C, Costello AJ, Dulcan MK, Burns BJ, Brent D. Psychopathology in pediatric primary care: the new hidden morbidity. Pediatrics. 1988;82(3 pt 2):415–424. [PubMed]

16. Glisson C, Himmelgarn A. The effects of organizational climate and inter-organizational coordination on the quality and outcomes of children's service systems. Child Abuse Negl. 1998;22(5):401–421. [PubMed]

17. Kelleher K, Long N. Barriers and new directions in mental health services research in the primary care setting. J Clin Child Psychol. 1994;23(2):133–142.

18. US Public Health Service Report of the Surgeon General's Conference on Children's Mental Health: A National Action Agenda. Washington, DC: Department of Health and Human Services; 2001.

19. US Public Health Service Youth Violence: A Report of the Surgeon General. Washington, DC: Department of Health and Human Services; 2001.

20. US Public Health Service Mental Health: Culture, Race, and Ethnicity (a Supplement to Mental Health: A Report of the Surgeon General). Rockville, MD: US Department of Health and Human Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental Health; 2001.

21. Gould MS, Greenberg T, Velting DM, Shaffer D. Youth suicide risk and preventive interventions: a review of the past 10 years. J Am Acad Child Adolesc Psychiatry. 2003;42(4):386–405. [PubMed]

22. Zuckerbrot RA, Cheung AH, Jensen PS, Stein RE, Laraque D.; GLAD-PC Steering Group Guidelines for Adolescent Depression in Primary Care (GLAD-PC): I. Identification, assessment, and initial management. Pediatrics. 2007;120(5). Available at: www.pediatrics.org/cgi/content/full/120/5/e1299. [PubMed]

23. Gardner W, Kelleher KJ, Pajer KA, Campo JV. Primary care clinicians' use of standardized tools to assess child psychosocial problems. Ambul Pediatr. 2003;3(4):191–195. [PubMed]

24. Valenstein M, Vijan S, Zeber JE, Boehm K, Buttar A. The cost-utility of screening for depression in primary care. Ann Intern Med. 2001;134(5):345–360. [PubMed]

25. Stevens J, Kelleher KJ, Gardner W, et al. Trial of computerized screening for adolescent behavioral concerns. Pediatrics. 2008;121(6):1099–1105. [PubMed]

26. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981;211(4481):453–458. [PubMed]

27. Kahneman D, Tversky A. Choices, Values, and Frames. New York, NY: Cambridge University Press; 2000.

28. Costello EJ, Egger H, Angold A. 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. J Am Acad Child Adolesc Psychiatry. 2005;44(10):972–986. [PubMed]

29. Cole JW. The National Cancer Act: the first five years. Yale J Biol Med. 1977;50(3):229–231. [PMC free article] [PubMed]

30. Eyre HJ. National Cancer Act 1971–1996. National Cancer Institute/American Cancer Society relationship—June 1996. Cancer. 1996;78(12):2609–2610. [PubMed]

31. Rauscher FJ., Jr Proceedings: the National Cancer Program and the National Cancer Act of 1971. Natl Cancer Inst Monogr. 1974;40:3–6. [PubMed]

32. Balshem AM, Amsel Z, Workman S, Engstrom P. The experience of cancer warning signs in a general population and its implications for public education. Prog Clin Biol Res. 1988;278:77–89. [PubMed]

33. Hintze B. History of the Seven Cancer Warning Signals. Atlanta, GA: American Cancer Society; 1992.

34. Nichols BS, Misra R, Alexy B. Cancer detection: how effective is public education? Cancer Nurs. 1996;19(2):98–103. [PubMed]

35. US Department of Health and Human Services Achieving the Promise: Transforming Mental Health Care in America—The President's New Freedom Commission on Mental Health Report. Bethesda, MD: US Department of Health and Human Services; 2003.

36. Lahey BB, Flagg EW, Bird HR, et al. The NIMH Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) study: background and methodology. J Am Acad Child Adolesc Psychiatry. 1996;35(7):855–864. [PubMed]

37. Ge X, Natsuaki MN, Conger RD. Trajectories of depressive symptoms and stressful life events among male and female adolescents in divorced and nondivorced families. Dev Psychopathol. 2006;18(1):253–273. [PubMed]

38. Roberts RE, Roberts CR, Xing Y. Rates of DSM-IV psychiatric disorders among adolescents in a large metropolitan area. J Psychiatr Res. 2007;41(11):959–967. [PMC free article] [PubMed]

39. Bird HR, Canino GJ, Davies M, et al. A study of disruptive behavior disorders in Puerto Rican youth: I. Background, design, and survey methods. J Am Acad Child Adolesc Psychiatry. 2006;45(9):1032–1041. [PubMed]

40. Bird HR, Davies M, Duarte CS, Shen S, Loeber R, Canino GJ. A study of disruptive behavior disorders in Puerto Rican youth: II. Baseline prevalence, comorbidity, and correlates in two sites. J Am Acad Child Adolesc Psychiatry. 2006;45(9):1042–1053. [PubMed]

41. Shaffer D, Fisher P, Lucas CP, Dulcan MK, Schwab-Stone ME. NIMH Diagnostic Interview Schedule for Children Version IV (NIMH DISC-IV): description, differences from previous versions, and reliability of some common diagnoses. J Am Acad Child Adolesc Psychiatry. 2000;39(1):28–38. [PubMed]

42. Shaffer D, Fisher P, Dulcan MK, et al. The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): description, acceptability, prevalence rates, and performance in the MECA Study. Methods for the Epidemiology of Child and Adolescent Mental Disorders Study. J Am Acad Child Adolesc Psychiatry. 1996;35(7):865–877. [PubMed]

43. Bird HR, Gould MS, Yager T, Staghezza B, Canino G. Risk factors for maladjustment in Puerto Rican children. J Am Acad Child Adolesc Psychiatry. 1989;28(6):847–850. [PubMed]

44. Bird HR, Gould MS, Staghezza BM. Patterns of diagnostic comorbidity in a community sample of children aged 9 through 16 years. J Am Acad Child Adolesc Psychiatry. 1993;32(2):361–368. [PubMed]

45. Flisher AJ, Kramer RA, Hoven CW, et al. Risk behavior in a community sample of children and adolescents. J Am Acad Child Adolesc Psychiatry. 2000;39(7):881–887. [PubMed]

46. Goodman SH, Hoven CW, Narrow WE, et al. Measurement of risk for mental disorders and competence in a psychiatric epidemiologic community survey: the National Institute of Mental Health Methods for the Epidemiology of Child and Adolescent Mental Disorders (MECA) study. Soc Psychiatry Psychiatr Epidemiol. 1998;33(4):162–173. [PubMed]

47. Roberts RE, Roberts CR, Chen IG. Impact of insomnia on future functioning of adolescents. J Psychosom Res. 2002;53(1):561–569. [PubMed]

48. Ge X, Conger RD. Adjustment problems and emerging personality characteristics from early to late adolescence. Am J Community Psychol. 1999;27(3):429–459. [PubMed]

49. Costello EJ, Angold A, Burns BJ, et al. The Great Smoky Mountains Study of Youth: goals, design, methods, and the prevalence of DSM-III-R disorders. Arch Gen Psychiatry. 1996;53(12):1129–1136. [PubMed]

50. Kessler RC, McGonagle KA, Zhao S, et al. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States: results from the National Comorbidity Survey. Arch Gen Psychiatry. 1994;51(1):8–19. [PubMed]

51. Barrett AE, Raskin White H. Trajectories of gender role orientations in adolescence and early adulthood: a prospective study of the mental health effects of masculinity and femininity. J Health Soc Behav. 2002;43(4):451–468. [PubMed]

52. Pumariega AJ, Johnson NP, Sheridan D, Cuffe SP. The influence of race and gender on depressive and substance abuse symptoms in high-risk adolescents. Cult Divers Ment Health. 1996;2(2):115–123. [PubMed]

53. Sturm R, Ringel JS, Andreyeva T. Geographic disparities in children's mental health care. Pediatrics. 2003;112(4). Available at: www.pediatrics.org/cgi/content/full/112/4/e308. [PubMed]

54. Kataoka SH, Zhang L, Wells KB. Unmet need for mental health care among U.S. children: variation by ethnicity and insurance status. Am J Psychiatry. 2002;159(9):1548–1555. [PubMed]

Articles from Pediatrics are provided here courtesy of
American Academy of Pediatrics