Training clinicians in mental health communication skills: Impact on primary care utilization

doi:10.1016/j.acap.2010.06.012

Acad Pediatr. Author manuscript; available in PMC 2011 September 1.

Published in final edited form as:

Acad Pediatr. 2010 Sep–Oct; 10(5): 346–352.

Published online 2010 August 4. doi: 10.1016/j.acap.2010.06.012

PMCID: PMC2933967

NIHMSID: NIHMS217777

Training clinicians in mental health communication skills: Impact on primary care utilization

Anne Gadomski, MD, MPH, Lawrence S. Wissow, MD, MPH, Eric Slade, PhD, and Paul Jenkins, PhD

Anne Gadomski, Research Institute, Bassett Healthcare, Cooperstown, NY;

Contributor Information.

Correspondence: Anne Gadomski, MD, MPH, Bassett Healthcare Research Institute, One Atwell Road, Cooperstown, NY 13326, Email: anne.gadomski/at/bassett.org, 607 547 3066 (o), 607 547 6914 (f)

The publisher's final edited version of this article is available at Acad Pediatr

Publisher's Disclaimer

Abstract

Objective

Although it is known that children with mental health problems utilize primary care services more than most other children, it is unknown how addressing mental health problems in primary care affects children’s subsequent services utilization. This study measures primary care utilization in the context of a randomized trial of a communication skills training program for primary care clinicians that had a positive impact on child mental health outcomes.

Methods

From 2002 to 2005, 48 pediatric primary care clinicians at 13 sites in rural upstate New York, urban Maryland, and Washington, DC were randomized to in-office training or to a control group. Consecutive primary care patients between the ages of 5 and 16 years were screened for mental health problems, as indicated by a possible or probable score on the Strengths and Difficulties Questionnaire (SDQ). For 397 screened children, primary care visits during the next 6 months were identified using chart review and administrative databases. Using generalized estimating equation regression to account for clustering at the clinician level, primary care utilization was compared by study group and SDQ status.

Results

The number of primary care visits to the trained clinicians did not differ significantly from those made to control clinicians (2.5 for both groups, p=0.63). Children with possible or probable SDQ scores made, on average, 0.38 or 0.65 more visits on a per-child basis, respectively, during the six-month follow-up period than SDQ unlikely children (p-value=0.0002).

Conclusions

Seeing a trained clinician did not increase subsequent primary care utilization. However, primary care utilization was greater among children with mental health problems as measured by the SDQ. Addressing children’s mental health in primary care does not increase the primary care visit burden. Research on overall health services utilization is needed.

Keywords: Strengths and Difficulties Questionnaire, primary care utilization, child mental health

Introduction

Understanding the impact of managing mental health problems on pediatricians’ work load is a critical element in decisions about the role of primary care in pediatric mental health service delivery. Several studies show that physicians are not screening and addressing mental health problems because of their concerns about the time and complexity of mental health visits as well as not being reimbursed appropriately. ¹^,²^,³^,⁴^,⁵ To date, however, evidence that might help predict the impact of treating more mental health in primary care is mixed.

It is known that the average cost of care is higher for children with mental health problems. Higher costs arising from increased utilization, such as higher acute primary, emergency and specialty health care utilization, are associated with child psychosocial morbidity, as well as maternal distress, parental depression, family conflict and parent-reported behavioral problems. ⁶^,⁷^,⁸^,⁹^,¹⁰^,¹¹^,¹² These higher costs have been associated with mental health problems regardless of the clinical setting they are identified in. As a result, the average health care costs for children with externalizing or internalizing symptoms can be at least twice the level of the health care costs for the clinical population as a whole.¹³^,¹⁴^,¹⁵^,¹⁶

There is less definitive evidence on how identifying and treating mental health problems in primary care affects primary care utilization. Identifying mental health problems in primary care might reduce primary care utilization when mental health issues are co-morbid with, or are misidentified as, medical problems.¹⁷^,¹⁸^,¹⁹ Alternatively, addressing previously undisclosed or untreated mental health problems could increase the number of primary care visits, their length and complexity, or the amount of provider work (e.g. calls to schools, review of standardized assessments) required outside of visits. One study found that adult primary care visits with psychosocial content are only slightly longer than those with strictly biomedical content.²⁰ A study that identified new adult patients with psychosocial distress found that it led to increases in ambulatory mental health visits but, within the statistical limits of the study, did not increase scheduled or non-scheduled non-mental health visits.²¹ A British study that trained general practitioners to work with adults with somatisation disorder found that it did not change the use of primary care services but significantly reduced the use of medical specialty care.²²

The analyses in this paper use data from a trial that tested a training program for pediatric primary care providers designed to improve their management of children’s mental health problems.²³ The program taught a set of communication skills for use in routine visits, including engaging parents and children and seeking an acceptable treatment plan. After the training, the providers’ patients were screened for existing mental health problems and followed for six months. In an ‘intent to treat’ analysis, parents seeing trained providers had greater reductions in distress during the follow-up period than parents seeing control providers, and the functioning of minority children seeing trained providers improved more than that of minority children seeing control providers. ²³ A subsequent analysis that compared outcomes to training uptake found that training was associated with improvements in child function and symptoms among the overall child study population. ²⁴ However, it is not known how the training affected utilization.

The goal of the analyses presented here was to examine the impact of the training on primary care utilization as measured by the number and types of primary care visits children made during the follow-up period. We hypothesized that children seeing trained providers would have less utilization than children seeing control providers, even for those children who had identified mental health problems.

Methods

Study design

The study data come from a cluster randomized, intervention trial is described in more detail elsewhere.²³^,²⁴ The outcome measure is primary care utilization, defined by number of visits as measured by billing data and chart review. Participants were clinicians in the intervention trial and patients of those clinicians.

Intervention

We developed a training program designed to help primary care providers learn a set of communication skills and apply them in routine visits.²³ Skills included engaging parents and children in the diagnostic and treatment process while addressing barriers to doing so; managing negative affect; promoting disclosure; and facilitating problem-solving.²⁴ Training was delivered to small groups (range 2–7) on site. It included 3 cycles, each containing a 50–60 minute structured group discussion followed by individual 10-minute practice visits, keyed to the session content, with standardized patients.

Provider participants

Pediatric primary care providers were drawn from 13 study sites. Rural sites included a solo practice, a hospital-based practice, four free-standing multi-specialty offices, and a practice of two family nurse practitioners. Urban sites included four community-based multi-provider clinics, a multi-provider private practice, a hospital-based family practice, and an independent center caring for families of recent Latino immigrants. Of 69 clinicians at the participating sites, 58 agreed to be randomized (84%); 27 became controls and 31 received training. Of the 58, 48 clinicians (19 controls and 29 trained), supplied the utilization data included in this analysis. Demographic characteristics did not different significantly between control and trained clinicians: 15 (31%) were trained in family practice, 33 (69%) in pediatrics and the majority (n=37, 77%) were medical doctors.

Child participants

From December 2002 to August 2005, children seeing clinicians participating in the training study were recruited before their primary care visit with that clinician. The recruitment target was ten children for each participating clinician. Interviewers systematically approached families in waiting areas during each recruitment day. Families were potentially eligible if the child scheduled for a visit was between 5 and 16 years old, not acutely ill or reporting pain and spoke English (or Spanish at the immigrant center). At the time of the index visit, parents supplied sociodemographic data and reported on their child’s use of mental health services in the prior six months.²³^,²⁴

Strengths and Difficulties Questionnaire

The Strengths and Difficulties Questionnaire (SDQ) is a widely-used brief survey of emotional and behavior symptoms and associated functional impairment. ²⁵^,²⁶^,²⁷^,²⁸ The SDQ has 25 symptom items and 8 that measure function (distress, interference with home, peer, school, and leisure activities) and burden on the family. ²⁹^,³⁰ For children 5–10 years old, SDQ reports were completed by parents and teachers; for youth 11–16, reports were completed by parents and youth themselves. Reports were combined using a standard algorithm that classifies children as either unlikely, possibly, or probably having a mental health problem. SDQ scores for each respondent (parent, teacher, and youth) by gender and age are described in detail elsewhere.³¹

Sample Selection

Of 871 families approached, 819 families could be screened for participation in the outcome evaluation (94% of families initially approached). Children were eligible for participation in the outcome evaluation if they scored in the “possible” or “probable” range on the Strengths and Difficulties Questionnaire (SDQ) at the time of screening, or if they were nominated by their clinician as likely to have an emotional or behavioral program. A total of 397 children/youth (150 SDQ possible, 147, probable and 110 clinician nominated) were followed by telephone at two weeks, three and six months following the index visit. These children were seen by 48 primary care clinicians, 19 in the control group and 29 in the treatment group.

Primary care utilization

None of the participating sites had a full electronic medical record system at the time of the study. To collect utilization data, we first asked each site to prepare, from registration and billing systems, a listing of all visits by enrolled study patients from the time of their index visit until 6 months later. These lists contained the patient’s name, medical record number, and for each visit the date, the clinician involved, and the CPT and ICD-9 codes designating the service(s) rendered and the diagnoses assigned. In addition, we abstracted paper medical records for each patient for the matching study period noting use of psychotropic medications and psychiatric referrals.

The study sites varied a great deal in terms of their position within health care systems, ranging from a free-standing community practice to being part of an integrated regional system. At most sites, the data available was restricted to primary care. To be consistent across sites, we restricted our main analyses to primary care visits, defined as acute or return medical visits and health maintenance visits, occurring in pediatrics or family practice. This included the following CPT codes ³²: 99211–5 (office visits, established patients) and 99201–4 (office visits, new patients), and 99393–4 (health maintenance, ages 5–17). When CPT codes were missing from electronic records (n =29 visits by 25 children), chart documentation was used to assign the visit type. The rural network provided specialty, emergency, and inpatient care and did allow enumeration of all visit types. Therefore, data from this network were used to analyze a subset of the data to explore the impact of training on overall utilization.

Figure 1 shows how visits were selected to measure utilization. The electronic data were converted to a common format, merged across sites, and examined for duplicates. Next, records were removed if visits were more than 180 days from the child/youth’s index visit, or if a child’s baseline SDQ data were missing. For 68 children, the recruitment date that should have been the date of the index visit did not match the first service date in the electronic data. For these children, an index office visit of the most common length (15 minutes, CPT = 99213) was added to the visit count as the child had to have had an index visit in order to be included in the study. There was no difference between the study groups in the frequency of this discrepancy (16.7% trained clinician group vs 17.7% control group, p = ns). For 18 children, no administrative data were available and no chart could be found. The percentage of children without visits of any kind documented did not differ significantly by study group (11.6% in the trained clinician group vs. 9.1% in the control group, Rao-Scott p =0.34). Each of these children was also assigned one index primary care visit to match the visit at which they had been enrolled. The final number of visits included in this analysis is 987 medical or health maintenance primary care visits.

Figure 1

Flow diagram for visits included in the analysis

In order to measure a child’s exposure to a trained clinician during the six-month follow-up period, a continuity ratio was computed for each patient. This ratio is equivalent to the number of primary care visits a child had with a trained clinician divided by the total number of visits (with clinicians who were in the trained, control or in neither group) during the follow-up period.

Statistical procedures

Bivariate analysis of study groups and SDQ subsets with respect to several covariates of health care utilization was performed and assessed for statistical significance, and adjusted for clustering by clinician, using the Rao-Scott chi square test for percentages or the cluster-adjusted t-test (clustered one-way ANOVA) for means. The unadjusted Fisher’s Exact test was used for instances where Rao-Scott was not possible (i.e. where one group had a zero cell count (0%)). Covariates that were tested for associations with utilization included child’s age, maternal age, sex, race/ethnicity, insurance, rural vs. urban residence, and receiving mental health counseling at the time of the index visit. Because chronic pediatric conditions can increase utilization, ³³ ICD-9 codes for chronic co-morbidities (asthma 493.9, allergic rhinitis 477.9, diabetes 250.0, seizures 345.0–345.9, sickle cell disease 282.6) were each tested as covariates. A p value of <0.05 was considered statistically significant. Statistical analyses were performed with SAS version 9.1.3 (Cary, NC) using the SURVEYREG and GENMOD procedures to adjust for clustering by provider.

Human subjects

The study was approved by institutional review boards of the Johns Hopkins Bloomberg School of Public Health, the Mary Imogene Bassett Hospital, the Children’s National Medical Center, and the Medstar Research Institute. Parents and clinicians provided written consent; youth 11–16 provided written assent. Parents and youth were told that participation was part of the evaluation of a training program “designed to help doctors give better care when children have problems with their behavior and mood.” They were not told if their child’s primary care provider had received training. They were told the information they gave to the study would not be shared with clinicians, but that they were free to disclose or discuss it. Clinicians were aware that some of their patients would be followed for six months but were unaware of which children would be included in follow-up.

Results

Success of randomization/tests for confounding

Children who saw trained clinicians were more likely to be female (49 vs. 39%, p=.02) and slightly older (10.7 vs. 10.1 years, p=.08) (see Table 1). Otherwise there was no difference between the study groups in SDQ classification, other demographics, insurance, prior mental health counseling or co-morbidity, or chronic disease ICD –9 codes. Parents of children seeing control and trained clinicians did not differ by age or educational level.

Table 1

Baseline comparison of study groups including children with primary care visits in the six month follow-up period (adjusted for clustering by clinician); n=397 children.

Continuity

Children whose index visit was with a trained clinician saw a trained clinician for 67% of their subsequent primary care visits, whereas children whose index visit was with a control clinician saw a control or other untrained primary care clinician for 92% of visits.

Confounding

Because of the slight age and sex imbalances at baseline between the intervention and control groups, the possibility of confounding was evaluated. There was no relationship between age and SDQ status (p=0.16) or age and the number of primary care visits (p=0.59). In addition, there was no relationship between the child’s sex and the number of primary care visits (p=0.99).

Utilization by SDQ status

Utilization of primary care varied by baseline SDQ status (Table 2). Compared to children classified as SDQ unlikely (but nominated by clinicians as having problems), children classified as SDQ possible made, on average, 0.38 more primary care visits (p=0.01), while children classified as SDQ probable made 0.65 more such visits during the six-month follow-up period (p < 0.0001, with overall p-value of 0.0002). The standard errors of the means are small, i.e. about 1/20^th of the mean, suggesting little dispersion and implying that many patients had just one more visit beyond the index as opposed to a few patients coming in more frequently. Specifically, for SDQ unlikely, the range of visits was 1–6 with 73% of observations being a 1 or 2 (mean = 2.1 vs median = 2.0). For SDQ possible, the range of visits was 1–9 with 56% of observations being a 1 or 2 (mean = 2.47 vs median = 2.0). For SDQ probable, the range of visits was 1–9 with 57% of observations being a 1 or 2 (mean = 2.75 vs median = 2.0).

Table 2

Mean number of primary care visits by SDQ status and study group (adjusted for clustering by clinician) with no significant interaction between SDQ status and study group (p=0.26); n =397 children.

Over the six month follow-up period, clinician-documented psychotropic medication prescription also varied significantly by SDQ status. Among children classified as SDQ unlikely, 11% had documentation of one or more psychotrophic medication prescriptions compared to 14% in the SDQ possible and 34% in the SDQ probable groups (Rao-Scott p < 0.0001). Similarly, there was a significant relationship between SDQ status and receipt of counseling before the index visit (p<0.0001). Psychiatric referral did not vary significantly by SDQ status, however the percentage of children referred was low (1 % SDQ unlikely, 3.6% SDQ possible, 5.8% SDQ probable, Rao- Scott p = 0.16). There was no relationship between SDQ status and having any chronic disease (p = 0.30).

Therefore, receipt of prior counseling was added to the model predicting primary care visits. When prior counseling was added to this model, there was no relationship between the number of primary care visits and prior counseling (p=0.84), nor was there a relationship with an interaction term for SDQ status and prior counseling (p=0.84). SDQ status remained significantly associated with the number of primary care visit in this model (p=0.008).

Impact of training on primary care utilization

The mean number of visits for children seeing trained clinicians was nearly identical to the mean among children seeing control clinicians (2.46 vs. 2.52) (Table 2). Table 3 shows the relationship of clinician training status to the mean number of visits per child for various types of visits. There were no statistically significant differences for any of the four levels of office visits (ranging from brief through 40 minutes), or for health maintenance visits (of children 5–11 and 12–17). There were no statistically significant differences in the proportion of children/youth who had visits for asthma, allergic rhinitis, or diabetes. There was a trend for trained clinicians to prescribe fewer psychotropic medications than control clinicians. Trained clinicians prescribed one or more psychotropic medications to 17% of the children they saw compared to 26% in the control group (Rao-Scott p =0.11). Trained clinicians documented psychiatric referral for 3.7% of the children they saw compared to 3.9% in the control group (Rao-Scott p =0.93).

Table 3

Comparison, by study group, of mean number of primary care visits stratified by CPT code, well child visits and visits for select chronic diseases (adjusted for clustering by clinician); n =397 children.

Subset analysis of all visit types

A subset analysis of all visits occurring in the integrated rural health network showed similar results when all visit types are included, although, as would be expected, the mean number of visits was higher. Of the 7 clinical sites included, there were 21 clinicians, 188 children and 593 visits of all types during the study period. The mean number of all visits by treatment group was not significantly different, i.e. 3.57 intervention versus 3.38 for control, p = 0.605, adjusted for clustering by clinician. However, the mean number of all visits did differ by SDQ status as follows: 2.98 for SDQ unlikely, 3.71 for SDQ possible and 3.64 for SDQ probable (p = 0.026). Further stratification by visit type was not possible due to sample size limitations.

Discussion

Training clinicians in a core set of mental health-related communication skills does not change the number and type of primary care visits among the children they care for, and therefore does not appear to increase the primary care visit burden. Because clinicians were not aware of which patients were enrolled in the study, they were unlikely to bias results by encouraging or discouraging follow-up visits. However children with mental health problems, as measured by the SDQ, still made more visits than those who were SDQ negative.

Training did not change the distribution of visits by CPT code, so, at least to the extent that it is represented by CPT codes, visit length and complexity did not differ between the study groups. Roter and colleagues’ study of communication training for adult primary care providers found that training did not increase the length of visits.³⁴ These findings may somewhat alleviate a common concern that, if primary care providers address mental health problems, they will be opening “Pandora’s box”, i.e. unleashing psychosocial issues that the provider has neither the time nor the skills to address.³⁵

Study limitations include using the SDQ as a screening instrument for mental health problems. Although the SDQ is a reliable and valid screen for identifying children who meet DSM IV criteria, it is not a diagnostic test. ²⁵^,³⁶ Clinical diagnosis of mental health problems requires longer diagnostic instruments, that are less feasible in routine primary care practice. Misclassification of mental problems by the SDQ in this study would lead to more conservative estimates, and thus could have partially obscured differences.

Other limitations pertain to the variation in medical documentation and administrative databases across the study sites. As described in the methods, we could not consistently capture utilization outside of primary care except for a subset of the participants. Therefore, our visit counts underestimates overall services utilization; however, this limitation applies equally to the study groups as well as the SDQ subsets. Practice variations in coding among sites may have also occurred, but again this limitation would have applied equally to the study groups and the SDQ subsets as randomization occurred within each site.

While our efficacy study showed that the training intervention increased skill uptake among providers ²⁴ and that it was associated with a clinical benefit for patients ²³, we still do not know the mechanism by which training was translated into benefit. The analysis we report here suggests that the mechanism does not involve more or more complex primary care visits, but we do not know about how the content of visits might have changed. Future studies will need to include more comprehensive capture of service utilization before and after exposure to the intervention, and may benefit from longer follow-up periods. Both could be facilitated by increasing adoption of electronic medical records in primary care. Future studies may also need to find ways of directly documenting what trained clinicians do differently as our study was only able to observe changes in clinicians’ interactions with standardized patients.²³^,²⁴

Acknowledgments

We are grateful to the families, clinicians and research assistants who took part in the study. The study was funded by grant RO1-MH62469 from National Institute of Mental Health. This study has been presented in part at the annual meeting of the Pediatric Academic Societies, San Francisco, May 2006.

Footnotes

Potential conflict of interest and corporate sponsors: none

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Contributor Information

Anne Gadomski, Research Institute, Bassett Healthcare, Cooperstown, NY.

Lawrence S. Wissow, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD.

Eric Slade, U.S. Department of Veterans Affairs, VISN5 Mental Illness Research and Education Clinical Center Baltimore, MD.

Paul Jenkins, Computing Center, Research Institute, Bassett Healthcare, Cooperstown, NY.

References

1. Perrin E, Stancin T. A continuing dilemma: whether and how to screen for concerns about children’s behavior in primary care settings. Pediatr Rev. 2002;23:264–275. [PubMed]

2. Cooper S, Valleley RJ, Polaha J, Begeny J, Evans JH. Running out of time: physician management of behavioral health concerns in rural pediatric primary care. Pediatrics. 2006;118(1):e132–8. [PubMed]

3. Heneghan A, Garner AS, Storfer-Isser A, et al. Pediatricians’role in providing mental health care for children and adolescents: do pediatricians and child and adolescent psychiatrists agree? J Dev Behav Pediatr. 2008;29(4):262–9. [PubMed]

4. Horwitz SM, Kelleher KJ, Stein RE, et al. Barriers to the Identification and Management of Psychosocial Issues in Children and Maternal Depression. Pediatrics. 2007;119(1):e208–e218. [PubMed]

5. American Academy of Child and Adolescent Psychiatry, Committee on Health care Access and Economics; American Academy of Pediatrics, Task Force on Mental Health. Improving mental health services in primary care: reducing administrative and financial barriers to access and collaboration. Pediatrics. 2009;123(4):1248–51. [PubMed]

6. Leibson CL, Katusic SK, Barbaresi WJ, et al. Use and costs of medical care for children and adolescents with and without attention-deficit/hyperactivity disorder. JAMA. 2001;285:60–66. [PubMed]

7. Kelleher K, Starfield B. Health care use by children receiving mental health services. Pediatrics. 1990;85(1):114–8. [PubMed]

8. Riley AW, Finney JW, Mellits ED, Starfield B, Kidwell S, Quaskey S. Determinants of children’s health care use: An investigation of psychosocial factors. Med Care. 1993;31(9):767–83. [PubMed]

9. Zuckerman B, Moore KA, Glei D. Association between child behavior problems and frequent physician visits. Arch Pediatr Adolesc Med. 1996;150:146–153. [PubMed]

10. Olfson M, Marcus SC, Druss B, Alan Pincus H, Weissman MM. Parental depression, child mental health problems, and health care utilization. Med Care. 2003;41(6):716–21. [PubMed]

11. Minkovitz CS, Strobino D, Scharfstein D, et al. Maternal depressive symptoms and children’s receipt of health care in the first 3 years of life. Pediatrics. 2005;115(2):306–14. [PubMed]

12. Sills MR, Shetterly S, Xu S, Magid D, Kempe A. Association between parental depression and children’s health care use. Pediatrics. 2007;119(4):e829–36. [PubMed]

13. Bernal P, Estroff DB, Abourdarham JF, Murphy M, Keller A, Jellinek MS. Psychosocial morbidity: the economic burden in a pediatric health maintenance organization sample. Arch Pediatr Adolesc Med. 2000;154(3):261–6. [PubMed]

14. Guervara J, Lozano P, Wickizer T, Mell L, Gephart H. Utilization and Cost of Health Care Services for Children With Attention-Deficit/Hyperactivity Disorder. Pediatrics. 2001;108:71–78. [PubMed]

15. Mandell DS, Guevara JP, Rostain AL, Hadley TR. Medical expenditures among children with psychiatric disorders in a Medicaid population. Psychiatric Services. 2003;54(4):465–7. [PubMed]

16. Chan E, Zhan C, Homer CJ. Health care use and costs for children with attention-deficit/hyperactivity disorder: national estimates from the medical expenditure panel survey. Arch Pediatr Adolesc Med. 2002;156(5):504–11. [PubMed]

17. Costello EJ, Edelbrock C, Costello AJ, Dulcan MK, Burns BJ, Brent D. Psychopathology in pediatric primary care: The new hidden morbidity. Pediatrics. 1988;82(3 Pt 2):415–24. [PubMed]

18. Olfson M, Sing M, Schlesinger HJ. Mental health/medical care cost offsets: opportunities for managed care. Health Aff (Millwood) 1999;18(2):79–90. [PubMed]

19. Jones D, Godwin J, Dodge KA, et al. Impact of the Fast Track Prevention Program on Health Services Use by Conduct-Problem Youth. Pediatrics. 2010;125 (1):e130–e136. [PMC free article] [PubMed]

20. Roter DL, Stewart M, Putnam SM, Lipkin M, Jr, Stiles W, Inui TS. Communication patterns of primary care physicians. JAMA. 1997;277(4):350–6. [PubMed]

21. Klerman GL, Budman S, Berwick D, Weissman MM, Damico-White J, Demby A, Feldstein M. Efficacy of a brief psychosocial intervention for symptoms of stress and distress among patients in primary care. Med Care. 1987;25(11):1078–88. [PubMed]

22. Morriss R, Gask L, Ronalds C, Downes-Grainger E, Thompson H, Leese B, Goldberg D. Cost-effectiveness of a new treatment for somatized mental disorder taught to GPs. Fam Pract. 1998;15(2):119–25. [PubMed]

23. Wissow LS, Gadomski A, Roter D, et al. Improving child and parent mental health in primary care: a cluster-randomized trial of communication skills training. Pediatrics. 2008;121(2):266–75. [PubMed]

24. Wissow LS, Gadomski A, Roter D, Larson S, Lewis B, Brown J. Aspects of mental health communication skills training that predict parent and child outcomes in pediatric primary care. Patient Education Counseling. 2010 May 4; [Epub ahead of print] [PMC free article] [PubMed]

25. Goodman A, Goodman R. Strengths and Difficulties Questionnaire as a dimensional measure of child mental health. J Am Acad Child Adolesc Psychiatry. 2009;48(4):400–3. [PubMed]

26. Goodman R, Scott S. Comparing the strengths and difficulties questionnaire and the child behavior checklist: Is small beautiful? J Abnorm Child Psychol. 1999;27(1):17–24. [PubMed]

27. Goodman R. The extended version of the strengths and difficulties questionnaire as a guide to child psychiatric caseness and consequent burden. J Child Psychol Psychiatry. 1999;40(5):791–9. [PubMed]

28. Vostanis P. Strengths and Difficulties Questionnaire: research and clinical applications. Curr Opin Psychiatry. 2006 Jul;19(4):367–72. [PubMed]

29. Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40(11):1337–45. [PubMed]

30. Bourdon KH, Goodman R, Rae DS, et al. The strengths and difficulties questionnaire: U.S. normative data and psychometric properties. J Am Acad Child Adolesc Psychiatry. 2005;44(6):557–64. [PubMed]

31. Brown JD, Wissow LS, Gadomski A, et al. Parent and teacher mental health ratings of children using primary-care services: inter-rater agreement and implications for mental health screening. Ambulatory Pediatrics. 2006;6(6):347–51. [PMC free article] [PubMed]

32. American Medical Association. CPT 2006: Current procedural terminology. Chicago: AMA Press; 2005.

33. Westbom L, Kornfalt R. Utilization of primary care versus specialized care in children with and without chronic illness. Acta Paediatr. 1991;80:534–541. [PubMed]

34. Roter DL, Hall JA, Kern DE, et al. Improving physicians’ interviewing skills and reducing patients’ emotional distress. Archives of Internal Medicine. 1995;155(17):1877–1884. [PubMed]

35. Sugg NK, Inui T. Primary care physicians’ response to domestic violence—opening Pandora’s box. JAMA. 1992;267(23):3157–60. [PubMed]

36. Goodman R, Ford T, Simmons H. Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. British Journal of Psychology. 2000;177:534–539. [PubMed]