Incorporating CanMEDS and subspecialty training into paediatric residency programs: Why are we still deficient?

Paediatr Child Health. 2012 January; 17(1): e7–e11.

PMCID: PMC3276534

Incorporating CanMEDS and subspecialty training into paediatric residency programs: Why are we still deficient?

Gautam Kumar, MD,¹ Andrew Ni, PhD,² Sarah E Lawrence, MD,³ and Asif Doja, MEd MD⁴

¹Division of Paediatrics, Sudbury Regional Hospital, Sudbury, Ontario;

²Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA;

³Division of Endocrinology, Children’s Hospital of Eastern Ontario;

⁴Division of Neurology, Children’s Hospital of Eastern Ontario, Ottawa, Ontario

Correspondence: Dr Asif Doja, Children’s Hospital of Eastern Ontario, 401 Smyth Road, Ottawa, Ontario K1H 8L1. Telephone 617-737-7600, email adoja/at/cheo.on.ca

Accepted February 7, 2011.

Abstract

BACKGROUND:

The Royal College of Physician and Surgeons of Canada mandates that paediatric training programs in Canada incorporate subspecialty training and the teaching and evaluation of the seven CanMEDS roles into their curriculum. The literature suggests that newly practicing paediatricians feel inadequately prepared in many subspecialties and CanMEDS roles.

HYPOTHESIS:

That either current training programs underestimate the importance of these areas for future practice, or that residents themselves feel that these areas are less important.

METHOD:

An online survey of Canadian paediatric residents and paediatric residency program directors was conducted to determine their views on various subspecialty areas and CanMEDS roles.

RESULTS:

Fourteen of 16 Canadian paediatric programs participated, and 127 of 486 (26%) paediatric residents completed the survey. Overall, trainees were satisfied with their current training (86%), and 90% believed they would be adequately prepared for independent practice. Forty-six residents (40%) believed training programs place less importance on 10 of the subspecialties that newly practicing paediatricians felt less comfortable with (from a previous study conducted in 2006). However, at least 25% of residents themselves placed less importance on nine of these 10 areas. Residents also place less importance on two CanMEDS competencies which practicing paediatricians felt less comfortable with, including the medical aspects of palliative care (medical expert) and managing an efficient office practice (manager).

CONCLUSIONS:

Residents and programs place less importance on specific areas of paediatric training, thus creating potential deficiencies in graduating paediatricians. Promotion of these topics during training may better prepare residents for future practice.

Keywords: Internship and residency, Medical education, Pediatrics/education, Questionnaires

Résumé

HISTORIQUE :

Le Collège royal des médecins et chirurgiens du Canada mandate les programmes de formation en pédiatrie du Canada d’intégrer à leur cursus la formation en surspécialité et l’enseignement et l’évaluation des sept rôles CanMEDS. Selon les publications, les pédiatres qui commencent à exercer se sentent mal préparés dans de nombreuses surspécialités et plusieurs rôles CanMEDS.

HYPOTHÈSE :

Les chercheurs postulent que les programmes de formation actuels sous-estiment l’importance de ces domaines pour la pratique future ou que les résidents eux-mêmes y accordent moins d’importance.

MÉTHODOLOGIE :

Les chercheurs ont tenu un sondage virtuel auprès des résidents en pédiatrie canadiens et des directeurs de programmes de pédiatrie afin de déterminer leur point de vue sur les divers domaines de surspécialité et sur les rôles CanMEDS.

RÉSULTATS :

Quatorze des 16 programmes de pédiatrie canadiens et 127 des 486 (26 %) résidents en pédiatrie ont rempli le sondage. L’ensemble des stagiaires se disait satisfait de leur formation (86 %), et 90 % pensaient qu’ils étaient bien préparés à une pratique autonome. Quarante-six résidents (40 %) pensaient que les programmes de formation accordent moins d’importance à dix des surspécialités dans lesquelles les nouveaux pédiatres en exercice se sentent moins à l’aise (selon une étude de 2006 menée par Lieberman et Hilliard). Cependant, au moins 25 % des résidents eux-mêmes accordent moins d’importance à neuf de ces dix domaines, de même qu’à deux compétences CanMEDS dans lesquelles les pédiatres en exercice se sentent moins à l’aise, soit les aspects médicaux des soins palliatifs (expert médical) et la gestion d’un bureau de pratique efficace (gestionnaire).

CONCLUSIONS:

Les résidents et les programmes accordent moins d’importance à des domaines précis de la formation en pédiatrie, ce qui risque de créer des lacunes chez les pédiatres diplômés. La promotion de ces sujets pendant la formation pourrait mieux préparer les résidents à leur future pratique.

Recently, the core components that comprise the field of paediatrics have continued to expand, forcing training programs to readjust their curricula to offer the most up to date education. Each paediatric residency program in Canada has developed their own curriculum, individualized to the strengths and limitations of each institution, while simultaneously needing to meet the fundamental objectives set out by the Royal College of Physicians and Surgeons of Canada (RCPSC) (1). Paediatric residency programs also need to expose trainees to the various subspecialities in paediatric medicine; the exact frequency and duration of rotations that one should complete in these subspecialties is unclear.

Given this, it is unclear whether residents graduating from paediatric training programs in Canada have obtained the core skills to begin independent practice. A recent study by Lieberman and Hilliard (2) surveyed newly graduated paediatricians to determine how prepared they felt in various subspecialties and in the seven CanMEDS competencies. These paediatricians identified themselves as feeling less than adequately trained in 10 subspecialty areas, including gynecology (73% felt training was inadequate), child psychiatry (64%), behavioural paediatrics (58%), surgical subspecialties (54%), adolescent medicine (46%), dermatology (44%), nutrition (43%), allergy and immunology (39%), metabolics (36%) and genetics (31%). There were no differences in overall perceptions of preparedness between community-based and university-based paediatricians. However, compared with paediatric subspecialists, general paediatricians were less satisfied with respect to their training in child development behavioural paediatrics and child psychiatry. More university-based paediatricians than community-based paediatricians felt they had too much training in neonatology.

The RCPSC has adopted the CanMEDS framework to ensure that essential physician competencies (medical expert, communicator, collaborator, scholar, health advocate, manager and professional) (3) are addressed in residency training. This framework has been integrated into the RCPSC accreditation standards, objectives of training, and final in-training evaluations. Lieberman and Hilliard’s study (2) examined the adequacy of training in various CanMEDS roles, selected in consultation with the Canadian Paediatric Society and the RCPSC Paediatric Specialty Committee. The roles examined included: medical expert (anticipatory guidance/well-child care, dealing with maltreatment/abuse, chronic care, palliative care, dealing with death and bereaved parents, and procedural skills); communicator (working with patients/ families, working with cultural/socio-economic differences); collaborator (working as a team); manager (learning principles of quality management, managing an efficient office practice); health advocate (for individual patients, for disadvantaged patients); scholar (ability to conduct a research project, ability to critically appraise literature); and professional (including ethical issues). Paediatricians in Lieberman and Hilliard’s study expressed concerns of inadequate training in three of the CanMEDS competencies, including the medical aspects of palliative care (medical expert), dealing with death and the bereaved family (medical expert), and managing an efficient office practice (manager).

These findings are corroborated by other studies from the United States, Australia and Turkey (4–11). Preliminary data from the Residency Review and Redesign (R3P) Project in the United States has also outlined many of these topics as needing greater focus according to both recently graduated paediatricians and current fellows (12).

Given that specific areas of deficiency have been identified, reasons behind these deficiencies on graduation need to be explored. We hypothesized that either current training programs underestimate the importance of, or the amount of emphasis placed on, these areas for future practicing paediatricians, or that residents themselves feel that these areas are less important to them during their training. To investigate this, we conducted a Canada-wide survey of paediatrics residents and paediatric residency program directors (PDs) to determine their views on these subspecialty areas as well as the various CanMEDS roles.

METHODS

A nine-page online questionnaire was designed using an online survey tool (www.surveymonkey.com) and was sent to all current Canadian paediatric residents across 16 institutions, through their respective PDs. The survey was a modified version of the previously validated questionnaire designed by Lieberman and Hilliard. Modifications included three additional pages that focused on personal demographics, and resident opinions on their current paediatric training program. Additionally, Lieberman and Hilliard’s survey grouped all surgical subspecialities together, whereas the present survey specifically asked questions about ophthalmology, otolaryngology and orthopedics to obtain viewpoints on these specific areas.

The present survey consisted of questions regarding the adequacy of current paediatric residency programs. Residents’ impressions on the importance of various subspecialties were contrasted with their program’s perceived emphasis on these same topics during their training. Similar comparisons were made for the seven CanMEDS roles and the importance they play in resident education. The present survey consisted primarily of Likert scale questions rated from 1 to 4 as follows: 1 (not important); 2 (somewhat important); 3 (important); 4 (very important). A translated francophone version of the survey was available to accommodate French-speaking residents. Identical questionnaires were also sent to all current Canadian PDs. Initial distribution was in September of 2008 and data were collected for a total of 10 weeks, and a reminder e-mail was sent to all eligible participants at the six-week mark. Ethics approval was obtained from the Children’s Hospital of Eastern Ontario Research Ethics Board.

The survey data were analyzed usign SPSS 16.0 (IBM Corporation, USA) for Windows (Microsoft Corporation, USA). Summary statistics were generated for the responses. For analysis, responses were dichotomized, in which responses 1 and 2 were classified in the overall category ‘Less Important’ and responses 3 and 4 were classified in the overall category ‘Important’. χ² tests were used to compare opinions of residents and PDs. P <0.05 was considered to be statistically significant. Cohen’s Kappa statistics were calculated with 95% CIs to assess the agreement between the residents’ ratings of the importance of the subspecialties and the residents’ perception of the importance that their programs place on these subspecialties beyond that expected by chance. A Kappa of 0.81 to 0.99 was interpreted as indicating almost perfect agreement, 0.61 to 0.80 was considered to be substantial agreement, 0.41 to 0.60 was considered moderate agreement, 0.21 to 0.40 was considered fair agreement, 0.01 to 0.20 was considered slight agreement and below 0 indicated poor or less than chance agreement (13).

RESULTS

General demographics

Fourteen of 16 accredited Canadian paediatric programs chose to participate. One-hundred twenty-seven of 486 active paediatric residents completed the survey (response rate 26%). Men accounted for 23% of respondents, while 77% were women, which at least in part, reflects the relative sex bias in paediatrics programs. The majority of respondents (70%) were between 20 and 30 years of age. There was an even distribution of post-graduate years (PGY) represented in the survey, with 31% of respondents in PGY-1, 31% in PGY-2, 22% in PGY-3 and the remainder in PGY4/5. Of the institutions participating in the present survey, 10 of 14 PDs completed the survey.

Of the residents surveyed, 54% planned to pursue a career in subspecialty paediatrics, and 46% planned to practice in general paediatrics. Only 10% of respondents felt they would not be adequately prepared for independent practice on graduation, and 86% either agreed or strongly agreed that they were satisfied with their current training.

Importance of subspecialty training

Residents were asked to rate how important they felt various subspecialty rotations were to their learning. The subspecialties rated as most important are shown in Table 1 (columns 1 and 2). Similarly, the subspecialties that residents rated as least important to their learning are shown in Table 2 (columns 1 and 2). At least 25% of residents rated nine of the 10 deficient areas as less important to their training (exact percentages shown in table). The majority of subspecialties identified as deficient in graduated paediatricians, as per Lieberman and Hilliard’s study, were considered to be less important by current residents.

TABLE 1

Subspecialty topics rated as more important to resident education

TABLE 2

Subspecialty topics rated as less important to resident education

Column 3 of Table 2 shows the percentage of residents that perceived their program to place less emphasis on each of the listed topics. At least 40% of programs were viewed as placing less importance on all 10 of Lieberman and Hilliard’s deficient areas (exact percentages shown in Table 2), indicating that residents also believe their programs place less emphasis in the exact same areas that are deficient in practicing physicians.

Column 4 (Table 2), which lists the percentage of PDs who felt that each of the corresponding subspecialty areas was less important to resident training, indicates that more than 50% of PDs felt that less importance should be placed in the areas of dermatology, metabolic disease, gynecology, orthopedics, general surgery and ophthalmology. When compared with PD’s, resident opinions differed significantly in two categories: a higher proportion of residents than PDs rated genetics and adolescent medicine as not important (Fisher’s exact test P<0.05).

The degree of emphasis residents placed on these topics versus the perceived importance their programs placed on these areas, was also compared. Kappa values were calculated to determine which topics showed statistically significant discrepancies between the two groups. Table 3 lists all of the subspecialties in which there were significant discrepancies (kappa ≤0.2) between how important residents felt a subject was and their perception of how much emphasis they felt their program was placing on that subject area. For example, only 25% of residents felt that dermatology was less important to their learning. However, 82% of residents perceived their programs as placing less emphasis on this topic. Residents therefore felt that their program should place more importance in this area, and this is reflected in the very low kappa (0.02). There were significant discrepancies between how much emphasis residents felt should be placed on a topic and how much emphasis they felt their programs placed on eight of the 10 areas identified as deficient in practicing paediatricians. Thus, residents and programs poorly agree on how much importance should be placed on these future areas of deficiency. Residents placed a higher percentage of importance on every topic compared with their programs except for general surgery and neonatology. These were the only two topics in which a higher number of programs were seen as placing more importance compared with residents themselves.

TABLE 3

Comparison of specialty importance between residents and training programs. Topics showing poor agreement in perceived level of importance between residents and programs are shown (defined by a kappa (ĸ) value of less than 0.2)

Importance of CanMEDS roles

The CanMeds roles were felt to play an important part in paediatric training for 78% of trainees surveyed. The roles rated as less important by residents are shown in column 2 of Table 4. Thirty-eight per cent of respondents felt that the ability to conduct a research project (scholar role) was less important to their education. The medical aspects of palliative care (medical expert role) and managing an efficient office practice (manager role) were two of the three roles identified as deficient by Lieberman and Hilliard, and were felt to be less important by 30% and 25% of all residents, respectively.

TABLE 4

CanMEDS roles rated as less important in resident education

Column 3 of Table 4 displays the CanMEDS roles that were perceived by trainees as having less emphasis by training programs. Seventy four per cent of programs were viewed as placing less emphasis on managing an efficient office practice (manager role, and 60% of programs were viewed as placing less emphasis on the medical aspects of palliative care (medical expert); both are roles that are deficient in newly practicing paediatricians. In total, more than 46% of all residents believe that training programs place less emphasis on all three of CanMEDS deficiencies identified by Lieberman and Hilliard.

PDs views on the importance of the various CanMEDS roles are also shown in Table 4. Column 4 lists the percentage of PDs who rated the corresponding role as less important to resident training. More than 80% of PDs felt that all of the CanMEDS roles were either important or very important to resident learning. When asked to evaluate training programs however, more than 50% of PDs felt that less emphasis was placed on the medical aspects of palliative care (medical expert role), learning principles of quality management (manager), managing an efficient office practice (manager) and effective use of resources (manager).

DISCUSSION

Lieberman and Hilliard showed that Canadian paediatricians felt inadequately trained in several areas, including gynecology, child psychiatry, behavioural paediatrics, surgical subspecialties, adolescent medicine, dermatology, nutrition, allergy and immunology, metabolics and genetics. Determining the reasons why these areas of deficiency arise was the primary goal of our study. Our findings point to both resident perceptions, as well as insufficient emphasis by training programs as potential explanations for these deficiencies.

Reassuringly, trainees were satisfied with their current training and felt they would be adequately prepared for independent practice. This is similar to the findings of Lieberman and Hilliard (2), who found that 96% of respondents felt they were adequately or well trained. However, we demonstrated that during training, residents place lesser importance on nine of the 10 areas identified in Lieberman and Hilliard’s study as deficient in future practicing paediatricians. It is possible that during training, residents may not appreciate the practical significance of these topics until after graduation.

It is also possible that training programs themselves place less emphasis on these topics. More than 40% of residents viewed their programs as placing less importance on all 10 areas deemed deficient by practicing paediatricians. There are several reasons why training programs may place less emphasis on these topics. There may be a lack of awareness that these are topics are important for future clinical practice. It may also be due to the absence of established subspecialty clinical programs at some institutions or a lack of available teachers. Programs may be able to improve knowledge in these areas by encouraging electives at other institutions, encouraging independent reading on these topics or presenting these topics as academic half days.

We found significant discrepancies in certain areas that were deemed important to residents, but were perceived as being less important by their training programs. These discrepancies may be the initial seed that has skewed resident interest and motivation in these topics in the first place. Programs may be giving the perception that these areas are not worth focusing on, and perhaps if training programs were to actively promote these topics, residents may be encouraged to place more emphasis on them during their training.

How then, should programs incorporate all of these areas into already full curricula? One option may be to extend the length of training; however, only 20% of the respondent’s in Lieberman and Hilliard’s study (3) felt that paediatrics training should be extended to five years. Programs were seen as placing more importance on general surgery and neonatology compared with residents themselves. This may mean that residents feel programs place more emphasis than is needed on these topics and may indicate areas that could be adjusted to accommodate other ‘deficient’ areas, although qualitative focus group research would help to better clarify this.

Although most residents feel the CanMEDS roles are important, one-quarter of residents did not place a great deal of importance in them. Specifically, residents placed less importance in the medical aspects of palliative care (medical expert role) and managing an efficient office practice (manager role), which were two of the three roles that were deficient in practicing physicians. Again, a lack of importance felt by residents in these areas during training is likely contributing to deficiencies on graduation. Similarly, residents felt their programs placed less emphasis on three of the CanMEDS roles that were identified as deficient in practicing paediatricians. Incorporating these roles more effectively into the paediatric curriculum may help to promote their importance, making residents more inclined to realize their impact on future practice.

Overall, PDs views on the importance of different areas of subspecialty training mirrored residents’ views At least 50% of PDs rated seven topics identified as being deficient in newly practicing paediatricians by Lieberman and Hillard as less important to resident learning. It would be interesting to know the representation of these topics in paediatric residency curriculums in Canada, and whether this is representative of PDs views on their importance. As PDs have a direct role-model influence on residents, the effects that their opinions have on resident education is substantial.

The main limitation of our study was the low response rate, and this needs to be taken into account when interpreting the data. Internet surveys are relatively new and have many potential advantages such as easier data analysis, faster response time and less paper waste. Unfortunately, they tend to produce significantly lower response rates than traditional paper versions. Several recent studies comparing internet and paper surveys found internet response rates vary anywhere between 11% and 58% (14–19). Thus, our response rates still fall well within the expected range for this type of survey. Our study could also have been strengthened by subdividing responses accordign to PGY, thereby allowing us to determine whether more senior residents had a different viewpoint on what should be emphasized in their training; this was not possible due to the low response rate. Additionally, it would have been ideal to compare resident perceptions of importance with external measures of resident knowledge, such as performance on the American Board of Pediatrics in-training examination or the RCPSC examination in pediatrics, to determine whether in fact, knowledge deficiencies were found in certain areas. However, this was beyond the scope of our study.

Thus, this perceived lack of importance felt by both residents and programs in various subspecialites and CanMEDS roles may contribute to deficiencies in newly graduated paediatricians. Qualitative research may be needed to understand the reasons why; knowing these reasons would better assist us in determining how we may alter the attitudes and training toward these future areas of deficiency. Promotion of these topics by training programs may also help to emphasize these areas, and better prepare residents for future practice.

Acknowledgments

The authors thank Dr Lani Lieberman and Dr Robert Hilliard for providing the survey instrument.

REFERENCES

1. Royal College of Physicians and Surgeons of Canada Objectives of training in pediatrics. < http://rcpsc.medical.org/information/?specialty=160&submit=Select> (Accessed on August 31, 2010).

2. Lieberman L, Hilliard R. How well do paediatric residency programmes prepare residents for clinical practice and their future careers? Med Educ. 2006;40:539–46. [PubMed]

3. Frank JR, editor. The CanMEDS 2005 physician competency framework Better standards Better physicians Better care. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2005.

4. Emans SJ, Bravender T, Knight J, et al. Adolescent medicine training in pediatric residency programs: Are we doing a good job? Pediatrics. 1998;102:588–95. [PubMed]

5. Gunasekera H, Buckmaster A. Training in general paediatrics: Is it time for change? Paediatr Child Health. 2004;40:510–16. [PubMed]

6. Fox HB. Adolescent medicine training in pediatric residency programs. Pediatrics. 2010;125:165–72. [PubMed]

7. Roberts KB, Starr S, DeWitt TG. The University of Massachusetts Medical Center office-based continuity experience: Are we preparing pediatrics residents for primary care practice? Pediatrics. 1997;100:E2. [PubMed]

8. Prose NS. Dermatology training during the pediatric residency. Clin Pediatr (Phila) 1988;27:100–3. [PubMed]

9. Freed GL, Dunham KM, Switalski KE, Jones MD, Jr, McGuinness GA., Research Advisory Committee of the American Board of Pediatrics Recently trained general pediatricians: Perspectives on residency training and scope of practice. Pediatrics. 2009;123:S38–43. [PubMed]

10. Hirfanoğlu T. A study of perceptions, attitudes, and level of knowledge among pediatricians towards attention-deficit/hyperactivity disorder. Turk J Pediatr. 2008;50:160–6. [PubMed]

11. McCabe ME, Hunt EA, Serwint JR. Pediatric residents’ clinical and educational experiences with end-of-life care. Pediatrics. 2008;121:e731–7. [PubMed]

12. Leslie LK. What can data tell us about the quality and relevance of current pediatric residency education? Pediatrics. 2009;123:S50–5. [PubMed]

13. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159. [PubMed]

14. Kim HL. Practice patterns in the treatment of female urinary incontinence: A postal and internet survey. Urology. 2001;57:45–8. [PubMed]

15. McMahon SR. Comparison of e-mail, fax, and postal surveys of pediatricians. Pediatrics. 2003;111:e299–303. [PubMed]

16. Leece P. Internet versus mailed questionnaires: A randomized comparison. J Med Internet Res. 2004;6:e30. [PMC free article] [PubMed]

17. Kongsved SM. Response rate and completeness of questionnaires: A randomized study of Internet versus paper-and-pencil versions. J Med Internet Res. 2007;9:e25. [PMC free article] [PubMed]

18. Leece P. Does flattery work? A comparison of 2 different cover letters for an international survey of orthopedic surgeons. Can J Surg. 2006;49:90–5. [PMC free article] [PubMed]

19. Raziano DB. E-mail versus conventional postal mail survey of geriatric chiefs. Gerontologist. 2001;41:799–804. [PubMed]

Articles from Paediatrics & Child Health are provided here courtesy of
Pulsus Group