PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1265431)

Clipboard (0)
None

Related Articles

1.  Patients’ assessment of professionalism and communication skills of medical graduates 
BMC Medical Education  2014;14:28.
Background
Professionalism and communication skills constitute important components of the integral formation of physicians which has repercussion on the quality of health care and medical education. The objective of this study was to assess medical graduates’ professionalism and communication skills from the patients’ perspective and to examine its association with patients’ socio-demographic variables.
Methods
This is a hospital based cross-sectional study. It involved 315 patients and 105 medical graduates selected by convenient sampling method. A modified and validated version of the American Board of Internal Medicine’s (ABIM) Patient Assessment survey questionnaire was used for data collection through a face to face interview. Data processing and analysis were performed using the Statistical Package for Social Science (SPSS) 16.0. Mean, frequency distribution, and percentage of the variables were calculated. A non-parametric Kruskal Wallis test was applied to verify whether the patients’ assessment was influenced by variables such as age, gender, education, at a level of significance, p ≤ 0.05.
Results
Female patients constituted 46% of the sample, whereas males constituted 54%. The mean age was 36 ± 16. Patients’ scoring of the graduate’s skills ranged from 3.29 to 3.83 with a mean of 3.64 on a five-point Likert scale. Items assessing the “patient involvement in decision-making” were assigned the minimum mean values, while items dealing with “establishing adequate communication with patient” assigned the maximum mean values. Patients, who were older than 45 years, gave higher scores than younger ones (p < 0.001). Patients with higher education reported much lower scores than those with lower education (p = 0.003). Patients’ gender did not show any statistically significant influence on the rating level.
Conclusion
Generally patients rated the medical graduates’ professionalism and communication skills at a good level. Patients’ age and educational level were significantly associated with the rating level.
doi:10.1186/1472-6920-14-28
PMCID: PMC3923249  PMID: 24517316
2.  Use of the Kalamazoo Essential Elements Communication Checklist (Adapted) in an Institutional Interpersonal and Communication Skills Curriculum 
Background
This study examined the psychometric properties of the Kalamazoo Essential Elements Communication Checklist (Adapted) (KEECC-A), which addresses 7 key elements of physician communication identified in the Kalamazoo Consensus Statement, in a sample of 135 residents in multiple specialties at a large urban medical center in 2008–2009. The KEECC-A was used by residents, standardized patients, and faculty as the assessment tool in a broader institutional curriculum initiative.
Methods
Three separate KEECC-A scores (self-ratings, faculty ratings, and standardized patient ratings) were calculated for each resident to assess the internal consistency and factor structure of the checklist. In addition, we analyzed KEECC-A ratings by gender and US versus international medical graduates, and collected American Board of Internal Medicine Patient Satisfaction Questionnaire (PSQ) scores for a subsample of internal medicine residents (n  =  28) to examine the relationship between this measure and the KEECC-A ratings to provide evidence of convergent validity.
Results
The KEECC-A ratings generated by faculty, standardized patients, and the residents themselves demonstrated a high degree of internal consistency. Factor analyses of the 3 different sets of KEECC-A ratings produced a consistent single-factor structure. We could not examine the relationship between KEECC-A and the PSQ because of substantial range restriction in PSQ scores. No differences were seen in the communication scores of men versus women. Faculty rated US graduates significantly higher than international medical graduates.
Conclusion
Our study provides evidence for the reliability and validity of the KEECC-A as a measure of physician communication skills. The KEECC-A appears to be a psychometrically sound, user-friendly communication tool, linked to an expert consensus statement, that can be quickly and accurately completed by multiple raters across diverse specialties.
doi:10.4300/JGME-D-10-00024.1
PMCID: PMC2941375  PMID: 21975614
3.  Clinical instructors' perception of a faculty development programme promoting postgraduate year-1 (PGY1) residents' ACGME six core competencies: a 2-year study 
BMJ Open  2011;1(2):e000200.
Objective
The six core competencies designated by Accreditation Council for Graduate Medical Education (ACGME) are essential for establishing a patient centre holistic medical system. The authors developed a faculty programme to promote the postgraduate year 1 (PGY1) resident, ACGME six core competencies. The study aims to assess the clinical instructors' perception, attitudes and subjective impression towards the various sessions of the ‘faculty development programme for teaching ACGME competencies.’
Methods
During 2009 and 2010, 134 clinical instructors participated in the programme to establish their ability to teach and assess PGY1 residents about ACGME competencies.
Results
The participants in the faculty development programme reported that the skills most often used while teaching were learnt during circuit and itinerant bedside, physical examination teaching, mini-clinical evaluation exercise (mini-CEX) evaluation demonstration, training workshop and videotapes of ‘how to teach ACGME competencies.’ Participants reported that circuit bedside teaching and mini-CEX evaluation demonstrations helped them in the interpersonal and communication skills domain, and that the itinerant teaching demonstrations helped them in the professionalism domain, while physical examination teaching and mini-CEX evaluation demonstrations helped them in the patients' care domain. Both the training workshop and videotape session increase familiarity with teaching and assessing skills. Participants who applied the skills learnt from the faculty development programme the most in their teaching and assessment came from internal medicine departments, were young attending physician and had experience as PGY1 clinical instructors.
Conclusions
According to the clinical instructors' response, our faculty development programme effectively increased their familiarity with various teaching and assessment skills needed to teach PGY1 residents and ACGME competencies, and these clinical instructors also then subsequently apply these skills.
Article summary
Article focus
In order to train PGY1 residents, we need to help clinical instructors to become familiar with the teaching and assessment skills that form the Accreditation Council for Graduate Medical Education six core-competencies.
Our study used a self-reported questionnaires based analysis to evaluate the clinical instructors' perception to our faculty development programme.
Key messages
Participants reported that their most commonly used skills were learnt from itinerant and circuit bedside teaching, and mini-clinical evaluation exercise evaluation demonstration in our programme.
Participants also reported that the 40 h basic training course improved their abilities to train and assess PGY1 residents in patient care, interpersonal and communication skills, and medical knowledge domains whereas postcourse training workshop and videotape session enhanced their ability in system-based practice, practice-based learning and improvement, and professionalism domains.
A serial follow-up questionnaire suggested that the degree of participant application of skills learnt from our programme increased progressively after finishing the 40 h basic training course, the postcourse training workshop and videotape session.
Strengths and limitations of this study
According to the clinical instructors' responses, our programme effectively increased their familiarity with teaching and assessment skills needed when teaching PGY1 residents' Accreditation Council for Graduate Medical Education competencies and that these skills were subsequently applies.
This study was limited by the fact that questionnaire used to track and assess the effectiveness of the training programme may have had information and recall bias. In addition, this study had a relatively small sample size and did not contain a control group. However, no controlled educational trials on this subject have been published as yet.
doi:10.1136/bmjopen-2011-000200
PMCID: PMC3225591  PMID: 22116089
4.  Use of a 360-Degree Evaluation in the Outpatient Setting: The Usefulness of Nurse, Faculty, Patient/Family, and Resident Self-Evaluation 
Background
Faculty have traditionally evaluated resident physician professionalism and interpersonal skills without input from patients, family members, nurses, or the residents themselves. The objective of our study was to use “360-degree evaluations,” as suggested by the Accreditation Council for Graduate Medical Education (ACGME), to determine if nonfaculty ratings of resident professionalism and interpersonal skills differ from faculty ratings.
Methods
Pediatrics residents were enrolled in a hospital-based resident continuity clinic during a 5-week period. Patient/families (P/Fs), faculty (MD [doctor of medicine]), nurses (RNs [registered nurses]), and residents themselves (self) completed evaluator-specific evaluations after each clinic session by using a validated 10-item questionnaire with a 5-point Likert scale. The average Likert score was tallied for each questionnaire. Mean Likert scale scores for each type of rater were compared by using analysis of variance, text with pair-wise comparisons when appropriate. Agreement between rater types was measured by using the Pearson correlation.
Results
A total of 823 evaluations were completed for 66 residents (total eligible residents, 69; 95% participation). All evaluators scored residents highly (mean Likert score range, 4.4 to 4.9). However, MDs and RNs scored residents higher than did P/Fs (mean scores: MD, 4.77, SD [standard deviation], 0.32; RN, 4.85, SD, 0.30; P/F, 4.53, SD, 0.96; P < .0001). MD and RN scores also were higher than residents' self-evaluation scores, but there was no difference between self-scores and P/F scores (average resident self-score, 4.44, SD, 0.43; P < .0001 compared to MD and RN; P  =  .19 compared to P/F). Correlation coefficients between all combinations of raters ranged from −0.21 to 0.21 and none were statistically significant.
Conclusion
Our study found high ratings for resident professionalism and interpersonal skills. However, different members of the health care team rated residents differently, and ratings are not correlated. Our results provide evidence for the potential value of 360-degree evaluations.
doi:10.4300/JGME-D-10-00013.1
PMCID: PMC2951785  PMID: 21976094
5.  New Tools for Systematic Evaluation of Teaching Qualities of Medical Faculty: Results of an Ongoing Multi-Center Survey 
PLoS ONE  2011;6(10):e25983.
Background
Tools for the evaluation, improvement and promotion of the teaching excellence of faculty remain elusive in residency settings. This study investigates (i) the reliability and validity of the data yielded by using two new instruments for evaluating the teaching qualities of medical faculty, (ii) the instruments' potential for differentiating between faculty, and (iii) the number of residents' evaluations needed per faculty to reliably use the instruments.
Methods and Materials
Multicenter cross-sectional survey among 546 residents and 629 medical faculty representing 29 medical (non-surgical) specialty training programs in the Netherlands. Two instruments—one completed by residents and one by faculty—for measuring teaching qualities of faculty were developed. Statistical analyses included factor analysis, reliability and validity exploration using standard psychometric methods, calculation of the numbers of residents' evaluations needed per faculty to achieve reliable assessments and variance components and threshold analyses.
Results
A total of 403 (73.8%) residents completed 3575 evaluations of 570 medical faculty while 494 (78.5%) faculty self-evaluated. In both instruments five composite-scales of faculty teaching qualities were detected with high internal consistency and reliability: learning climate (Cronbach's alpha of 0.85 for residents' instrument, 0.71 for self-evaluation instrument, professional attitude and behavior (0.84/0.75), communication of goals (0.90/0.84), evaluation of residents (0.91/0.81), and feedback (0.91/0.85). Faculty tended to evaluate themselves higher than did the residents. Up to a third of the total variance in various teaching qualities can be attributed to between-faculty differences. Some seven residents' evaluations per faculty are needed for assessments to attain a reliability level of 0.90.
Conclusions
The instruments for evaluating teaching qualities of medical faculty appear to yield reliable and valid data. They are feasible for use in medical residencies, can detect between-faculty differences and supply potentially useful information for improving graduate medical education.
doi:10.1371/journal.pone.0025983
PMCID: PMC3193529  PMID: 22022486
6.  Measuring Residents’ Perceived Preparedness and Skillfulness to Deliver Cross-cultural Care 
Journal of General Internal Medicine  2009;24(9):1053-1056.
BACKGROUND
As patient populations become increasingly diverse, we need to be able to measure residents’ preparedness and skillfulness to provide cross-cultural care.
OBJECTIVE
To develop a measure that assesses residents’ perceived readiness and abilities to provide cross-cultural care.
DESIGN
Survey items were developed based on an extensive literature review, interviews with experts, and seven focus groups and ten individual interviews, as part of a larger national mailed survey effort of graduating residents in seven specialties. Reliability and weighted principal components analyses were performed with items that assessed perceived preparedness and skillfulness to provide cross-cultural care. Construct validity was assessed.
PARTICIPANTS
A total of 2,047 of 3,435 eligible residents participated (response rate = 60%).
MEASUREMENTS AND MAIN RESULTS
The final scale consisted of 18 items and 3 components (general cross-cultural preparedness, general cross-cultural skillfulness, and cross-cultural language preparedness and skillfulness), and yielded a Cronbach’s alpha = 0.92. Construct validity was supported; the scale total was inversely correlated with a measure of helplessness when providing care to patients of a different culture (p < 0.001).
CONCLUSIONS
We developed a three-component cross-cultural preparedness and skillfulness scale that was internally consistent and demonstrated construct validity. This measure can be used to evaluate residents’ perceived effectiveness of cross-cultural medical training programs and could be used in future work to validate residents’ self assessments with objective assessments.
doi:10.1007/s11606-009-1046-1
PMCID: PMC2726883  PMID: 19557481
cultural competency; measurement; medical education-graduate
7.  Assessment of Resident Physicians in Professionalism, Interpersonal and Communication Skills: a Multisource Feedback 
Objective: To assess the internal validity and reliability of a multisource feedback (MSF) program by China Medical Board for resident physicians in China.
Method: Multisource feedback was used to assess professionalism, interpersonal and communication skills. 258 resident physicians were assessed by attending doctors, self-evaluation, resident peers, nurses, office staffs, and patients who completed a sealed questionnaire at 19 hospitals in China. Cronbach's alpha coefficient was used to assess reliability. Validity was assessed by exploratory factor analyses and by profile ratings.
Results: 4128 questionnaires were collected from this study. All responses had high internal consistency and reliability (Cronbach's α> 0.90), which suggests that both questions and form data were internally consistent. The exploratory factor analysis with varimax rotation for the evaluators' questionnaires was able to account for 70 to 74% of the total variance.
Conclusion: The current MSF assessment tools are internally valid and reliable for assessing resident physician professionalism and interpersonal and communication skills in China.
doi:10.7150/ijms.3353
PMCID: PMC3348527  PMID: 22577337
Resident physician; Multisource feedback; Professionalism; Interpersonal and Communication Skills; international
8.  A Simplified Observation Tool for Residents in the Outpatient Clinic 
Background
The Accreditation Council for Graduate Medical Education promotes direct observation of residents as a key assessment tool for competency in patient care, professionalism, and communication skills. Although tools exist, validity and reliability have not been demonstrated for most, and many tools may have limited feasibility because of time constraints and other reasons. We conducted a study to measure feasibility of a simplified observation tool to evaluate these competencies and provide timely feedback.
Methods
In the pediatric resident continuity clinic of a large children's hospital, we used a direct observation form with a 3-point scale for 16 items in the domains of patient care, professionalism, and communication skills. The form was divided by portion of visit, with specific items mapped to 1 or more of the competencies, and was used to provide direct oral feedback to the resident. Faculty and residents completed surveys rating the process (ease of use, satisfaction, and self-assessed usefulness) on a 5-point Likert scale.
Results
The study encompassed 89 surveys completed by attending physicians; 98% (87 of 89) of the time the form was easy to use, 99% (88) of the time its use did not interfere with patient flow, and 93% (83) of the observations provided useful information for resident feedback. Residents completed 70 surveys, with the majority (69%, 48) reporting they were comfortable about being observed by an attending physician; 87% (61) thought that direct observation did not significantly affect their efficiency. Ninety-seven percent of the time (68) residents reported that direct observation provided useful feedback.
Conclusion
The data suggest the form was well-received by both faculty and residents, and enabled attending physicians to provide useful feedback.
doi:10.4300/JGME-D-09-00090.1
PMCID: PMC2931216  PMID: 21975895
9.  Perceptions of medical school graduates and students regarding their academic preparation to teach 
Postgraduate Medical Journal  2006;82(971):607-612.
Purpose
How medical students learn and develop the characteristics associated with good teaching in medicine is not well known. Information about this process can improve the academic preparation of medical students for teaching responsibilities. The purpose of this study was to determine how different experiences contributed to the knowledge, skills, and attitudes of medical school graduates and students regarding medical teaching.
Methods
A questionnaire was developed, addressing reliability and validity considerations, and given to first year residents and third year medical students (taught by those residents). Completed questionnaires were collected from 76 residents and 110 students (81% of the sample group). Item responses were analysed using descriptive and inferential statistics.
Results
Most residents (n = 54; 71%) positively viewed opportunities they had to practice teaching when they were seniors. Residents rated three activities for learning to teach highest: (1) observing teachers as they teach; (2) reviewing the material to be taught; and (3) directly teaching students; representing both individual and participatory ways of learning. Residents' self ratings of teaching behaviours improved over time and this self assessment by the residents was validated by the students' responses. Comparison between residents' self ratings and students' views of typical resident teaching behaviours showed agreement on levels of competence, confidence, and motivation. The students rated characteristics of enthusiasm, organisation, and fulfilment lower (p<0.002) than residents rated themselves.
Conclusions
The residents and students in this study viewed academic preparation for teaching responsibilities positively and showed agreement on characteristics of good teaching that may be helpful indicators in the process of developing medical teachers.
doi:10.1136/pgmj.2006.045393
PMCID: PMC2585736  PMID: 16954460
medical education, undergraduate; medical education, internship and residency; teaching methods; experiential learning; educational techniques
10.  How patients perceive the therapeutic communications skills of their general practitioners, and how that perception affects adherence: use of the TCom-skill GP scale in a specific geographical area 
Background
To study: (1) the structure and test-retest reliability of a measure of how patients perceive the therapeutic communications skills of their general practitioners (TCom-skill GP), and (2) the associations of that scale with socio-demographic and health-related characteristics, and adherence.
Methods
A total of 393 people who lived in the same geographic area and invited to attend a preventive medical centre for a check up were asked to complete a self-administered questionnaire concerning TCom-skill GP (15 items), socio-demographic and health-related characteristics, and to answer two questions on perceived adherence.
Results
The average age of respondents was 46.8 years (SD 14), and 50.4% were men. The TCom-skill GP score was one-dimensional, had high internal coherence (Cronbach α 0.92), and good test-retest reliability (intra-class correlation coefficient 0.74). The overall score was positively related to increasing age. Respondents aged 60+ were more likely to be adherent. The higher the score, the higher the probability of adherence. Multivariate analysis showed that the TCom-skill score was associated with advancing age and the number of consultations with the GP during the previous 3 months, but not with gender, living alone, being employed, job category or educational level. Multivariate analysis also showed that adherence was associated with TCom-skill GP score which concealed the association between adherence and advancing age observed in univariate analysis.
Conclusion
The TCom-skill GP scale probably has value in assessing the quality of doctor-patient relationships and therapeutic communications. The psychometric properties of the TCom-skill GP scale were appropriate for its use in this context. Adherence related to the TCom-skill GP and the latter related to the age of patients and the number of their previous consultations. The TCom-skill GP scale may be a useful way to assess, in a specific geographical location, the impact of medical professional training on therapeutic communication.
doi:10.1186/1472-6963-8-244
PMCID: PMC2612661  PMID: 19046433
11.  How was the intern year?: self and clinical assessment of four cohorts, from two medical curricula 
BMC Medical Education  2014;14:123.
Background
Problem-based curricula have provoked controversy amongst educators and students regarding outcome in medical graduates, supporting the need for longitudinal evaluation of curriculum change. As part of a longitudinal evaluation program at the University of Adelaide, a mixed method approach was used to compare the graduate outcomes of two curriculum cohorts: traditional lecture-based ‘old’ and problem-based ‘new’ learning.
Methods
Graduates were asked to self-assess preparedness for hospital practice and consent to a comparative analysis of their work-place based assessments from their intern year. Comparative data were extracted from 692 work-place based assessments for 124 doctors who graduated from the University of Adelaide Medical School between 2003 and 2006.
Results
Self-assessment: Overall, graduates of the lecture-based curriculum rated the medical program significantly higher than graduates of the problem-based curriculum. However, there was no significant difference between the two curriculum cohorts with respect to their preparedness in 13 clinical skills. There were however, two areas where the cohorts rated their preparedness in the 13 broad practitioner competencies as significantly different: problem-based graduates rated themselves as better prepared in their ‘awareness of legal and ethical issues’ and the lecture-based graduates rated themselves better prepared in their ‘understanding of disease processes’.
Work-place based assessment: There were no significant differences between the two curriculum cohorts for ‘Appropriate Level of Competence’ and ‘Overall Appraisal’. Of the 14 work-place based assessment skills assessed for competence, no significant difference was found between the cohorts.
Conclusions
The differences in the perceived preparedness for hospital practice of two curriculum cohorts do not reflect the work-place based assessments of their competence as interns. No significant difference was found between the two cohorts in relation to their knowledge and clinical skills. However results suggest a trend in ‘communication with peers and colleagues in other disciplines’ (χ2 (3, N = 596) =13.10, p = 0.056) that requires further exploration. In addition we have learned that student confidence in a new curriculum may impact on their self-perception of preparedness, while not affecting their actual competence.
doi:10.1186/1472-6920-14-123
PMCID: PMC4081487  PMID: 24961171
Evaluation/assessment; Clinical performance; Curriculum development/evaluation; Problem-based; Intern/house officer training; Competence
12.  Emotional Intelligence and the ACGME Competencies 
Background
Residency programs desire assessment tools for teaching and measuring resident attainment of the Accreditation Council for Graduate Medical Education competencies, including interpersonal and communication skills.
Objective
We sought to evaluate the use of emotional intelligence (EI) assessment and training tools in assessing and enhancing interpersonal and communication skills.
Methods
We used a quasi-experimental design, with an intervention and control group composed of 1 class each of family medicine residents. The intervention was EI coaching. The assessment used the Emotional and Social Competence Inventory, a 360-degree EI survey consisting of self and other (colleague) ratings for 12 EI competencies.
Results
There were 21 participants in each of the 3 assessments (test, posttest, and control). Our EI coaching intervention had very limited participation due to a lack of protected time for EI coaching and residents' competing obligations. Return rates for self surveys were 86% to 91% and 66% to 68% for others. On all 3 trials, ratings by others were significantly higher than self ratings for every competence (range, P < .001–.045). None of the self ratings by the intervention group increased significantly for any of the competencies. None of the intervention group self ratings increased significantly on posttesting, whereas ratings by others increased significantly for coach/mentor (P < .001). The teamwork rating decreased significantly on both self and other ratings (P < .001). Achievement orientation was the highest intervention group posttest rating, and teamwork was the lowest.
Conclusions
EI is a necessary skill in today's health care environment, and our study found that a tool from another sector was useful in assessing resident EI skills. Because our EI coaching intervention was unsuccessful, the effects of coaching on interpersonal and communication skills could not be assessed.
doi:10.4300/JGME-D-10-00080.1
PMCID: PMC3010931  PMID: 22132269
13.  Burnout and Distress Among Internal Medicine Program Directors: Results of A National Survey 
Journal of General Internal Medicine  2013;28(8):1056-1063.
BACKGROUND
Physician burnout and distress has been described in national studies of practicing physicians, internal medicine (IM) residents, IM clerkship directors, and medical school deans. However, no comparable national data exist for IM residency program directors.
OBJECTIVE
To assess burnout and distress among IM residency program directors, and to evaluate relationships of distress with personal and program characteristics and perceptions regarding implementation and consequences of Accreditation Council for Graduate Medical Education (ACGME) regulations.
DESIGN AND PARTICIPANTS
The 2010 Association of Program Directors in Internal Medicine (APDIM) Annual Survey, developed by the APDIM Survey Committee, was sent in August 2010 to the 377 program directors with APDIM membership, representing 99.0 % of the 381 United States categorical IM residency programs.
MAIN MEASURES
The 2010 APDIM Annual Survey included validated items on well-being and distress, including questions addressing quality of life, satisfaction with work-life balance, and burnout. Questions addressing personal and program characteristics and perceptions regarding implementation and consequences of ACGME regulations were also included.
KEY RESULTS
Of 377 eligible program directors, 282 (74.8 %) completed surveys. Among respondents, 12.4 % and 28.8 % rated their quality of life and satisfaction with work-life balance negatively, respectively. Also, 27.0 % reported emotional exhaustion, 10.4 % reported depersonalization, and 28.7 % reported overall burnout. These rates were lower than those reported previously in national studies of medical students, IM residents, practicing physicians, IM clerkship directors, and medical school deans. Aspects of distress were more common among younger program directors, women, and those reporting greater weekly work hours. Work–home conflicts were common and associated with all domains of distress, especially if not resolved in a manner effectively balancing work and home responsibilities. Associations with program characteristics such as program size and American Board of Internal Medicine (ABIM) pass rates were not found apart from higher rates of depersonalization among directors of community-based programs (23.5 % vs. 8.6 %, p = 0.01). We did not observe any consistent associations between distress and perceptions of implementation and consequences of program regulations.
CONCLUSIONS
The well-being of IM program directors across domains, including quality of life, satisfaction with work-life balance, and burnout, appears generally superior to that of medical trainees, practicing physicians, and other medical educators nationally. Additionally, it is reassuring that program directors' perceptions of their ability to respond to current regulatory requirements are not adversely associated with distress. However, the increased distress levels among younger program directors, women, and those at community-based training programs reported in this study are important concerns worthy of further study.
doi:10.1007/s11606-013-2349-9
PMCID: PMC3710382  PMID: 23595924
graduate medical education; residency; burnout; well-being
14.  Implementing the 2009 Institute of Medicine recommendations on resident physician work hours, supervision, and safety 
Long working hours and sleep deprivation have been a facet of physician training in the US since the advent of the modern residency system. However, the scientific evidence linking fatigue with deficits in human performance, accidents and errors in industries from aeronautics to medicine, nuclear power, and transportation has mounted over the last 40 years. This evidence has also spawned regulations to help ensure public safety across safety-sensitive industries, with the notable exception of medicine.
In late 2007, at the behest of the US Congress, the Institute of Medicine embarked on a year-long examination of the scientific evidence linking resident physician sleep deprivation with clinical performance deficits and medical errors. The Institute of Medicine’s report, entitled “Resident duty hours: Enhancing sleep, supervision and safety”, published in January 2009, recommended new limits on resident physician work hours and workload, increased supervision, a heightened focus on resident physician safety, training in structured handovers and quality improvement, more rigorous external oversight of work hours and other aspects of residency training, and the identification of expanded funding sources necessary to implement the recommended reforms successfully and protect the public and resident physicians themselves from preventable harm.
Given that resident physicians comprise almost a quarter of all physicians who work in hospitals, and that taxpayers, through Medicare and Medicaid, fund graduate medical education, the public has a deep investment in physician training. Patients expect to receive safe, high-quality care in the nation’s teaching hospitals. Because it is their safety that is at issue, their voices should be central in policy decisions affecting patient safety. It is likewise important to integrate the perspectives of resident physicians, policy makers, and other constituencies in designing new policies. However, since its release, discussion of the Institute of Medicine report has been largely confined to the medical education community, led by the Accreditation Council for Graduate Medical Education (ACGME).
To begin gathering these perspectives and developing a plan to implement safer work hours for resident physicians, a conference entitled “Enhancing sleep, supervision and safety: What will it take to implement the Institute of Medicine recommendations?” was held at Harvard Medical School on June 17–18, 2010. This White Paper is a product of a diverse group of 26 representative stakeholders bringing relevant new information and innovative practices to bear on a critical patient safety problem. Given that our conference included experts from across disciplines with diverse perspectives and interests, not every recommendation was endorsed by each invited conference participant. However, every recommendation made here was endorsed by the majority of the group, and many were endorsed unanimously. Conference members participated in the process, reviewed the final product, and provided input before publication. Participants provided their individual perspectives, which do not necessarily represent the formal views of any organization.
In September 2010 the ACGME issued new rules to go into effect on July 1, 2011. Unfortunately, they stop considerably short of the Institute of Medicine’s recommendations and those endorsed by this conference. In particular, the ACGME only applied the limitation of 16 hours to first-year resident physicans. Thus, it is clear that policymakers, hospital administrators, and residency program directors who wish to implement safer health care systems must go far beyond what the ACGME will require. We hope this White Paper will serve as a guide and provide encouragement for that effort.
Resident physician workload and supervision
By the end of training, a resident physician should be able to practice independently. Yet much of resident physicians’ time is dominated by tasks with little educational value. The caseload can be so great that inadequate reflective time is left for learning based on clinical experiences. In addition, supervision is often vaguely defined and discontinuous. Medical malpractice data indicate that resident physicians are frequently named in lawsuits, most often for lack of supervision. The recommendations are: The ACGME should adjust resident physicians workload requirements to optimize educational value. Resident physicians as well as faculty should be involved in work redesign that eliminates nonessential and noneducational activity from resident physician dutiesMechanisms should be developed for identifying in real time when a resident physician’s workload is excessive, and processes developed to activate additional providersTeamwork should be actively encouraged in delivery of patient care. Historically, much of medical training has focused on individual knowledge, skills, and responsibility. As health care delivery has become more complex, it will be essential to train resident and attending physicians in effective teamwork that emphasizes collective responsibility for patient care and recognizes the signs, both individual and systemic, of a schedule and working conditions that are too demanding to be safeHospitals should embrace the opportunities that resident physician training redesign offers. Hospitals should recognize and act on the potential benefits of work redesign, eg, increased efficiency, reduced costs, improved quality of care, and resident physician and attending job satisfactionAttending physicians should supervise all hospital admissions. Resident physicians should directly discuss all admissions with attending physicians. Attending physicians should be both cognizant of and have input into the care patients are to receive upon admission to the hospitalInhouse supervision should be required for all critical care services, including emergency rooms, intensive care units, and trauma services. Resident physicians should not be left unsupervised to care for critically ill patients. In settings in which the acuity is high, physicians who have completed residency should provide direct supervision for resident physicians. Supervising physicians should always be physically in the hospital for supervision of resident physicians who care for critically ill patientsThe ACGME should explicitly define “good” supervision by specialty and by year of training. Explicit requirements for intensity and level of training for supervision of specific clinical scenarios should be providedCenters for Medicare and Medicaid Services (CMS) should use graduate medical education funding to provide incentives to programs with proven, effective levels of supervision. Although this action would require federal legislation, reimbursement rules would help to ensure that hospitals pay attention to the importance of good supervision and require it from their training programs
Resident physician work hours
Although the IOM “Sleep, supervision and safety” report provides a comprehensive review and discussion of all aspects of graduate medical education training, the report’s focal point is its recommendations regarding the hours that resident physicians are currently required to work. A considerable body of scientific evidence, much of it cited by the Institute of Medicine report, describes deteriorating performance in fatigued humans, as well as specific studies on resident physician fatigue and preventable medical errors.
The question before this conference was what work redesign and cultural changes are needed to reform work hours as recommended by the Institute of Medicine’s evidence-based report? Extensive scientific data demonstrate that shifts exceeding 12–16 hours without sleep are unsafe. Several principles should be followed in efforts to reduce consecutive hours below this level and achieve safer work schedules. The recommendations are: Limit resident physician work hours to 12–16 hour maximum shiftsA minimum of 10 hours off duty should be scheduled between shiftsResident physician input into work redesign should be actively solicitedSchedules should be designed that adhere to principles of sleep and circadian science; this includes careful consideration of the effects of multiple consecutive night shifts, and provision of adequate time off after night work, as specified in the IOM reportResident physicians should not be scheduled up to the maximum permissible limits; emergencies frequently occur that require resident physicians to stay longer than their scheduled shifts, and this should be anticipated in scheduling resident physicians’ work shiftsHospitals should anticipate the need for iterative improvement as new schedules are initiated; be prepared to learn from the initial phase-in, and change the plan as neededAs resident physician work hours are redesigned, attending physicians should also be considered; a potential consequence of resident physician work hour reduction and increased supervisory requirements may be an increase in work for attending physicians; this should be carefully monitored, and adjustments to attending physician work schedules made as needed to prevent unsafe work hours or working conditions for this group“Home call” should be brought under the overall limits of working hours; work load and hours should be monitored in each residency program to ensure that resident physicians and fellows on home call are getting sufficient sleepMedicare funding for graduate medical education in each hospital should be linked with adherence to the Institute of Medicine limits on resident physician work hours
Moonlighting by resident physicians
The Institute of Medicine report recommended including external as well as internal moonlighting in working hour limits. The recommendation is: All moonlighting work hours should be included in the ACGME working hour limits and actively monitored. Hospitals should formalize a moonlighting policy and establish systems for actively monitoring resident physician moonlighting
Safety of resident physicians
The “Sleep, supervision and safety” report also addresses fatigue-related harm done to resident physicians themselves. The report focuses on two main sources of physical injury to resident physicians impaired by fatigue, ie, needle-stick exposure to blood-borne pathogens and motor vehicle crashes. Providing safe transportation home for resident physicians is a logistical and financial challenge for hospitals. Educating physicians at all levels on the dangers of fatigue is clearly required to change driving behavior so that safe hospital-funded transport home is used effectively. Fatigue-related injury prevention (including not driving while drowsy) should be taught in medical school and during residency, and reinforced with attending physicians; hospitals and residency programs must be informed that resident physicians’ ability to judge their own level of impairment is impaired when they are sleep deprived; hence, leaving decisions about the capacity to drive to impaired resident physicians is not recommendedHospitals should provide transportation to all resident physicians who report feeling too tired to drive safely; in addition, although consecutive work should not exceed 16 hours, hospitals should provide transportation for all resident physicians who, because of unforeseen reasons or emergencies, work for longer than consecutive 24 hours; transportation under these circumstances should be automatically provided to house staff, and should not rely on self-identification or request
Training in effective handovers and quality improvement
Handover practice for resident physicians, attendings, and other health care providers has long been identified as a weak link in patient safety throughout health care settings. Policies to improve handovers of care must be tailored to fit the appropriate clinical scenario, recognizing that information overload can also be a problem. At the heart of improving handovers is the organizational effort to improve quality, an effort in which resident physicians have typically been insufficiently engaged. The recommendations are: Hospitals should train attending and resident physicians in effective handovers of careHospitals should create uniform processes for handovers that are tailored to meet each clinical setting; all handovers should be done verbally and face-to-face, but should also utilize written toolsWhen possible, hospitals should integrate hand-over tools into their electronic medical records (EMR) systems; these systems should be standardized to the extent possible across residency programs in a hospital, but may be tailored to the needs of specific programs and services; federal government should help subsidize adoption of electronic medical records by hospitals to improve signoutWhen feasible, handovers should be a team effort including nurses, patients, and familiesHospitals should include residents in their quality improvement and patient safety efforts; the ACGME should specify in their core competency requirements that resident physicians work on quality improvement projects; likewise, the Joint Commission should require that resident physicians be included in quality improvement and patient safety programs at teaching hospitals; hospital administrators and residency program directors should create opportunities for resident physicians to become involved in ongoing quality improvement projects and root cause analysis teams; feedback on successful quality improvement interventions should be shared with resident physicians and broadly disseminatedQuality improvement/patient safety concepts should be integral to the medical school curriculum; medical school deans should elevate the topics of patient safety, quality improvement, and teamwork; these concepts should be integrated throughout the medical school curriculum and reinforced throughout residency; mastery of these concepts by medical students should be tested on the United States Medical Licensing Examination (USMLE) stepsFederal government should support involvement of resident physicians in quality improvement efforts; initiatives to improve quality by including resident physicians in quality improvement projects should be financially supported by the Department of Health and Human Services
Monitoring and oversight of the ACGME
While the ACGME is a key stakeholder in residency training, external voices are essential to ensure that public interests are heard in the development and monitoring of standards. Consequently, the Institute of Medicine report recommended external oversight and monitoring through the Joint Commission and Centers for Medicare and Medicaid Services (CMS). The recommendations are: Make comprehensive fatigue management a Joint Commission National Patient Safety Goal; fatigue is a safety concern not only for resident physicians, but also for nurses, attending physicians, and other health care workers; the Joint Commission should seek to ensure that all health care workers, not just resident physicians, are working as safely as possibleFederal government, including the Centers for Medicare and Medicaid Services and the Agency for Healthcare Research and Quality, should encourage development of comprehensive fatigue management programs which all health systems would eventually be required to implementMake ACGME compliance with working hours a “ condition of participation” for reimbursement of direct and indirect graduate medical education costs; financial incentives will greatly increase the adoption of and compliance with ACGME standards
Future financial support for implementation
The Institute of Medicine’s report estimates that $1.7 billion (in 2008 dollars) would be needed to implement its recommendations. Twenty-five percent of that amount ($376 million) will be required just to bring hospitals into compliance with the existing 2003 ACGME rules. Downstream savings to the health care system could potentially result from safer care, but these benefits typically do not accrue to hospitals and residency programs, who have been asked historically to bear the burden of residency reform costs. The recommendations are: The Institute of Medicine should convene a panel of stakeholders, including private and public funders of health care and graduate medical education, to lay down the concrete steps necessary to identify and allocate the resources needed to implement the recommendations contained in the IOM “Resident duty hours: Enhancing sleep, supervision and safety” report. Conference participants suggested several approaches to engage public and private support for this initiativeEfforts to find additional funding to implement the Institute of Medicine recommendations should focus more broadly on patient safety and health care delivery reform; policy efforts focused narrowly upon resident physician work hours are less likely to succeed than broad patient safety initiatives that include residency redesign as a key componentHospitals should view the Institute of Medicine recommendations as an opportunity to begin resident physician work redesign projects as the core of a business model that embraces safety and ultimately saves resourcesBoth the Secretary of Health and Human Services and the Director of the Centers for Medicare and Medicaid Services should take the Institute of Medicine recommendations into consideration when promulgating rules for innovation grantsThe National Health Care Workforce Commission should consider the Institute of Medicine recommendations when analyzing the nation’s physician workforce needs
Recommendations for future research
Conference participants concurred that convening the stakeholders and agreeing on a research agenda was key. Some observed that some sectors within the medical education community have been reluctant to act on the data. Several logical funders for future research were identified. But above all agencies, Centers for Medicare and Medicaid Services is the only stakeholder that funds graduate medical education upstream and will reap savings downstream if preventable medical errors are reduced as a result of reform of resident physician work hours.
doi:10.2147/NSS.S19649
PMCID: PMC3630963  PMID: 23616719
resident; hospital; working hours; safety
15.  The resident-as-teacher educational challenge: a needs assessment survey at the National Autonomous University of Mexico Faculty of Medicine 
BMC Medical Education  2010;10:17.
Background
The role of residents as educators is increasingly recognized, since it impacts residents, interns, medical students and other healthcare professionals. A widespread implementation of resident-as-teacher courses in developed countries' medical schools has occurred, with variable results. There is a dearth of information about this theme in developing countries. The National Autonomous University of Mexico (UNAM) Faculty of Medicine has more than 50% of the residency programs' physician population in Mexico. This report describes a needs assessment survey for a resident as teacher program at our institution.
Methods
A cross-sectional descriptive survey was developed based on a review of the available literature and discussion by an expert multidisciplinary committee. The goal was to identify the residents' attitudes, academic needs and preferred educational strategies regarding resident-as-teacher activities throughout the residency. The survey was piloted and modified accordingly. The paper anonymous survey was sent to 7,685 residents, the total population of medical residents in UNAM programs in the country.
Results
There was a 65.7% return rate (5,186 questionnaires), a broad and representative sample of the student population. The residents felt they had knowledge and were competent in medical education, but the majority felt a need to improve their knowledge and skills in this discipline. Most residents (92.5%) felt that their role as educators of medical students, interns and other residents was important/very important. They estimated that 45.5% of their learning came from other residents. Ninety percent stated that it was necessary to be trained in teaching skills. The themes identified to include in the educational intervention were mostly clinically oriented. The educational strategies in order of preference were interactive lectures with a professor, small groups with a moderator, material available in a website for self-learning, printed material for self-study and homework, and small group web-based learning.
Conclusions
There is a large unmet need to implement educational interventions to improve residents' educational skills in postgraduate educational programs in developing countries. Most perceived needs of residents are practical and clinically oriented, and they prefer traditional educational strategies. Resident as teachers educational interventions need to be designed taking into account local needs and resources.
doi:10.1186/1472-6920-10-17
PMCID: PMC2830225  PMID: 20156365
16.  Systematic Evaluation of the Teaching Qualities of Obstetrics and Gynecology Faculty: Reliability and Validity of the SETQ Tools 
PLoS ONE  2011;6(5):e19142.
Background
The importance of effective clinical teaching for the quality of future patient care is globally understood. Due to recent changes in graduate medical education, new tools are needed to provide faculty with reliable and individualized feedback on their teaching qualities. This study validates two instruments underlying the System for Evaluation of Teaching Qualities (SETQ) aimed at measuring and improving the teaching qualities of obstetrics and gynecology faculty.
Methods and Findings
This cross-sectional multi-center questionnaire study was set in seven general teaching hospitals and two academic medical centers in the Netherlands. Seventy-seven residents and 114 faculty were invited to complete the SETQ instruments in the duration of one month from September 2008 to September 2009. To assess reliability and validity of the instruments, we used exploratory factor analysis, inter-item correlation, reliability coefficient alpha and inter-scale correlations. We also compared composite scales from factor analysis to global ratings. Finally, the number of residents' evaluations needed per faculty for reliable assessments was calculated. A total of 613 evaluations were completed by 66 residents (85.7% response rate). 99 faculty (86.8% response rate) participated in self-evaluation. Factor analysis yielded five scales with high reliability (Cronbach's alpha for residents' and faculty): learning climate (0.86 and 0.75), professional attitude (0.89 and 0.81), communication of learning goals (0.89 and 0.82), evaluation of residents (0.87 and 0.79) and feedback (0.87 and 0.86). Item-total, inter-scale and scale-global rating correlation coefficients were significant (P<0.01). Four to six residents' evaluations are needed per faculty (reliability coefficient 0.60–0.80).
Conclusions
Both SETQ instruments were found reliable and valid for evaluating teaching qualities of obstetrics and gynecology faculty. Future research should examine improvement of teaching qualities when using SETQ.
doi:10.1371/journal.pone.0019142
PMCID: PMC3086887  PMID: 21559275
17.  Improving Residents' Code Status Discussion Skills: A Randomized Trial 
Journal of Palliative Medicine  2012;15(7):768-774.
Abstract
Background
Inpatient Code Status Discussions (CSDs) are commonly facilitated by resident physicians, despite inadequate training. We studied the efficacy of a CSD communication skills training intervention for internal medicine residents.
Methods
This was a prospective, randomized controlled trial of a multimodality communication skills educational intervention for postgraduate year (PGY) 1 residents. Intervention group residents completed a 2 hour teaching session with deliberate practice of communication skills, online modules, self-reflection, and a booster training session in addition to assigned clinical rotations. Control group residents completed clinical rotations alone. CSD skills of residents in both groups were assessed 2 months after the intervention using an 18 item behavioral checklist during a standardized patient encounter. Average scores for intervention and control group residents were calculated and between-group differences on the CSD skills assessment were evaluated using two-tailed independent sample t tests.
Results
Intervention group residents displayed higher overall scores on the simulated CSD (75.1% versus 53.2%, p<0.0001) than control group residents. The intervention group also displayed a greater number of key CSD communication behaviors and facilitated significantly longer conversations. The training, evaluation, and feedback sessions were rated highly.
Conclusion
A focused, multimodality curriculum can improve resident performance of simulated CSDs. Skill improvement lasted for at least 2 months after the intervention. Further studies are needed to assess skill retention and to set minimum performance standards.
doi:10.1089/jpm.2011.0446
PMCID: PMC3387757  PMID: 22690890
18.  Attitudes and experiences of residents in pursuit of postgraduate fellowships: A national survey of Canadian trainees 
Introduction:
There have been significant pressures on urology training in North America over the last decade due to both the constantly evolving skill set required and the external demands around delivery of urological care, particularly in Canada. We explore the attitudes and experience of Canadian urology residents toward their postgraduate decisions on fellowship opportunities.
Methods:
The study consisted of a self-report questionnaire of 4 separate cohorts of graduating urology residents from 2008 to 2011. The first cohort graduating in 2008 and 2009 were sent surveys through SurveyMonkey.com after graduation from residency; those graduating in 2010 and 2011 were prospectively invited as a convenience sample attending a Queen’s Urology Examination Skills Training Program review course just prior to graduation. The survey included both open- and closed-ended questions, employing a 5-point Likert scale, and explored the attitudes and experience of fellowship choices. Likert scores for each question were reported as means ± standard deviation (SD). Descriptive and correlative statistics were used to analyze the responses. In addition, an agreement score was created for those responding with “strongly agree” and “agree” on the Likert scale.
Results:
A total of 104 surveys were administered, with 84 respondents (80.8% response rate). As a whole, 84.9% of respondents agreed that they pursued fellowships; oncology and minimally invasive urology were the most popular choices throughout the 4 years. Respondents stated that reasons for pursuing a fellowship included: interest in pursuing an academic career (mean 3.73± 1.1 (SD): agreement score 61.1%) as well as acquiring marketable skills to obtain an urology position (3.59 ± 1.3: 64.4%). Most agreed or strongly agreed (84.9%) that a reason for pursing a fellowship was an interest in focusing their practice to this sub-specialty area. In comparison, most graduates disagreed that a reason for pursuing a fellowship was that residency did not equip them with the necessary skills to practice urology (2.49 ± 1.2: 19%). Most (81.2%) of graduates agreed they knew enough about academic urology to know if it would be a suitable career choice for them versus 54.7% regarding community urology (p < 0.0001). Surprisingly, only 61.7% of residents agreed that they completed a community elective during training, and most felt they would have benefited from additional elective time in the community.
Conclusions:
Urology residents graduating from Canadian programs pursue postgraduate training to enhance their surgical skill set and to achieve marketability, but also to facilitate a potential academic career. Responses from the trainees suggest that exposure to community practice appears suboptimal and may be an area of focus for programs to aid in career counselling and professional development.
doi:10.5489/cuaj.2136
PMCID: PMC4277525  PMID: 25553159
19.  Using the Cross-Cultural Care Survey to Assess Cultural Competency in Graduate Medical Education 
Background
Cultural competency is an important part of medical policy and practice, yet the evidence base for the effectiveness of training in this area is weak. One reason is the lack of valid, reliable, and feasible tools to quantify measures of knowledge, skill, and attitudes before and/or after cultural training. Given that cultural competency is a critical aspect of “professionalism” and “interpersonal and communication skills,” such a tool would aid in assessing the impact of such training in residency programs.
Objectives
The aim of this study is to enhance the feasibility and extend the validity of a tool to assess cultural competency in resident physicians. The work contributes to efforts to evaluate resident preparedness for working with diverse patient populations.
Method
Eighty-four residents (internal medicine, psychiatry, obstetrics-gynecology, and surgery) completed the Cross-Cultural Care Survey (CCCS) to assess their self-reported knowledge, skill, and attitudes regarding the provision of cross-cultural care. The study entailed descriptive analyses, factor analysis, internal consistency, and validity tests using bivariate correlations.
Results
Feasibility of using the CCCS was demonstrated with reduced survey completion time and ease of administration, and the survey reliably measures knowledge, skill, and attitudes for providing cross-cultural care. Resident characteristics and amount of postgraduate training relate differently to the 3 different subscales of the CCCS.
Conclusions
Our study confirmed that the CCCS is a reliable and valid tool to assess baseline attitudes of cultural competency across specialties in residency programs. Implications of the subscale scores for designing training programs are discussed.
doi:10.4300/JGME-D-09-00100.1
PMCID: PMC2931205  PMID: 21975893
20.  Assessing Intern Core Competencies With an Objective Structured Clinical Examination 
Background
Residents are evaluated using Accreditation Council for Graduate Medical Education (ACGME) core competencies. An Objective Structured Clinical Examination (OSCE) is a potential evaluation tool to measure these competencies and provide outcome data.
Objective
Create an OSCE to evaluate and demonstrate improvement in intern core competencies of patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice before and after internship.
Methods
From 2006 to 2008, 106 interns from 10 medical specialties were evaluated with a preinternship and postinternship OSCE at Madigan Army Medical Center. The OSCE included eight 12-minute stations that collectively evaluated the 6 ACGME core competencies using human patient simulators, standardized patients, and clinical scenarios. Interns were scored using objective and subjective criteria, with a maximum score of 100 for each competency. Stations included death notification, abdominal pain, transfusion consent, suture skills, wellness history, chest pain, altered mental status, and computer literature search. These stations were chosen by specialty program directors, created with input from board-certified specialists, and were peer reviewed.
Results
All OSCE testing on the 106 interns (ages 25 to 44 [average, 28.6]; 70 [66%] men; 65 [58%] allopathic medical school graduates) resulted in statistically significant improvement in all ACGME core competencies: patient care (71.9% to 80.0%, P < .001), medical knowledge (59.6% to 78.6%, P < .001), practice-based learning and improvement (45.2% to 63.0%, P < .001), interpersonal and communication skills (77.5% to 83.1%, P < .001), professionalism (74.8% to 85.1%, P < .001), and systems-based practice (56.6% to 76.5%, P < .001).
Conclusion
An OSCE during internship can evaluate incoming baseline ACGME core competencies and test for interval improvement. The OSCE is a valuable assessment tool to provide outcome measures on resident competency performance and evaluate program effectiveness.
doi:10.4300/01.01.0006
PMCID: PMC2931201  PMID: 21975704
21.  Rubric Evaluation of Pediatric Emergency Medicine Fellows 
Objectives
To develop and validate a rubric assessment instrument for use by pediatric emergency medicine (PEM) faculty to evaluate PEM fellows and for fellows to use to self-assess.
Methods
This is a prospective study at a PEM fellowship program. The assessment instrument was developed through a multistep process: (1) development of rubric format items, scaled on the modified Dreyfus model proficiency levels, corresponding to the 6 Accreditation Council for Graduate Medical Education core competencies; (2) determination of content and construct validity of the items through structured input and item refinement by subject matter experts and focus group review; (3) collection of data using a 61-item form; (4) evaluation of psychometrics; (5) selection of items for use in the final instrument.
Results
A total of 261 evaluations were collected from 2006 to 2007; exploratory factor analysis yielded 5 factors with Eigenvalues >1.0; each contained ≥4 items, with factor loadings >0.4 corresponding with the following competencies: (1) medical knowledge and practice-based learning and improvement, (2) patient care and systems-based practice, (3) interpersonal skills, (4) communication skills, and (5) professionalism. Cronbach α for the final 53-item instrument was 0.989. There was also significant responsiveness of the tool to the year of training.
Conclusion
A substantively and statistically validated rubric evaluation of PEM fellows is a reliable tool for formative and summative evaluation.
doi:10.4300/JGME-D-10-00083.1
PMCID: PMC3010934  PMID: 22132272
22.  Learning procedural skills in family medicine residency 
Canadian Family Physician  2006;52(5):622-623.
OBJECTIVE
To determine whether family medicine residents graduating from rural programs assess themselves as more experienced and competent in a range of procedural skills than graduates of urban programs do.
DESIGN
Self-administered written survey.
SETTING
Ontario.
PARTICIPANTS
Residents from 5 Ontario family medicine programs in 2000 and 2001; a total of 535 surveys were available for analysis (response rate of 78%).
MAIN OUTCOME MEASURES
Mean self-assessed experience and competence scores for 53 procedures at residency entry, end of year 1, and graduation.
RESULTS
Upon entry, there was no difference in mean procedural experience (2.89 vs 2.85, P = .54) or mean competence (2.34 vs 2.36, P = .88) scores between rural residents and their urban counterparts. There was a significant increase in procedural experience (P < .001) and competence (P < .001) scores during residency training. At graduation, mean experience (3.98 vs 3.70, P < .001) and competence (3.67 vs 3.39, P = .004) scores were significantly higher for rural residents than for their urban colleagues. A statistically larger proportion of residents graduating from rural programs assessed themselves as competent in 16 procedures. These included skills necessary for treating patients in emergency settings (establish intravenous lines for adults and infants, obtain arterial blood gas measurements, intubate adults and neonates, perform cautery for epistaxis, remove corneal foreign body, aspirate or inject knee and shoulder joints, and apply forearm or walking casts), for diagnostic procedures (endometrial biopsy and bone marrow aspiration), and for management of labour and delivery (vaginal delivery; vacuum extraction; and repair of first-, second-, and third-degree tears).
CONCLUSION
Graduates of rural programs who have had a substantial component of training in communities of fewer than 10 000 people report greater self-assessed experience and competence in procedural skills than graduates of urban programs do. The difference likely reflects the unique aspects of rural training sites, including preceptors’ competence in performing procedures.
PMCID: PMC1531718  PMID: 17327892
23.  Are we getting through? A national survey on the CanMEDS communicator role in urology residency 
Introduction:
Physician communication skills are paramount to patient satisfaction and are linked to important clinical outcomes. Although well-codified in the Royal College of Physicians and Surgeons of Canada (RCPSC) CanMEDS program, the knowledge, skills, and assessment of communication skills in surgical specialty training are rarely addressed. We assess Canadian urology residents’ experience of and attitudes towards this crucial competency in training and practice.
Methods:
An anonymous, cross-sectional, self-reported questionnaire was administered to all final year urology residents in Canada from 2 consecutive graduating years (2010 and 2011). A closed-ended 5-point Likert scale was used to assess familiarity with the concept of the RCPSC Communicator role and its application and importance to training and practice. Descriptive and correlative statistics were used to analyze the responses, such as the availability of formal training and resident participation in activities involving health communication. For ease of reporting, an agreement score was created for those responding with “strongly agree” and “agree” on the Likert scale.
Results:
There was a 100% response rate from the chief residents for both of the 2 years of the survey (n = 58). When questioned about the RCPSC CanMEDS roles, only 45% could identify the correct number of roles, and only 19% could correctly list all 7 roles. However, most residents were well aware of the Communicator role (90% agreement [mean 4.47 ± 0.78]), and most agreed that it plays an important role during training and future practice (83% [4.16 ± 0.84], 90% [4.39 ± 0.84] respectively). This is in stark contrast to perceived formal training. Only 31% (3.00 ± 1.04) agreed that formal training or mentorship in communication was available at their institution, and only 38% (3.14 ± 1.19) felt that communication had been formally addressed during explicit sessions. Despite most of the respondents agreeing they had a significant mentor/role model to emulate regarding communication skills, only 48% believed that faculty frequently addressed communication during clinical learning experiences.
Conclusions:
Despite knowledge and acceptance of the importance of the Communicator role, there is a perceived lack of formal and explicit training in this essential non-medical expert role of urology residency. It would seem apparent from this needs assessment that there may be an opportunity to coordinate efforts to ensure formal instruction and evaluation in our training programs.
doi:10.5489/cuaj.264
PMCID: PMC3876445  PMID: 24381664
24.  Differences in residents’ self-reported confidence and case experience between two post-graduate rotation curricula: results of a nationwide survey in Japan 
BMC Medical Education  2014;14:141.
Background
In Japan, all trainee physicians must begin clinical practice in a standardized, mandatory junior residency program, which encompasses the first two years of post-graduate medical training (PGY1 – PGY2). Implemented in 2004 to foster primary care skills, the comprehensive rotation program (CRP) requires junior residents to spend 14 months rotating through a comprehensive array of clinical departments including internal medicine, surgery, anesthesiology, obstetrics-gynecology (OBGYN), pediatrics, psychiatry, and rural medicine. In 2010, Japan’s health ministry relaxed this curricular requirement, allowing training programs to offer a limited rotation program (LRP), in which core departments constitute 10 months of training, with electives geared towards residents’ choice of career specialty comprising the remaining 14 months. The effectiveness of primary care skill acquisition during early training warrants evaluation. This study assesses self-reported confidence with clinical competencies, as well as case experience, between residents in CRP versus LRP curricula.
Methods
A nation-wide cross-sectional study of all PGY2 physicians in Japan was conducted in March 2011. Primary outcomes were self-report confidence for 98 clinical competency items, and number of cases experienced for 85 common diseases. We compared confidence scores and case experience between residents in CRP and LRP programs, adjusting for parameters relevant to training.
Results
Among 7506 PGY2 residents, 5052 replied to the survey (67.3%). Of 98 clinical competency items, CRP residents reported higher confidence in 12 items compared to those in an LRP curriculum, 10 of which remained significantly higher after adjustment. CRP trainees reported lower confidence scores in none of the items. Out of 85 diseases, LRP residents reported less experience with 11 diseases. CRP trainees reported lower case experience with one disease, though this did not remain significant on adjusted analysis. Confidence and case experience with OBGYN- and pediatrics-related items were particularly low among LRP trainees.
Conclusions
Residents in the specialty-oriented LRP curriculum showed less confidence and less case experience compared to peers training in the broader CRP residency curriculum. In order to foster competence in independent primary care practice, junior residency programs requiring experience in a breadth of core departments should continue to be mandated to ensure adequate primary care skills.
doi:10.1186/1472-6920-14-141
PMCID: PMC4105122  PMID: 25016304
Japanese junior residency education; Clinical competency
25.  A short questionnaire to assess pediatric resident’s competencies: the validation process 
Background
In order to help assess resident performance during training, the Residency Affair Committee of the Pediatric Residency Program of the University of Padua (Italy) administered a Resident Assessment Questionnaire (ReAQ), which both residents and faculty were asked to complete. The aim of this article is to present the ReAQ and its validation.
Methods
The ReAQ consists of 20 items that assess the six core competencies identified by the Accreditation Council of Graduate Medical Education (ACGME). A many-facet Rasch measurement analysis was used for validating the ReAQ.
Results
Between July 2011 and June 2012, 211 evaluations were collected from residents and faculty. Two items were removed because their functioning changed with the gender of respondents. The step calibrations were ordered. The self evaluations (residents rating themselves) positively correlated with the hetero evaluations (faculty rating residents; Spearman’s ρ = 0.75, p < 0.001). Unfortunately, the observed agreement among faculty was smaller than expected (Exp = 47.1%; Obs = 41%), which indicates that no enough training to faculty for using the tool was provided.
Conclusions
In its final form, the ReAQ provides a valid unidimensional measure of core competences in pediatric residents. It produces reliable measures, distinguishes among groups of residents according to different levels of performance, and provides a resident evaluation that holds an analogous meaning for residents and faculty.
doi:10.1186/1824-7288-39-41
PMCID: PMC3726326  PMID: 23830041
Resident; Pediatrics; Evaluation; Medical residency

Results 1-25 (1265431)