|Home | About | Journals | Submit | Contact Us | Français|
Competence in using an electronic health record (EHR) is considered a critical skill for physicians practicing in patient-centered medical homes (PCMHs), but few studies have examined the impact of EHR training for residents preparing to practice in PCMHs. This study explored the educational outcomes associated with comprehensive EHR training for family medicine residents.
The PCMH EHR training consisted of case-based routine clinic visits delivered to 3 resident cohorts (N = 18). Participants completed an EHR competency self-assessment between 2011 and 2016 (N = 127), examining 6 EHR/PCMH core skills. We compared baseline characteristics for residents by low vs high exposure to EHR training. Multivariate regression estimated whether self-reported competencies improved over time and whether high PCMH EHR training exposure was associated with incremental improvement in self-reported competencies over time.
Residents completed an average of 8.2 sessions: low-exposure residents averaged 5.3 sessions (standard deviation = 1.5), and high-exposure residents averaged 9.0 sessions (standard deviation = 0.9). High-exposed residents had higher posttest scores at training completion (84.4 vs 70.7). Over time, adjusted mean scores (confidence interval) for both groups improved (p < 0.001) from 12.2 (9.6–14.8), with low-exposed residents having greater score improvement (p < 0.001) because of their much lower baseline scores.
Comprehensive training designed to improve EHR competencies among residents practicing in a PCMH resulted in improved assessment scores. Our findings indicate EHR training as part of resident exposure to the PCMH measurably improves self-assessed competencies, even among residents less engaged in EHR training.
When introduced, electronic health records (EHRs) were regarded as key tools to improve the quality of health care delivery. However, the potential for EHRs to improve quality has been clouded because of growing frustration with unintended consequences, including decreased patient encounter duration, limited capacity to support care management, and technology implementation glitches.1–5 Nevertheless, the functionality and use of EHRs are increasingly important for health care practice and policy. Use of EHRs supports practice transformation efforts such as the patient-centered medical home (PCMH) model for chronic disease management and point-of-care decision making, aids in achieving financial incentives from the Centers for Medicare and Medicaid Services through Meaningful Use, and moves practices closer toward the Institute of Medicine’s vision of building learning health systems that adapt to our nation’s evolving health care needs.6–8
However, the content of existing EHR training curricula focuses predominantly on mastering software functionalities alone rather than applying the EHR to optimize the quality of patient care.9,10 Because EHR training content is generally not embedded in medical education curricula, training has primarily been delivered in limited, discreet sessions, and often taught as a standard Human Resources onboarding component rather than a professional competency.11 Moreover, assessing the impact of EHR training for future physicians is still in its infancy. Reports on EHR learners in both undergraduate medical education (UME) and graduate medical education (GME) state the tremendous value of being a competent EHR user, not only for measuring performance for the Accreditation Council for Graduate Medical Education competencies such as systems-based practice and practice-based learning but also to promote lifelong learning as future physicians enter practice.12–14 In recent years, studies on EHR curriculum in core skills for effective patient care and communication, clinical efficiency, and EHR functionality have been reported to be a useful foundation for teaching residents.12–15
Moreover, longitudinal and continuous training programs have been recently popularized in medical education. The cumulative exposure of learning has been shown to improve both learner outcomes and performance in health systems.15–17 Coupled with simulation training that provides immersive training that “replace and amplify real experiences with guided ones,” EHR training using clinical cases that simulate practice transformation, such as the PCMH, provides a platform to develop a resident’s knowledge and skills, with a potential added benefit to improve health systems.18,19
The purpose for this study is to describe the Group Health Family Medicine Residency (GHFMR) longitudinal PCMH case-based EHR curriculum and to assess the outcomes of the training on residents’ EHR self-assessment scores over time. The EHR training was grounded in clinical cases, taught through an accredited Accreditation Council for Graduate Medical Education program, delivered longitudinally during the three years of the family medicine residency program, and included EHR self-assessments delivered every trimester, totaling nine possible assessments per resident. We hypothesized that residents’ EHR training scores would improve in the context of longitudinal EHR training. We also hypothesized that residents with more exposure to EHR training sessions would achieve greater improvements over time compared with residents with less exposure to training.
The GHFMR is an 18-resident (6 per year of residency), community-based, urban GME program affiliated with the University of Washington School of Medicine. The GHFMR outpatient practice mirrors attending physicians in the sponsoring institution, Group Health Cooperative (Group Health), an integrated health care delivery system throughout Washington state. Residents are assigned a panel of patients through their training period of 3 years and use EpicCare (Epic Systems Corp, Verona, WI) for their EHR.
The GHFMR’s EHR is the foundational tool for Group Health’s PCMH, a nationally recognized practice transformation designed to achieve the Triple Aim.20–23 The PCMH optimizes health information technology to improve physician point-of-care decision making in the patient encounter, facilitates better chronic care management, enhances visit preparation, delivers expanded clinician access (secure messaging and telephone encounters), and increases capacity for patient outreach.21,24
From 2003 to 2009, GHFMR’s physician EHR training program was taught by an information technology professional without the knowledge or authority to provide trainees with clinical information on how to use the EHR to improve patient care. The focus of the training was on software navigation. In response to persistent negative program feedback from residents and faculty about EHR training insufficiencies coupled with the lack of a documented GME EHR curriculum in the literature, the GHFMR designed a longitudinal EHR training curriculum in conjunction with implementing the Group Health PCMH care model.25
The GHFMR EHR curriculum was implemented in 2010. All residents were scheduled to a three-hour training provided every trimester each year. The goals were derived from patient-centered care attributes to improve the core skills of EHR use with chronic disease management and team-based care, patient communication, software navigation, and clinical efficiency and quality. Appendix 1 (available online at: www.thepermanentejournal.org/files/2017/16-122-Appendix-1.pdf) describes the curriculum in detail.
All residents (N = 18) from 3 entering family medicine resident cohorts (2011, 2012, and 2013) each completed 3 years of longitudinal EHR training, for a total of 127 self-assessments. We anticipated heterogeneity in our incoming trainees’ EHR knowledge, which may include characteristic differences incorporated during UME; thus, we examined resident demographic differences at baseline.
We developed an EHR self-assessment instrument to measure EHR core clinical skills of central focus to PCMH. The online assessment is a 20-minute self-administered, 73-item form completed after each training session.
Cronbach α tests were performed for our EHR skill categories. We assessed an overall score (α = 0.99) and 6 separate skill categories: In-Basket Management (α = 0.95), Encounters (α = 0.97), Chart Review (α = 0.93), Other Activities (α = 0.93), Tools (α= 0.93), and Clinical Efficiency and Quality (α= 0.91). Resident core skills under these categories were assessed using a 4-item ordinal scale: 1) not proficient, 2) needs review, 3) proficient, and 4) mastery. We then converted the assessment scale (1–4) for each of the 6 core skill categories and overall EHR score to a standardized scale from 0 to 100. Table 1 summarizes the core skills categories.
All residents received training but differed in the number of sessions they attended. We compared residents by the number of EHR training sessions attended, a continuous measure, to clarify the extent to which intensity of exposure had an effect. For regression analyses, we used a dichotomous variable that categorized residents on the basis of median exposure level of eight sessions of a total of nine possible sessions. Residents who attended fewer than eight training sessions were categorized as “low exposed.” Residents attending eight or more sessions were categorized as “high exposed.”
Residents were categorized as having prior exposure to the EHR in UME vs no exposure. Residents who attended the University of Washington School of Medicine had prior exposure to the Group Health EHR while on their clinical rotations at our residency program. Residents not attending medical school at University of Washington were classified as not exposed to EHR training in UME, because these students did not rotate at GHFMR for their clinical rotations.
We compared age, sex, UME training location (locally trained vs not), and number of EHR training sessions attended. We performed t-tests for age and number of training sessions, and χ2 tests for sex and UME. Results for which p was less than 0.05 were considered statistically significant.
We conducted paired t-tests comparing low vs high exposure resident to EHR training sessions to analyze pretest and posttest mean scores from the baseline assessment to final EHR training assessment. Then, we estimated multivariate linear regression models to examine whether overall EHR scores improved over time, as well as for the six core skill categories. These regression models controlled for EHR exposure in UME, but age and sex were not included as control variables because of collinearity concerns with the modest sample size of residents.26 An interaction examining term EHR training exposure and time was also included in the regression models to examine whether residents with high exposure to EHR sessions improved scores more over time relative to low-exposure residents.
Data were analyzed using Stata/IC software, Version 14 (StataCorp LLC, College Station, TX) and Tableau 10.0 (Tableau Software, Seattle, WA). The curriculum, self-assessment tool, and analysis were approved by the Group Health Human Subjects Division.
Table 2 summarizes resident characteristics from the 3 cohorts of the GHFMR. Most residents were women (13/18; 72.2%), with a mean age of 28.4 (standard deviation [SD] = 0.50) years. Five (33.3%) of 18 residents had previous EHR exposure through local UME training. No significant differences in residents’ characteristics by EHR training exposure were found. Residents completed an average of 8.2 sessions: low-exposure residents averaged 5.3 sessions (SD = 1.5) and high-exposure residents averaged 9.0 sessions (SD = 0.9).
Table 3 summarizes residents’ pre- and posttest EHR competency scores at baseline and training completion, by high vs low exposure. For each EHR core skill category, low-exposed residents reported baseline scores ranging from 5.9 to 38.5 and completion scores ranging from 55.6 to 89.3, with improvement for all categories demonstrating statistically significant improvement. The mean overall EHR score improvement for low-exposed residents was 56.0 points (p < 0.001). High-exposed residents reported higher baseline scores from 35.9 to 48.5 and completion scores ranging from 73.9 to 97.7, with all categories statistically significant. The mean overall EHR score improvement for high-exposed residents was 40.9 points (p < 0.001). Although low-exposed residents reported lower baseline scores and greater improvement in scores than the high-exposed subgroup, high-exposed residents reported overall higher posttest scores at training completion.
Figure 1 compares the pre- and posttest mean scores for each category by low- and high-exposed resident at baseline and completion.
Our multivariate regression analysis estimated the effect of EHR training over time, controlling for UME exposure to EHR training. Table 4 reports the over-time effect, high- vs low-exposure effect, and the incremental effect of high exposure. For every session a resident attended, improvements ranged from 6.7 to 14.4 points across the 6 skill categories. For the overall EHR score, an improvement score of 12.2 for each session was estimated (p < 0.001). High-exposed residents were more likely to achieve greater improvement for each of the core skill categories (12.9 to 39.8) and more likely to have overall EHR score improvement of 28.3 (p < 0.001). The interaction results examining differential improvement over time for residents with high vs low EHR training exposure confirm the pattern observed in unadjusted analyses; residents with reported low exposure had more improvement in competencies over time, but this was largely driven by low baseline scores.
We described a longitudinal EHR training using PCMH case-based content, analyzed self-assessed EHR scores, and found that training improved self-reported EHR competencies over time. We designed a case-based EHR longitudinal training curriculum that includes a step-by-step breakdown of the PCMH clinic visit through outpatient teaching cases. We also developed a self-assessed EHR training instrument with a convenient standardized scale (0–100) that measures both core skill categories and an overall EHR training score. This could provide utility in developing faculty/trainer-based assessments for EHR training programs that wish to assess over-time EHR skills in a PCMH.
The Group Health PCMH practice transformation aims to facilitate various types of physician access to patients, including both face-to-face and electronic visits.25 Given a recent study that reported residents spent a 1:5 ratio with patient time vs computer time, training residents to master technologies in the clinical setting has key implications for physicians-in-training and for lifelong learning as they continue to use the EHR in practice.27 Our results argue that EHR training that simulates the PCMH through a case-based curriculum should be incorporated into physician EHR training. The lack of clinically relevant training content in EHR training from our earlier experiences speaks to this experience, and relates to policy efforts to facilitate EHR meaningful use in practice.28–30
Our findings also support the growing medical education literature that longitudinal or cumulative exposure to learning experiences influences rates of learning.15–17,31 Our analysis for differing learning rates based on high vs low exposure to EHR training helps estimate the value of longitudinal training for residents, even when residents are unable to fully engage in all sessions. Low-exposed residents who reported the lowest scores at baseline may be less engaged at the start of training or uncertain about their EHR core skills but subsequently found EHR training more helpful as they attended more sessions.32 Our results indicate that our residents report improvements with their EHR skills, thus speaking to the potential value of EHR training in the core skills related to a PCMH.
This study has important limitations. First, generalizability to other training settings may be limited because of our small sample size in a single GME program, our training program designed around the PCMH model, and self-assessed competencies that are subject to reporting or social desirability bias. Second, we had no control group because the EHR curriculum was a foundational aspect of orientation to the PCMH and it would be impractical to randomize trainees in experimental groups. Despite the limitations of not having a control group, our analyses highlight that residents report improved competencies from EHR training and that improvement is possible for residents with less-than-optimal exposure to the sessions. Our next steps are to explore these differences influenced by EHR training exposure and to study the link between self-assessed resident competencies to faculty-assigned competency achievements and clinical outcome measures related to the PCMH.
Importantly, we found evidence that early physician engagement in longitudinal EHR training can substantially improve EHR self-reported competency scores among family medicine residents. The results highlight the need to incorporate medical education when implementing new technologies and for health systems undergoing practice changes such as the PCMH.
The art of medicine was to be properly earned from its practice and its exercise.— Thomas Sydenham, 1624–1689, English physician, known as “The English Hippocrates”
The authors thank Robert Winter and Chris Thayer, MD, for their efforts with the development and implementation of the curriculum.
This work was presented at the American Academy of Family Physicians Workshop for Directors of Family Medicine Residencies (PDW), June 3 to 5, 2012, and April 5 to 7, 2013, Kansas City, MO; and at the Society of Teachers of Family Medicine 46th Annual Spring Conference, May 1 to 5, 2013, Baltimore, MD.
Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.
A spectrum of outpatient care cases with the clinic visit, broken into 8 routine steps, and residents role-play a stepwise approach to the visit (Figure A). Residents are provided single-problem patient visits and progressively work through complex visits, preventive visits, telephone visits, secure messages, and inbox management. Clinical efficiency training includes touch typing, recording and eliciting the history simultaneously, writing orders with the patient, making the visit summary assessment and plan, “close as you go” charting, keeping the in-basket clean, and promoting clinic flow efficiency.
Group Health’s integrated care delivery model and the Family Medicine Residency are based on the patient-centered medical home (Figure B). Patient-centered teaching topics include agenda setting using the computer, clarifying a roadmap for the patient visit, creating patient care plans using the computer, patient empowerment and motivational interviewing by creating the visit summary, and multimodal patient education (“say it, type it, read it, share it”). See Table A.
From the start of residency training, three-hour trimester sessions are taught over three years, and divided into two parts: 1) structured curriculum and 2) self-assessed competency-driven topics. The structured curriculum covers basics of the six core skill categories listed below. The second half of each training session addresses the areas of relative weakness identified from the most recent completed self-assessments.
We designed an EHR competency self-assessment tool that integrates EHR core clinical skills using the objectives from 1 to 3 above. The competency assessment is a 20-minute self-administered, online 73-item form completed after each session. It assesses 6 core skill domains: In-Basket Management, Encounters, Chart Review, Other Activities, Tools, and Clinical Efficiency/Quality:
|Skill taught||Example||PCMH principlesa||EHR integration|
|Agenda setting||“I’d like to discuss your diabetes today. I understand you’d like me to look at a mole. Is there anything else we should work on today?”||Elicit patient’s agenda items at the beginning of the visit.|
PCMH Principles 1 and 2
|Use computer monitor, enter known diagnoses (especially preexisting chronic conditions).|
|Visit roadmap||“Let’s start by discussing each problem, then I’ll do an exam, then we’ll make a plan together.”||Communicate the structure of the visit, so the patient knows what to expect.|
PCMH Principles 1 and 2
|Create a care plan||Diabetes plan includes hemoglobin A1C/ cholesterol/blood pressure goals, frequency of visits, medications and doses, exercise, and food plans.||Make the plan with the patient and give him/her a copy. Write exercise and food plans in the patient’s own words, but advise him/her on it.|
PCMH Principles 1 and 2
|Care plans for chronic illnesses go in the patient problem list.|
|Motivational interviewing||“You’ve told me you’re interested in losing weight. What are some ways you plan to work on that?”||Activate the patient to work on his/her lifestyle choices that are negatively affecting his/her health.|
PCMH Principles 1 and 2
|Type the plan directly into the visit summary.|
Read your recommendations aloud as you type them.
Type the patient’s words from motivational interviewing directly into his/her plan.
Summary is printed for patient and electronically copied into your note.
|Multimodal patient communication||As the plan is made, type it on the screen so the patient can read it, and speak it out loud so s/he can hear it. Then print it out so s/he can read it again later.|
“Speak it, type it, share it.”
|Excellent patient communication|
PCMH Principles 1 and 2
|Initiate and respond to electronic patient messages||Patient asks for clarification after a visit and gets a rapid response from his/her physician||Medical care is accessible, including online.|
PCMH Principle 3
|Respond to patient secure messages promptly and professionally.|
New messages are written to notify patients of laboratory results or other follow-up.
|Telephone visits||Clinician conducts a scheduled phone visit to follow up on depression symptoms 2 weeks after initiating SSRI therapy.||Medical care is accessible, including by telephone.|
PCMH Principle 3
|Telephone visit is structured and clearly documented in a Telephone Encounter in the EHR.|
|Staff communication||Clinician sends the clinic RN a brief note asking him to review a new diabetic patient’s records for any missing preventive health needs.||Primary care clinician is the central organizer of care and relies on the team to help implement care.|
PCMH Principle 4
|Staff messages and result notes are used.|
Telephone Encounters can be routed to another staff member.
|Chronic disease panel management||RN and clinician sit down together with list of diabetic patients on the clinician’s panel and review and adjust care plans.||PCMH Principle 5||EHR is queried on populations with specific diagnoses, with quality measures.|
Chart Problem list is kept up to date.
EHR = electronic health record; PCMH = patient-centered medical home; RN = registered nurse; SSRI = selective serotonin reuptake inhibitor.
The author(s) have no conflicts of interest to disclose.
The Group Health Human Subjects Division granted an exemption for the use of human subjects for this study on September 3, 2013.