PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of permanentejHomeEditorial BoardCurrent TOCSubscribeSubmit
 
Perm J. 2017; 21: 16-122.
Published online 2017 July 7. doi:  10.7812/TPP/16-122
PMCID: PMC5528802

Impact of Longitudinal Electronic Health Record Training for Residents Preparing for Practice in Patient-Centered Medical Homes

Jung G Kim, MPH, CPH
Clinical Teaching Associate in the Department of Family Medicine at the University of Washington School of Medicine in Seattle. E-mail: gro.pk@mik.g.gnuj.
Hector P Rodriguez, PhD, MPH
Co-Director for the Center of Healthcare Organizational Innovation Research and a Professor of Health Policy and Management at the University of California School of Public Health in Berkeley. E-mail: ude.yelekreb@dorh.
Katherine AT Estlin, MD
Physician at Open Door Community Health Centers in Arcata, CA. E-mail: moc.liamg@niltsek.
Carl G Morris, MD, MPH

Abstract

Introduction

Competence in using an electronic health record (EHR) is considered a critical skill for physicians practicing in patient-centered medical homes (PCMHs), but few studies have examined the impact of EHR training for residents preparing to practice in PCMHs. This study explored the educational outcomes associated with comprehensive EHR training for family medicine residents.

Methods

The PCMH EHR training consisted of case-based routine clinic visits delivered to 3 resident cohorts (N = 18). Participants completed an EHR competency self-assessment between 2011 and 2016 (N = 127), examining 6 EHR/PCMH core skills. We compared baseline characteristics for residents by low vs high exposure to EHR training. Multivariate regression estimated whether self-reported competencies improved over time and whether high PCMH EHR training exposure was associated with incremental improvement in self-reported competencies over time.

Results

Residents completed an average of 8.2 sessions: low-exposure residents averaged 5.3 sessions (standard deviation = 1.5), and high-exposure residents averaged 9.0 sessions (standard deviation = 0.9). High-exposed residents had higher posttest scores at training completion (84.4 vs 70.7). Over time, adjusted mean scores (confidence interval) for both groups improved (p < 0.001) from 12.2 (9.6–14.8), with low-exposed residents having greater score improvement (p < 0.001) because of their much lower baseline scores.

Conclusion

Comprehensive training designed to improve EHR competencies among residents practicing in a PCMH resulted in improved assessment scores. Our findings indicate EHR training as part of resident exposure to the PCMH measurably improves self-assessed competencies, even among residents less engaged in EHR training.

INTRODUCTION

When introduced, electronic health records (EHRs) were regarded as key tools to improve the quality of health care delivery. However, the potential for EHRs to improve quality has been clouded because of growing frustration with unintended consequences, including decreased patient encounter duration, limited capacity to support care management, and technology implementation glitches.15 Nevertheless, the functionality and use of EHRs are increasingly important for health care practice and policy. Use of EHRs supports practice transformation efforts such as the patient-centered medical home (PCMH) model for chronic disease management and point-of-care decision making, aids in achieving financial incentives from the Centers for Medicare and Medicaid Services through Meaningful Use, and moves practices closer toward the Institute of Medicine’s vision of building learning health systems that adapt to our nation’s evolving health care needs.68

However, the content of existing EHR training curricula focuses predominantly on mastering software functionalities alone rather than applying the EHR to optimize the quality of patient care.9,10 Because EHR training content is generally not embedded in medical education curricula, training has primarily been delivered in limited, discreet sessions, and often taught as a standard Human Resources onboarding component rather than a professional competency.11 Moreover, assessing the impact of EHR training for future physicians is still in its infancy. Reports on EHR learners in both undergraduate medical education (UME) and graduate medical education (GME) state the tremendous value of being a competent EHR user, not only for measuring performance for the Accreditation Council for Graduate Medical Education competencies such as systems-based practice and practice-based learning but also to promote lifelong learning as future physicians enter practice.1214 In recent years, studies on EHR curriculum in core skills for effective patient care and communication, clinical efficiency, and EHR functionality have been reported to be a useful foundation for teaching residents.1215

Moreover, longitudinal and continuous training programs have been recently popularized in medical education. The cumulative exposure of learning has been shown to improve both learner outcomes and performance in health systems.1517 Coupled with simulation training that provides immersive training that “replace and amplify real experiences with guided ones,” EHR training using clinical cases that simulate practice transformation, such as the PCMH, provides a platform to develop a resident’s knowledge and skills, with a potential added benefit to improve health systems.18,19

The purpose for this study is to describe the Group Health Family Medicine Residency (GHFMR) longitudinal PCMH case-based EHR curriculum and to assess the outcomes of the training on residents’ EHR self-assessment scores over time. The EHR training was grounded in clinical cases, taught through an accredited Accreditation Council for Graduate Medical Education program, delivered longitudinally during the three years of the family medicine residency program, and included EHR self-assessments delivered every trimester, totaling nine possible assessments per resident. We hypothesized that residents’ EHR training scores would improve in the context of longitudinal EHR training. We also hypothesized that residents with more exposure to EHR training sessions would achieve greater improvements over time compared with residents with less exposure to training.

METHODS

Setting and Participants

The GHFMR is an 18-resident (6 per year of residency), community-based, urban GME program affiliated with the University of Washington School of Medicine. The GHFMR outpatient practice mirrors attending physicians in the sponsoring institution, Group Health Cooperative (Group Health), an integrated health care delivery system throughout Washington state. Residents are assigned a panel of patients through their training period of 3 years and use EpicCare (Epic Systems Corp, Verona, WI) for their EHR.

The GHFMR’s EHR is the foundational tool for Group Health’s PCMH, a nationally recognized practice transformation designed to achieve the Triple Aim.2023 The PCMH optimizes health information technology to improve physician point-of-care decision making in the patient encounter, facilitates better chronic care management, enhances visit preparation, delivers expanded clinician access (secure messaging and telephone encounters), and increases capacity for patient outreach.21,24

From 2003 to 2009, GHFMR’s physician EHR training program was taught by an information technology professional without the knowledge or authority to provide trainees with clinical information on how to use the EHR to improve patient care. The focus of the training was on software navigation. In response to persistent negative program feedback from residents and faculty about EHR training insufficiencies coupled with the lack of a documented GME EHR curriculum in the literature, the GHFMR designed a longitudinal EHR training curriculum in conjunction with implementing the Group Health PCMH care model.25

The GHFMR EHR curriculum was implemented in 2010. All residents were scheduled to a three-hour training provided every trimester each year. The goals were derived from patient-centered care attributes to improve the core skills of EHR use with chronic disease management and team-based care, patient communication, software navigation, and clinical efficiency and quality. Appendix 1 (available online at: www.thepermanentejournal.org/files/2017/16-122-Appendix-1.pdf) describes the curriculum in detail.

Data Sources

All residents (N = 18) from 3 entering family medicine resident cohorts (2011, 2012, and 2013) each completed 3 years of longitudinal EHR training, for a total of 127 self-assessments. We anticipated heterogeneity in our incoming trainees’ EHR knowledge, which may include characteristic differences incorporated during UME; thus, we examined resident demographic differences at baseline.

We developed an EHR self-assessment instrument to measure EHR core clinical skills of central focus to PCMH. The online assessment is a 20-minute self-administered, 73-item form completed after each training session.

Measures

Electronic Health Record Competencies

Cronbach α tests were performed for our EHR skill categories. We assessed an overall score (α = 0.99) and 6 separate skill categories: In-Basket Management (α = 0.95), Encounters (α = 0.97), Chart Review (α = 0.93), Other Activities (α = 0.93), Tools (α= 0.93), and Clinical Efficiency and Quality (α= 0.91). Resident core skills under these categories were assessed using a 4-item ordinal scale: 1) not proficient, 2) needs review, 3) proficient, and 4) mastery. We then converted the assessment scale (1–4) for each of the 6 core skill categories and overall EHR score to a standardized scale from 0 to 100. Table 1 summarizes the core skills categories.

Table 1
Electronic health record skill categories assessment with internal consistency reliability statistics

Training Exposure

All residents received training but differed in the number of sessions they attended. We compared residents by the number of EHR training sessions attended, a continuous measure, to clarify the extent to which intensity of exposure had an effect. For regression analyses, we used a dichotomous variable that categorized residents on the basis of median exposure level of eight sessions of a total of nine possible sessions. Residents who attended fewer than eight training sessions were categorized as “low exposed.” Residents attending eight or more sessions were categorized as “high exposed.”

Medical School Electronic Health Record Exposure

Residents were categorized as having prior exposure to the EHR in UME vs no exposure. Residents who attended the University of Washington School of Medicine had prior exposure to the Group Health EHR while on their clinical rotations at our residency program. Residents not attending medical school at University of Washington were classified as not exposed to EHR training in UME, because these students did not rotate at GHFMR for their clinical rotations.

Statistical Analyses

We compared age, sex, UME training location (locally trained vs not), and number of EHR training sessions attended. We performed t-tests for age and number of training sessions, and χ2 tests for sex and UME. Results for which p was less than 0.05 were considered statistically significant.

We conducted paired t-tests comparing low vs high exposure resident to EHR training sessions to analyze pretest and posttest mean scores from the baseline assessment to final EHR training assessment. Then, we estimated multivariate linear regression models to examine whether overall EHR scores improved over time, as well as for the six core skill categories. These regression models controlled for EHR exposure in UME, but age and sex were not included as control variables because of collinearity concerns with the modest sample size of residents.26 An interaction examining term EHR training exposure and time was also included in the regression models to examine whether residents with high exposure to EHR sessions improved scores more over time relative to low-exposure residents.

Data were analyzed using Stata/IC software, Version 14 (StataCorp LLC, College Station, TX) and Tableau 10.0 (Tableau Software, Seattle, WA). The curriculum, self-assessment tool, and analysis were approved by the Group Health Human Subjects Division.

RESULTS

Group Health Residents and Electronic Health Record Training Characteristics

Table 2 summarizes resident characteristics from the 3 cohorts of the GHFMR. Most residents were women (13/18; 72.2%), with a mean age of 28.4 (standard deviation [SD] = 0.50) years. Five (33.3%) of 18 residents had previous EHR exposure through local UME training. No significant differences in residents’ characteristics by EHR training exposure were found. Residents completed an average of 8.2 sessions: low-exposure residents averaged 5.3 sessions (SD = 1.5) and high-exposure residents averaged 9.0 sessions (SD = 0.9).

Table 2
Comparison of family medicine resident demographics with low and high exposure to electronic health record training (2011–2016)

Pre- and Posttest Scores by Level of EHR Training Exposure

Table 3 summarizes residents’ pre- and posttest EHR competency scores at baseline and training completion, by high vs low exposure. For each EHR core skill category, low-exposed residents reported baseline scores ranging from 5.9 to 38.5 and completion scores ranging from 55.6 to 89.3, with improvement for all categories demonstrating statistically significant improvement. The mean overall EHR score improvement for low-exposed residents was 56.0 points (p < 0.001). High-exposed residents reported higher baseline scores from 35.9 to 48.5 and completion scores ranging from 73.9 to 97.7, with all categories statistically significant. The mean overall EHR score improvement for high-exposed residents was 40.9 points (p < 0.001). Although low-exposed residents reported lower baseline scores and greater improvement in scores than the high-exposed subgroup, high-exposed residents reported overall higher posttest scores at training completion.

Table 3
Pre- and posttest unadjusted mean scores by low and high exposure to electronic health record training (EHR) training (2011–2016)

Figure 1 compares the pre- and posttest mean scores for each category by low- and high-exposed resident at baseline and completion.

Figure 1
Pretest and posttest unadjusted mean scores for low and high exposure to electronic health record (EHR) training (2011–2016).a

Multivariate Analyses

Our multivariate regression analysis estimated the effect of EHR training over time, controlling for UME exposure to EHR training. Table 4 reports the over-time effect, high- vs low-exposure effect, and the incremental effect of high exposure. For every session a resident attended, improvements ranged from 6.7 to 14.4 points across the 6 skill categories. For the overall EHR score, an improvement score of 12.2 for each session was estimated (p < 0.001). High-exposed residents were more likely to achieve greater improvement for each of the core skill categories (12.9 to 39.8) and more likely to have overall EHR score improvement of 28.3 (p < 0.001). The interaction results examining differential improvement over time for residents with high vs low EHR training exposure confirm the pattern observed in unadjusted analyses; residents with reported low exposure had more improvement in competencies over time, but this was largely driven by low baseline scores.

Table 4
Multivariate regression analysesa for score improvements with individual electronic health record training (EHR) assessment categories and total EHR score (N = 127 exposures)

DISCUSSION

We described a longitudinal EHR training using PCMH case-based content, analyzed self-assessed EHR scores, and found that training improved self-reported EHR competencies over time. We designed a case-based EHR longitudinal training curriculum that includes a step-by-step breakdown of the PCMH clinic visit through outpatient teaching cases. We also developed a self-assessed EHR training instrument with a convenient standardized scale (0–100) that measures both core skill categories and an overall EHR training score. This could provide utility in developing faculty/trainer-based assessments for EHR training programs that wish to assess over-time EHR skills in a PCMH.

The Group Health PCMH practice transformation aims to facilitate various types of physician access to patients, including both face-to-face and electronic visits.25 Given a recent study that reported residents spent a 1:5 ratio with patient time vs computer time, training residents to master technologies in the clinical setting has key implications for physicians-in-training and for lifelong learning as they continue to use the EHR in practice.27 Our results argue that EHR training that simulates the PCMH through a case-based curriculum should be incorporated into physician EHR training. The lack of clinically relevant training content in EHR training from our earlier experiences speaks to this experience, and relates to policy efforts to facilitate EHR meaningful use in practice.2830

Our findings also support the growing medical education literature that longitudinal or cumulative exposure to learning experiences influences rates of learning.1517,31 Our analysis for differing learning rates based on high vs low exposure to EHR training helps estimate the value of longitudinal training for residents, even when residents are unable to fully engage in all sessions. Low-exposed residents who reported the lowest scores at baseline may be less engaged at the start of training or uncertain about their EHR core skills but subsequently found EHR training more helpful as they attended more sessions.32 Our results indicate that our residents report improvements with their EHR skills, thus speaking to the potential value of EHR training in the core skills related to a PCMH.

This study has important limitations. First, generalizability to other training settings may be limited because of our small sample size in a single GME program, our training program designed around the PCMH model, and self-assessed competencies that are subject to reporting or social desirability bias. Second, we had no control group because the EHR curriculum was a foundational aspect of orientation to the PCMH and it would be impractical to randomize trainees in experimental groups. Despite the limitations of not having a control group, our analyses highlight that residents report improved competencies from EHR training and that improvement is possible for residents with less-than-optimal exposure to the sessions. Our next steps are to explore these differences influenced by EHR training exposure and to study the link between self-assessed resident competencies to faculty-assigned competency achievements and clinical outcome measures related to the PCMH.

Importantly, we found evidence that early physician engagement in longitudinal EHR training can substantially improve EHR self-reported competency scores among family medicine residents. The results highlight the need to incorporate medical education when implementing new technologies and for health systems undergoing practice changes such as the PCMH.

Practice and Exercise

The art of medicine was to be properly earned from its practice and its exercise.

— Thomas Sydenham, 1624–1689, English physician, known as “The English Hippocrates”

Acknowledgments

The authors thank Robert Winter and Chris Thayer, MD, for their efforts with the development and implementation of the curriculum.

This work was presented at the American Academy of Family Physicians Workshop for Directors of Family Medicine Residencies (PDW), June 3 to 5, 2012, and April 5 to 7, 2013, Kansas City, MO; and at the Society of Teachers of Family Medicine 46th Annual Spring Conference, May 1 to 5, 2013, Baltimore, MD.

Kathleen Louden, ELS, of Louden Health Communications provided editorial assistance.

APPENDIX 1. GROUP HEALTH FAMILY MEDICINE RESIDENCY PATIENT-CENTERED MEDICAL HOME ELECTRONIC HEALTH RECORD CURRICULUM

1. Teach in a clinical context

A spectrum of outpatient care cases with the clinic visit, broken into 8 routine steps, and residents role-play a stepwise approach to the visit (Figure A). Residents are provided single-problem patient visits and progressively work through complex visits, preventive visits, telephone visits, secure messages, and inbox management. Clinical efficiency training includes touch typing, recording and eliciting the history simultaneously, writing orders with the patient, making the visit summary assessment and plan, “close as you go” charting, keeping the in-basket clean, and promoting clinic flow efficiency.

2. Integrate with the Group Health patient-centered medical home and patient care

Group Health’s integrated care delivery model and the Family Medicine Residency are based on the patient-centered medical home (Figure B). Patient-centered teaching topics include agenda setting using the computer, clarifying a roadmap for the patient visit, creating patient care plans using the computer, patient empowerment and motivational interviewing by creating the visit summary, and multimodal patient education (“say it, type it, read it, share it”). See Table A.

3. Provide training longitudinally

From the start of residency training, three-hour trimester sessions are taught over three years, and divided into two parts: 1) structured curriculum and 2) self-assessed competency-driven topics. The structured curriculum covers basics of the six core skill categories listed below. The second half of each training session addresses the areas of relative weakness identified from the most recent completed self-assessments.

4. Measure core electronic health record (EHR) competency skills from self-assessment

We designed an EHR competency self-assessment tool that integrates EHR core clinical skills using the objectives from 1 to 3 above. The competency assessment is a 20-minute self-administered, online 73-item form completed after each session. It assesses 6 core skill domains: In-Basket Management, Encounters, Chart Review, Other Activities, Tools, and Clinical Efficiency/Quality:

  • In-Basket Management: Functions relating to secure electronic communication with care team and patients; physician-based ordering
  • Encounters: Using the EHR during a patient encounter
  • Chart Review: Patient clinical information, test results
  • Other Activities: Patient information related to preventive care, panel management
  • Tools: EHR software functions to assist with patient care
  • Clinical Efficiency and Quality: Clinical efficiency training as described in 1 above.

Figure A

An external file that holds a picture, illustration, etc.
Object name is 16-122f2.jpg

Routine steps for an outpatient visit at Group Health Family Medicine Residency.

Figure B

An external file that holds a picture, illustration, etc.
Object name is 16-122f3.jpg

Principles of Group Health Cooperative’s patient-centered medical home.

Table A

Patient-centered medical home principles with electronic health record integration and skills taught

Skill taughtExamplePCMH principlesaEHR integration
Agenda setting“I’d like to discuss your diabetes today. I understand you’d like me to look at a mole. Is there anything else we should work on today?”Elicit patient’s agenda items at the beginning of the visit.
PCMH Principles 1 and 2
Use computer monitor, enter known diagnoses (especially preexisting chronic conditions).
Visit roadmap“Let’s start by discussing each problem, then I’ll do an exam, then we’ll make a plan together.”Communicate the structure of the visit, so the patient knows what to expect.
PCMH Principles 1 and 2
Create a care planDiabetes plan includes hemoglobin A1C/ cholesterol/blood pressure goals, frequency of visits, medications and doses, exercise, and food plans.Make the plan with the patient and give him/her a copy. Write exercise and food plans in the patient’s own words, but advise him/her on it.
PCMH Principles 1 and 2
Care plans for chronic illnesses go in the patient problem list.
Motivational interviewing“You’ve told me you’re interested in losing weight. What are some ways you plan to work on that?”Activate the patient to work on his/her lifestyle choices that are negatively affecting his/her health.
PCMH Principles 1 and 2
Type the plan directly into the visit summary.
Read your recommendations aloud as you type them.
Type the patient’s words from motivational interviewing directly into his/her plan.
Summary is printed for patient and electronically copied into your note.
Multimodal patient communicationAs the plan is made, type it on the screen so the patient can read it, and speak it out loud so s/he can hear it. Then print it out so s/he can read it again later.
“Speak it, type it, share it.”
Excellent patient communication
PCMH Principles 1 and 2
Initiate and respond to electronic patient messagesPatient asks for clarification after a visit and gets a rapid response from his/her physicianMedical care is accessible, including online.
PCMH Principle 3
Respond to patient secure messages promptly and professionally.
New messages are written to notify patients of laboratory results or other follow-up.
Telephone visitsClinician conducts a scheduled phone visit to follow up on depression symptoms 2 weeks after initiating SSRI therapy.Medical care is accessible, including by telephone.
PCMH Principle 3
Telephone visit is structured and clearly documented in a Telephone Encounter in the EHR.
Staff communicationClinician sends the clinic RN a brief note asking him to review a new diabetic patient’s records for any missing preventive health needs.Primary care clinician is the central organizer of care and relies on the team to help implement care.
PCMH Principle 4
Staff messages and result notes are used.
Telephone Encounters can be routed to another staff member.
Chronic disease panel managementRN and clinician sit down together with list of diabetic patients on the clinician’s panel and review and adjust care plans.PCMH Principle 5EHR is queried on populations with specific diagnoses, with quality measures.
Chart Problem list is kept up to date.
aSee Figure B.

EHR = electronic health record; PCMH = patient-centered medical home; RN = registered nurse; SSRI = selective serotonin reuptake inhibitor.

Footnotes

Disclosure Statement

The author(s) have no conflicts of interest to disclose.

Ethical Approval

The Group Health Human Subjects Division granted an exemption for the use of human subjects for this study on September 3, 2013.

References

1. Alkureishi MA, Lee WW, Lyons M, et al. Impact of electronic medical record use on the patient-doctor relationship and communication: A systematic review. J Gen Intern Med. 2016 May;31(5):548–60. DOI: https://doi.org/10.1007/s11606-015-3582-1. [PMC free article] [PubMed]
2. Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010 Aug 6;10:231. DOI: https://doi.org/10.1186/1472-6963-10-231. [PMC free article] [PubMed]
3. Hsu J, Huang J, Fung V, Robertson N, Jimison H, Frankel R. Health information technology and physician-patient interactions: Impact of computers on communication during outpatient primary care visits. J Am Med Inform Assoc. 2005 Jul-Aug;12(4):474–80. DOI: https://doi.org/10.1197/jamia.m1741. [PMC free article] [PubMed]
4. Nagy VT, Kanter MH. Implementing the electronic medical record in the exam room: The effect on physician-patient communication and patient satisfaction. Perm J. 2007 Spring;11(2):21–4. DOI: https://doi.org/10.7812/tpp/06-118. [PMC free article] [PubMed]
5. Rizer MK, Kaufman B, Sieck CJ, Hefner JL, McAlearney AS. Top 10 lessons learned from electronic medical record implementation in a large academic medical center. Perspect Health Inf Manag. 2015 Jul 1;12:1g. [PMC free article] [PubMed]
6. Kern LM, Edwards A, Kaushal R. The patient-centered medical home, electronic health records, and quality of care. Ann Intern Med. 2014 Jun 3;160(11):741–9. DOI: https://doi.org/10.7326/m13-1798. [PubMed]
7. CMS.gov. Electronic health records (EHR) incentive programs [Internet] Baltimore, MD: Centers for Medicare & Medicaid Services; updated 2017 Feb 8 [cited 2016 Dec 30]. Available from: www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/index.html?redirect=/EHRIncentivePrograms/30_Meaningful_Use.asp.
8. Institute of Medicine of the National Academies. Best care at lower cost: The path to continuously learning health care in America. Washington, DC: The National Academies Press; 2013.
9. Duke P, Frankel RM, Reis S. How to integrate the electronic health record and patient-centered communication into the medical visit: A skills-based approach. Teach Learn Med. 2013;25(4):358–65. DOI: https://doi.org/10.1080/10401334.2013.827981. [PubMed]
10. Krist AH. Electronic health record innovations for healthier patients and happier doctors. J Am Board Fam Med. 2015 May-Jun;28(3):299–302. DOI: https://doi.org/10.3122/jabfm.2015.03.150097. [PMC free article] [PubMed]
11. Pantaleoni JL, Stevens LA, Mailes ES, Goad BA, Longhurst CA. Successful physician training program for large scale EMR implementation. Appl Clin Inform. 2015 Feb 11;6(1):80–95. DOI: https://doi.org/10.4338/aci-2014-09-cr-0076. [PMC free article] [PubMed]
12. Nuovo J, Hutchinson D, Balsbaugh T, Keenan C. Establishing electronic health record competency testing for first-year residents. J Grad Med Educ. 2013 Dec;5(4):658–61. DOI: https://doi.org/10.4300/jgme-d-13-00013.1. [PMC free article] [PubMed]
13. Tierney MJ, Pageler NM, Kahana M, Pantaleoni JL, Longhurst CA. Medical education in the electronic medical record (EMR) era: Benefits, challenges, and future directions. Acad Med. 2013 Jun;88(6):748–52. DOI: https://doi.org/10.1097/acm.0b013e3182905ceb. [PubMed]
14. Hersh WR, Gorman PN, Biagioli FE, Mohan V, Gold JA, Mejicano GC. Beyond information retrieval and electronic health record use: Competencies in clinical informatics for medical education. Adv Med Educ Pract. 2014 Jul 1;5:205–12. DOI: https://doi.org/10.2147/amep.s63903. [PMC free article] [PubMed]
15. Bowen JL, Hirsh D, Aagaard E, et al. Advancing educational continuity in primary care residencies: An opportunity for patient-centered medical homes. Acad Med. 2015 May;90(5):587–93. DOI: https://doi.org/10.1097/acm.0000000000000589. [PubMed]
16. Hirsh DA, Holmboe ES, ten Cate O. Time to trust: Longitudinal integrated clerkships and entrustable professional activities. Acad Med. 2014 Feb;89(2):201–4. DOI: https://doi.org/10.1097/acm.0000000000000111. [PubMed]
17. Pisano GP, Boomer RM, Edmonson AC. Organizational differences in rates of learning: Evidence from the adoption of minimally invasive cardiac surgery. Manage Sci. 2001 Jun;47(6):752–68. DOI: https://doi.org/10.1287/mnsc.47.6.752.9811.
18. Lateef F. Simulation-based learning: Just like the real thing. J Emerg Trauma Shock. 2010 Oct;3(4):348–52. DOI: https://doi.org/10.4103/0974-2700.70743. [PMC free article] [PubMed]
19. Schmidt E, Goldhaber-Fiebert SN, Ho LA, McDonald KM. Simulation exercises as a patient safety strategy: A systematic review. Ann Intern Med. 2013 Mar 5;158(5 Pt 2):426–32. DOI: https://doi.org/10.7326/0003-4819-158-5-201303051-00010. Erratum in: Ann Intern Med 2013 Jul 16;159(2):160. DOI: https://doi.org/10.7326/0003-4819-159-2-201307160-00023. [PubMed]
20. Reid RJ, Coleman K, Johnson EA, et al. The Group Health medical home at year two: Cost savings, higher patient satisfaction, and less burnout for providers. Health Aff (Millwood) 2010 May;29(5):835–43. DOI: https://doi.org/10.1377/hlthaff.2010.0158. [PubMed]
21. Hsu C, Coleman K, Ross TR, et al. Spreading a patient-centered medical home redesign: A case study. J Ambul Care Manage. 2012 Apr-Jun;35(2):99–108. DOI: https://doi.org/10.1097/jac.0b013e318249e066. [PubMed]
22. McCarthy D, Mueller K, Tillmann I. Group Health Cooperative: Reinventing primary care by connecting patients with a medical home. New York, NY: The Commonwealth Fund; 2009. Jul, p. 2.
23. Charles D, Gabriel M, Furukawa MF. ONC Data Brief, no. 16 [Internet] Washington, DC: The Office of the National Coordinator for Health Information Technology; 2014. May, Adoption of electronic health record systems among U.S. non-federal acute care hospitals: 2008–2013. [cited 2017 Feb 17]. Available from: www.healthit.gov/sites/default/files/oncdatabrief16.pdf.
24. Kim JG, Morris CG, Heidrich FE. A tool to assess family medicine residents’ patient encounters using secure messaging. J Grad Med Educ. 2015 Dec;7(4):649–53. DOI: https://doi.org/10.4300/jgme-d-14-00558.1. [PMC free article] [PubMed]
25. Kim JG, Morris CG, Ford P. Teaching today in the practice setting of the future: Implementing innovations in graduate medical education. Acad Med. 2017 May;92(5):662–5. DOI: https://doi.org/10.1097/ACM.0000000000001510. [PubMed]
26. Hardin JW. Variance inflation factors and variance-decomposition proportions. Stata Technical Bulletin. 1995;4(24):17–22.
27. Mamykina L, Vawdrey DK, Hripcsak G. How do residents spend their shift time? A time and motion study with a particular focus on the use of computers. Acad Med. 2016 Jun;91(6):827–32. DOI: https://doi.org/10.1097/acm.0000000000001148. [PMC free article] [PubMed]
28. Goveia J, Van Stiphout F, Cheung Z, et al. Educational interventions to improve the meaningful use of electronic health records: A review of the literature: BEME guide no. 29. Med Teach. 2013 Nov;35(11):e1551–60. DOI: https://doi.org/10.3109/0142159x.2013.806984. [PubMed]
29. Rouf E, Whittle J, Lu N, Schwartz MD. Computers in the exam room: Differences in physician-patient interaction may be due to physician experience. J Gen Intern Med. 2007 Jan;22(1):43–8. DOI: https://doi.org/10.1007/s11606-007-0112-9. [PMC free article] [PubMed]
30. Schenarts PJ, Schenarts KD. Educational impact of the electronic medical record. J Surg Educ. 2012 Jan-Feb;69(1):105–12. DOI: https://doi.org/10.1016/j.jsurg.2011.10.008. [PubMed]
31. Pusic MV, Boutis K, Hatala R, Cook DA. Learning curves in health professions education. Acad Med. 2015 Aug;90(8):1034–42. DOI: https://doi.org/10.1097/acm.0000000000000681. [PubMed]
32. Sargeant J, Armson H, Chesluk B, et al. The processes and dimensions of informed self-assessment: A conceptual model. Acad Med. 2010 Jul;85(7):1212–20. DOI: https://doi.org/10.1097/acm.0b013e3181d85a4e. [PubMed]

Articles from The Permanente Journal are provided here courtesy of Kaiser Permanente