PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jgimedspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
 
J Gen Intern Med. May 2006; 21(5): 506–509.
PMCID: PMC1484802
The PRIME Curriculum
Clinical Research Training During Residency
RJ Kohlwes, MD, MPH,1,2 RL Shunk, MD,1,2 A Avins, MD,1,2 J Garber, MD,1,2 S Bent, MD,1,2 and MG Shlipak, MD, MPH1,2
1General Internal Medicine Section, Veterans Affairs Medical Center, San Francisco, CA, USA
2Department of Medicine, University of California, San Francisco, CA, USA
The authors have no conflict of interest to declare
Address correspondence and requests for reprints to Dr. Kohlwes: VAMC Box 111, 4150 Clement St., San Francisco, CA 94121 Email:(e-mail: jeffk/at/medicine.ucsf.edu.)
AIM
The Primary Medical Education (PRIME) program is an outpatient-based, internal medicine residency track nested within the University of California, San Francisco (UCSF) categorical medicine program. Primary Medical Education is based at the San Francisco Veteran's Affairs Medical Center (VAMC), 1 of 3 teaching hospitals at UCSF. The program accepts 8 UCSF medicine residents annually, who differentiate into PRIME after internship. In 2000, we implemented a novel research methods curriculum with the dual purposes of teaching basic epidemiology skills and providing mentored opportunities for clinical research projects during residency.
SETTING
Single academic internal medicine program.
PROGRAM DESCRIPTION
The PRIME curriculum utilizes didactic lecture, frequent journal clubs, work-in-progress sessions, and active mentoring to enable residents to “try out” a clinical research project during residency.
PROGRAM EVALUATION
Among 32 residents in 4 years, 22 residents have produced 20 papers in peer-reviewed journals, 1 paper under review, and 2 book chapters. Their clinical evaluations are equivalent to other UCSF medicine residents.
DISCUSSION
While learning skills in evidence-based medicine, residents can conduct high-quality research. Utilizing a collaboration of General Internal Medicine researchers and educators, our curriculum affords residents the opportunity to “try-out” clinical research as a potential future career choice.
Keywords: graduate medical education, evidence based medicine, clinicl research training
The Accreditation Council on Graduate Medical Education (ACGME) includes evidence-based medicine, and practice-based learning, among the core competencies.1 The goal of this requirement is to enable residents to read and evaluate the medical literature, so that they can best use the available evidence to make appropriate clinical decisions. There is a wealth of literature that instructs physicians how to analyze clinical evidence,2 and most residencies have incorporated a method for training their residents to evaluate the medical literature. These are excellent resources, but do not allow the learners to experience the pitfalls that occur during research or to understand the decision making that underlies the execution of any clinical research project. Nor do most residents gain experience at attempting a mentored project in clinical research as a vehicle for informing their own future career decisions.
The Primary Medical Education (PRIME) program is an internal medicine track that is nested within the larger University of California, San Francisco (UCSF) categorical medicine program, based at the San Francisco Veteran Affairs Medical Center (VAMC). The VAMC is 1 of 3 affiliated teaching hospitals at UCSF. This track enables UCSF interns to select a 2-year program that focuses on outpatient clinical training, behavioral medicine and clinical research skills. The program accepts 8 residents per year who divide their time evenly between the standard categorical inpatient rotations and the PRIME outpatient blocks.
In 2000, we implemented a new program in evidence-based medicine (EBM) and clinical research training for the PRIME program. The goal of this curriculum is to teach residents the fundamentals of clinical evidence by giving them the skills in clinical research methods, mentoring, and protected time to complete a clinical research project during residency. Our goal is to give the residents a chance to “try out” a career in clinical outcomes research during residency by completing an academic “PRIME project,” while ensuring that they gain necessary clinical skills. This curriculum supplements the preexisting evening journal clubs, EBM lunchtime lectures, and outpatient seminars that are available to all UCSF medicine residents.
“Primary Medical Education projects” can be any academic project that the resident is motivated to complete. Thus far, they have ranged from educational interventions (e.g., an examination of residents' screening habits for domestic violence) to secondary data analyses in outcomes research (e.g., evaluation of kidney disease as a predictor of revascularization outcomes),3 to meta-analysis (e.g. examination of the test-characteristics of α-feto protein levels in screening for hepatocellular carcinoma).4 Residents pursue topics that they feel passionate about and that can be utilized to become experts in their field. Building this expertise also enables them to decide whether the “nuts and bolts” of executing a clinical research project are exciting enough to encompass their future career.
Primary Medical Education is evaluated using several metrics; the number of resident publications, the number of residents entering careers in clinical outcome research, the number of our residents elected to be chief residents, the percentage of residents who pass the internal medicine board exam and the clinical and teaching evaluations of our residents during residency.
Our purpose in this paper is to show that inclusion of a rigorous clinical research curriculum during residency can enhance clinical training by helping residents decide whether clinical research should be part of their future careers. Furthermore, we have found that resident training in clinical research has helped unify our General Internal Medicine (GIM) section by bringing clinician-educators and clinician investigators together as partners for resident training and career development.
Primary Medical Education residents rotate every 3 months throughout the year in 2 teams. For 6 months each year, PRIME residents perform the typical inpatient duties of an internal medicine house officer. While on outpatient block, the typical weekly PRIME schedule is a mix of 3 half days of primary care clinic, 3 to 4 half days of subspecialty elective clinics, 2 half days of didactic time, and 1–2 half days of research time. PRIME residents become well known to the physician staff at the San Francisco VA Medical Center and are heavily relied upon for staffing (with appropriate attending supervision) of both our primary and subspecialty clinics. Our residents see a large volume of patients seeking primary and specialty care, and serve an important role in access to care for our veterans. The clinical relationships built with generalist and specialist attendings have frequently led to research projects and career mentoring.
Didactic time is divided evenly between clinical outpatient topics, behavioral medicine, and epidemiology/research methods training. Our clinical didactics cover the ACGME recommended core topics that are not covered in other learning opportunities provided by the residency. The behavioral medicine program, run by Drs. Calvin Chou and Kewchang Lee, focuses on important skills for improving patient care and utilizes videotaped and interactive group feedback sessions.
The epidemiology/research methods section of the PRIME didactic curriculum has 4 components that we repeat in different forums to reinforce the key concepts of EBM. The curriculum is a synthesis of the Advanced Training in Clinical Research (ATCR) research training program available to clinical fellows at UCSF, with curriculum used by the Robert Wood Johnson Clinical Scholars Program Fellowship at the University of Washington. The first component is the epidemiology didactic lecture series. This 12 seminar series is provided in consecutive weekly 90-minute sessions. Concepts are presented practically, augmented with assigned reading from the chosen textbook5 and reinforced with short homework assignments. The curriculum sequentially introduces key concepts in a practical manner building from the basics to progressively more difficult concepts as listed in Table 1.
Table 1
Table 1
The PRIME Epidemiology Curriculum
This lecture series is repeated twice yearly, every year, to ensure all the residents have the opportunity to participate. The seminar leaders are members of our clinical investigator faculty. This interaction early in the year fosters future collaborations between residents and faculty. In later blocks other research topics are presented to the residents depending upon their needs. Topics that have been covered in past years include: power calculations, qualitative research methods, survey research, decision analysis and treatment thresholds, cost-effectiveness analysis, use of standardized patients in research, community-based research and process evaluation, international research, administrative data utilization, and deciphering a spreadsheet.
The second component of our EBM/clinical research program is weekly afternoon small group journal clubs. This curriculum reinforces the concepts from the didactic seminar series by discussing the full range of common clinical research study types. During the first half of the year, diagnostic test evaluation, case-control studies, cohort studies, randomized control trials, meta-analysis, decision analysis, cost-effectiveness analysis, practice guidelines, and clinical overviews are covered. The second half of the year, we move to weekly “classic” journal clubs where residents choose articles that they frequently hear quoted on the medical wards. After reading the article the residents evaluate the quality of the “classic” to understand if and how they should apply the study in daily practice.
This journal club series has assigned background reading on the various epidemiology study types. Each week, a resident leads the seminar and prepares a 15-minute presentation that highlights the key learning points for each study type. The discussion is led on their chosen article to reinforce the key points that have been presented. We use a journal club format with a list of questions to be addressed as described in Table 2. We find this format useful as it is simple to remember, easy to apply, and applicable to each study type we review. The journal clubs are precepted by a program director to help with the flow of the session, and to answer questions that arise.
Table 2
Table 2
Simplified Criteria for Journal Article Reviews
The third component involves presentation of the “PRIME projects.” Residents present their projects quarterly in 90-minute work-in-progress sessions that are attended by the residents on outpatient block, the 2 program directors and a rotating member of the GIM clinician-investigator faculty. These sessions are similar to those utilized by research divisions across the academic community. They are collegial, interactive seminars where group effort is used to improve the individual's project ideas.
The final component of the PRIME curriculum is mentorship. The section of General Internal Medicine at the San Francisco VA Medical Center is vital to the success of the program, as are many specialist faculties at the San Francisco VA Medical Center. The clinician investigators and the educators of the division have rallied together to the support of the program. Each part of the division takes pride in the accomplishments of the PRIME program, as does the leadership of the medical center. Participation is almost universal at evening journal clubs, seminar leadership, career guidance, clinical teaching, or project mentoring sessions. The work is spread evenly throughout the section so that each individual participates willingly to a manageable extent.
Most faculty involved in mentoring PRIME residents give several lectures each year (2–4 1-hour lectures), participate in work-in-progress sessions (2 to 4 1-hour sessions), and actively supervise 1–2 PRIME residents on research projects, which typically involves 1–2 1-hour meetings per month in addition to reviewing and editing drafts of proposals and manuscripts. The time expenditure for faculty members varies from week to week, but results in approximately 1 to 2 hours of dedicated time per week. Because faculty members are matched with residents sharing an interest in their area of expertise, the PRIME projects are often complementary to their own work and fulfill part of their academic mission to provide teaching to students at the university. Because the PRIME residents have the explicit expectation that they will publish their research findings, faculty members are more willing to invest the time as mentors.
Residents are steered by the program directors to an appropriate mentoring relationship at the VA, or 1 of the other UCSF affiliated hospitals. Residents interact frequently with the program directors either in the group didactic sessions or during office hours. The primary duty of the program directors is to oversee the direction of the resident to find a project that meets his/her needs. We believe that this model is replicable within many academic GIM sections across the country.
Since 2000, a growing number of residents have applied to the PRIME program because of the popularity of its curriculum. The number of residents applying for the program grew from 7 to 8 per year to 15 to 18 per year over the last 4 years. Residents were selected for PRIME using the criteria in Table 3. Now the program has expanded to accept all 16 applicants this year.
Table 3
Table 3
Criteria for PRIME Program Selection
We had 8 residents per year graduate from the program from 2001 to 2004, for a total of 32 alumni exposed to our curriculum. All PRIME residents passed the Internal Medicine Board Exam on their first attempt which is true of virtually all UCSF medicine residents.
The overall clinical competence scores as rated by 613 students, peers and attending physicians evaluating the 32 PRIME residents gave an overall average score of 8.23 on a 9.0 point scale. This was significantly better than the average of 8.09 from 2,294 responses for the rest of the internal medicine program (P<.001). Seven of the 32 PRIME residents were asked to be Chief Residents (21.8%) compared with 17 of 185 from the other UCSF internal medicine programs (9.2%) (P = .03). PRIME has become the most popular resident rated outpatient rotation in internal medicine at UCSF over that period of time with a score of 4.57 (0 to 5 scale) for PRIME compared with 4.29 for the categorical outpatient experience although this difference does not reach statistical significance (P = .17).
Amongst the 32 PRIME residents in 4 years, 22 have produced 20 original research papers in peer-reviewed journals that are published or in-press, 2 clinical-review papers, 1 paper currently under review and 2 book chapters.626 Most of these published papers were presented at a local or national research meeting. Five other residents presented their projects at our annual Floyd Rector Resident Research Symposium. In large part because of presentations from the PRIME residents, the average number of resident publications at the Rector Symposium rose from 6 to 9 presentations per year in the 1990s to 13 to 21 presentations since 2001. Importantly, this production represents creative ideas and hard work. Residents are nearly always the lead authors of these original research articles, which is a position they earn by leading every aspect from conception and design to manuscript production under the guidance of their mentor.
The VA PRIME Program at UCSF utilizes a novel curriculum for giving academically motivated internal medicine residents an experience in clinical research. PRIME residents had at least equivalent clinical competency and satisfaction scores when compared with other UCSF categorical medicine residents. The significance of this is 2-fold. First, it demonstrates that PRIME residents' clinical skills did not suffer while participating in our clinical research curriculum. Second, it shows that residents are motivated by the opportunity of performing mentored clinical research projects and take personal satisfaction from accomplishing this task during residency.
Our descriptive study has an important limitation. We lacked a comparison group for research productivity for our residents as they are clearly “self-selected” to produce projects during residency. Second, we cannot determine the extent to which the curriculum contributed to the resident's research productivity beyond the talent of the individual mentee and mentor. We believe, however, that the residents' success depends heavily on the structured curriculum, willing mentors and their own enthusiasm.
We believe that creating opportunities for resident research during residency fosters critical evaluation skills, provides insights into the benefits and limitations of practicing evidence-based clinical medicine and empowers residents to shape their academic careers from a position of insight into the joys and sorrows of clinical research. We believe that similar programs can be adopted at many academic teaching programs by combining the skills and energy of dedicated clinician educators and clinician investigators into a productive partnership.
Because of the popularity of PRIME, we have decided to expand access to our curriculum throughout the UCSF Categorical Residency program. Beginning in the current academic year, we accepted all applicants from the internship class to provide them access to the clinical experience, didactics, teaching and mentoring at the San Francisco VA Medical Center, that have made PRIME successful.
Acknowledgments
Calvin Chou, MD, PhD, Kewchang Lee, MD, Patricia Cornett, MD, Sharad Jain, MD, Mary Whooley, MD, Tracy Minichiello, MD, Robert Baron, MD, MPH, Elizabeth Harleman, MD, Thomas Koepsell, MD, MPH, and Richard Deyo, MD, MPH. We also appreciate the input and leadership of Dr. Harry Hollander, the Internal Medicine Residency Director at the University of California, San Francisco.
2. Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How to get started. The evidence-based medicine working group. JAMA. 1993;270:2093–5. [PubMed]
3. Ix JH, Shlipak MG, Liu HH, Schiller NB, Whooley MA. Association between renal insufficiency and inducible ischemia in patients with coronary artery disease: the heart and soul study. J Am Soc Nephrol. 2003;14:3233–8. [PMC free article] [PubMed]
4. Gupta S, Bent S, Kohlwes J. Test characteristics of alpha-fetoprotein for detecting hepatocellular carcinoma in patients with hepatitis C. A systematic review and critical analysis. Ann Intern Med. 2003;139:46–50. [PubMed]
5. Hulley SB, Cummings SR. Baltimore: Williams & Wilkins; 1998. Designing Clinical Research: An Epidemiologic Approach.
6. Behler C, Shade S, Gregory K, Abrams D, Volberding P. Anemia and HIV in the antiretroviral era; potential significance of testosterone. AIDS Res Hum Retroviruses. 2005. pp. 200–6. [PubMed]
7. Beinart G, Damon L. Thrombosis associated with l-asparaginase therapy and low fibrinogen levels in adult acute lymphoblastic leukemia. Am J Hematol. 2004;77:331–5. [PubMed]
8. Liu R, Nair D, Ix J, Moore DH, Bent S. N-acetylcysteine for the prevention of contrast-induced nephropathy. A systematic review and meta-analysis. J Gen Intern Med. 2005;20:193–200. [PMC free article] [PubMed]
9. Nair D, Shlipak MG, Angeja B, Liu HH, Schiller NB, Whooley MA. Association of anemia with diastolic dysfunction among patients with coronary artery disease in the Heart and Soul Study. Am J Cardiol. 2005;95:332–6. [PMC free article] [PubMed]
10. Young LS, Perdreau-Remington F, Winston LG. Clinical, epidemiologic, and molecular evaluation of a clonal outbreak of methicillin-resistant Staphylococcus aureus infection. Clin Infect Dis. 2004;38:1075–83. Epub April 5, 2004. [PubMed]
11. Ix JH, McCulloch CE, Chertow GM. Theophylline for the prevention of radiocontrast nephropathy; a meta-analysis. Nephrol Dial Transplant. 19:2747–53. Epub August 24, 2004. [PubMed]
12. Gunn N, Damon L, Varosy P, et al. High CD34+ cell dose promotes faster platelet recovery after autologous stem cell transplantation for acute myeloid leukemia. Biol Blood Marrow Transplant. 2003;9:643–8. [PubMed]
13. Jun S, Stollman N. Epidemiology of diverticular disease. Best Pract Res Clin Gastroenterol. 2002;16:529–42. [PubMed]
14. Pham MX, Whooley MA, Evans GT, Jr, et al. Prognostic value of low-level cardiac troponin-I elevations in patients without definite acute coronary syndromes. Am Heart J. 2004;148:776–82. [PubMed]
15. Haberer J, Trivedi NN, Kohlwes J, Tierney L., Jr Clinical problem-solving.a gut feeling. N Engl J Med. 2003;349:73–8. [PubMed]
16. Haberer JE, Ix JH, Tierney LM., Jr Clinical problem-solving. One foot away. N Engl J Med. 2002;346:1394–7. [PubMed]
17. Terasawa T, Blackmore CC, Bent S, Kohlwes RJ. Systematic review: computed tomography and ultrasonography to detect acute appendicitis in adults and adolescents. Ann Intern Med. 2004;141:537–46. [PubMed]
18. Phipps W, Stanley H, Kohn R, Klausner JD. Syphilis, chlamydia and gonorrhea screening in HIV infected patients in primary care, San Francisco, 2003. AIDS Patient Care STDS. 2005;19:495–8. [PubMed]
19. Phipps W, Kent C, Kohn R, Klausner JD. Risk factors for repeat syphilis infection, San Francisco, 2001–2003, AIDS Patient Care STDS, in press.
20. Hwang J, Bitarakwate E, Pai M, Reingold A, Rosenthal PJ, Dorsey J. Chloroquine or amodiaquine combined with sulfadoxine-pyrimethamine for uncomplicated malaria: a systematic review. Tropical Medicine and International Health. in press. [PubMed]
21. Ghaly C, Peabody J. Incentives for increases in research and development for neglected diseases in the developing world. in press.
22. Chang L, Vidyarthi A, Kohlwes J. Baseline duty hours recorded with timecards: a pre-regulation study of internal medicine residents. Med Edu. in press. [PubMed]
23. McManus D, Pipkin SS, Whooley MA. Screening for depression in patients with coronary heart disease: data from the heart and soul study. Am J Cardiol. 2005;96:1076–81. Epub August 30, 2005. [PMC free article] [PubMed]
24. Chang LW, Phipps WT, Kennedy GE, Rutherford GW. Antifungal interventions for the primary prevention of cryptococcal disease in adults with HIV. The Cochrane Database of Systematic Reviews 2005, Issue 3. Art. No.: CD004773.pub2. doi: 10.1002/14651858.CD004773.pub2. [PubMed]
25. Clark BC, Grossman E, White MC, Goldenson J, Tulsky JP. The state of diabetes care in the San Francisco jail. Am J Public Health. in press. [PubMed]
26. Young LS, Winston LG. Preoperative mupirocin for the prevention of healthcare-associated Staphylococcus aureusinfections: A cost effectiveness analysis. Infect Cont Hosp Epidemiol. in press. [PubMed]
Articles from Journal of General Internal Medicine are provided here courtesy of
Society of General Internal Medicine