PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of canfamphysLink to Publisher's site
 
Can Fam Physician. 2011 October; 57(10): e381–e389.
PMCID: PMC3192104

Implementation of electronic medical records

Effect on the provision of preventive services in a pay-for-performance environment

Mise en œuvre des dossiers médicaux électroniques

Michelle Greiver, MD MSc CCFP FCFP
Assistant Professor in the Department of Family and Community Medicine at the University of Toronto in Ontario
Jan Barnsley, MES PhD
Associate Professor in the Department of Health Policy, Management and Evaluation at the University of Toronto and Adjunct Scientist at the Institute for Clinical Evaluative Sciences in Toronto
Richard H. Glazier, MD MPH CCFP
Senior Scientist at the Institute for Clinical Evaluative Sciences, Scientist at the Centre for Research on Inner City Health at St Michael’s Hospital in Toronto, and Professor in the Department of Family and Community Medicine at the University of Toronto
Rahim Moineddin, MSc PhD
Associate Professor in the Department of Family and Community Medicine at the University of Toronto, Scientist at the Institute for Clinical Evaluative Sciences, and Associate Professor in the Dalla Lana School of Public Health at the University of Toronto
Bart J. Harvey, MD PhD MEd FRCPC

Abstract

Objective

To study the effect of electronic medical record (EMR) implementation on preventive services covered by Ontario’s pay-for-performance program.

Design

Prospective double-cohort study.

Participants

Twenty-seven community-based family physicians.

Setting

Toronto, Ont.

Intervention

Eighteen physicians implemented EMRs, while 9 physicians continued to use paper records.

Main outcome measure

Provision of 4 preventive services affected by pay-for-performance incentives (Papanicolaou tests, screening mammograms, fecal occult blood testing, and influenza vaccinations) in the first 2 years of EMR implementation.

Results

After adjustment, combined preventive services for the EMR group increased by 0.7%, a smaller increase than that seen in the non-EMR group (P = .55, 95% confidence interval −2.8 to 3.9).

Conclusion

When compared with paper records, EMR implementation had no significant effect on the provision of the 4 preventive services studied.

Résumé

Objectif

Vérifier l’effet de la mise en œuvre des dossiers médicaux électroniques (DMÉ) sur les services de prévention couverts par le programme ontarien de rémunération au rendement.

Type d’étude

Étude prospective à double cohorte.

Participants

Vingt-sept médecins de famille pratiquant en milieu communautaire.

Contexte

Toronto, Ontario.

Interventions

Dix-huit médecins avaient adopté les DMÉ alors que les 9 autres continuaient d’utiliser les dossiers papier.

Principaux paramètres à l’étude

Effet du programme incitatif de rémunération au rendement sur la prestation de 4 services préventifs (tests de Papanicolaou, dépistage par mammographies, recherche du sang occulte dans les selles et vaccination contre la grippe) au cours des 2 ans suivant l’introduction des DMÉ.

Résultats

Après ajustement, l’ensemble des services préventifs pour le groupe DMÉ a augmenté de 0,7 %, une augmentation inférieure à celle observée pour le groupe non DMÉ (P = 0,55, intervalle de confiance à 95 % −2,8 à 3,9).

Conclusion

Par rapport aux dossiers papiers, la mise en œuvre des DMÉ n’a pas eu d’effet significatif sur la prestation des 4 services préventifs étudiés.

Electronic medical records (EMRs) have been identified as critical to quality improvement efforts,14 and policies favouring the establishment of EMRs are being implemented in the United States and in most Canadian provinces.2,5,6 In 2005, the Ontario government offered a subsidy to selected primary care physicians for the purchase of EMRs.5 Physicians who were eligible were those who had chosen a blended capitation payment system based largely on the age and sex of patients enrolled in their practices. By the end of the program in 2009, most physicians offered the subsidy had purchased EMRs.7 This provided an opportunity to compare groups of physicians in the same communities implementing and not implementing EMRs.

Although EMRs invite hope for improvement,8 there is still much that is unknown about the effect of EMRs on quality and performance. A recent systematic review found that computerized decision-support systems improved practitioner performance, especially with regard to immunizations.9 However, these systems were not commercial, off-the-shelf EMRs like those commonly found in primary care practices. Several studies have found that EMRs might not improve care1015 and might even be a source of errors.1517 Studies of EMR implementation have often been descriptive, with very few evaluating measurable outcomes.16 At present, the effect of EMR implementation on the quality of care in small community-based family practices is unknown.

Electronic medical record systems are complex, and their implementation involves changes to many processes; only some changes can be implemented early on. In this study, we focused on preventive services targeted by Ontario’s pay-for-performance program as markers of early progress in EMR implementation. These services were attached to financial incentives and were perceived as representing good care.17 The pay-for-performance target numbers varied by physician and were based on the percentage of enrolled family practice patients being provided with Papanicolaou smears, mammograms, influenza vaccinations, fecal occult blood screening, and primary vaccinations (in children younger than 2 years).18,19

In order to be eligible for funding in Ontario, an EMR system had to include the ability to manage the provision of such preventive services through point-of-care alerts and reminder letters. Although physicians using paper records were also offered financial incentives and were expected to produce an increase in preventive services, those using EMRs had electronic tools that could lead to a greater increase.

Our objective was to compare the differences in provision of these preventive services between community-based family physicians implementing EMRs and those continuing to use paper-based records.

METHODS

Participants

We followed 2 cohorts of physicians: a group of 18 physicians implementing EMRs and a group of 9 physicians using paper records (the non-EMR cohort). Physicians in both cohorts were community-based, were affiliated with a local hospital, and were located in the Toronto area. All were members of the same local after-hours clinic. Physicians were signed out to the clinic and took turns providing walk-in care at a single location on evenings and weekends. No preventive services were offered at the clinic and no clinic data were included in the study.

The physicians in the EMR cohort had previously participated in a pay-for-performance study,20 so data on their characteristics were already available. They switched to the blended capitation model at the end of 2004 and began EMR implementation in the first 4 months of 2006 using the same EMR software. We studied the change in preventive services in the first 2 years of EMR implementation (2006 and 2007). The principal investigator was also a participant in this study (M.G.).

We recruited the non-EMR physicians through a letter of invitation and brief survey sent to all members of the after-hours clinic who were not part of the EMR cohort (125 physicians). These physicians were not eligible for the EMR subsidy. The recruitment process is shown in Figure 1.

Figure 1.
Recruitment process for the non-EMR cohort

Data sources

The Ontario Ministry of Health and Long-Term Care provides family physicians with a list of patients in their practices who are eligible for each of the 4 preventive care services. We selected charts from these lists, using a random numbers table, from which we recorded the following: each patient’s age and sex, the presence or absence of a service within the required time period, and any record that a reminder letter had been sent to patients overdue for a service. We entered the data into an Epi Info database.21

We examined information on factors possibly associated with the provision of preventive services.2232 Data were obtained through a questionnaire administered to all physicians. Following the methods of Glazier et al,33 practice-level data were also derived from linked administrative databases at the Institute for Clinical Evaluative Sciences (ICES). Income quintiles were derived from 2001 census data linked to patient postal codes. Neighbourhood income quintiles and recent immigration status were derived using published methods.33,34 Morbidity and comorbidity (adjusted diagnosis groups and resource utilization bands) were derived using the Johns Hopkins Adjusted Clinical Groups software,35 available from ICES, which has been described elsewhere.3643 Numbers and percentages of patients with chronic diseases were obtained from linked ICES health databases.3844

We audited paper charts for physicians in the non-EMR cohort and obtained data from EMR charts for physicians in the EMR cohort. When data were unavailable in the EMR, we retrieved and obtained data from the paper chart.

Five data auditors abstracted data from both paper charts and EMRs. The research coordinator initially audited 10 charts for each service in 2 practices and reviewed this with the principal investigator (M.G.). The coordinator then trained each data auditor, and reviewed at least 10 charts for each service from each auditor. The data were collected on paper forms and entered in the Epi Info database by 2 data entry clerks. Each clerk entered data from a training sample of at least 10 charts for each service. A randomly selected 10% data sample for each service, each year, and each physician was re-audited and entered in the database; we used the κ statistic to compare the 2 audits.

Outcome measures

The study’s end point was whether or not a preventive service was provided and documented within a required time period for each eligible patient. The target patient population consisted of all eligible patients enrolled in the participating practices. Documentation that the patient received the service through another health care provider was acceptable. Information on services is presented in Table 1.

Table 1.
Eligibility criteria, exclusion criteria, and required period for preventive service provision in a pay-for-performance family practice setting

We calculated a composite process score44 by using the total number of charts audited for eligible patients for each service per physician as the denominator and the total number of services documented in the audits as the numerator. We could not obtain lists of eligible children for 2006 in the non-EMR cohort. As a result, children’s vaccinations were excluded from the composite score.

We compared the change in the proportion of patients who received preventive services between the EMR and non-EMR cohorts during EMR implementation. A 5% increase was considered the minimum clinically important difference.

Sample size calculation

We calculated the sample size required to have an 80% power to detect a clinically important increase in service provision of 5% or higher20 in the first 2 years of EMR implementation (assuming relatively stable rates in the non-EMR cohort and with rates of influenza vaccination ranging from 83% to 88%) using an α level of .05, and determined that 40 charts per service per provider would be required. To further increase statistical power, we audited 50 charts per year per service per physician. Physicians who practise in groups might influence one another; this would reduce the variation in practice and might inflate the observed effect.45 Based on our previous pay-for-performance study,20 the estimated intraclass correlation coefficient46 due to clustering of physicians within practices was 0.01. If we assume that, on average, the recruited physicians were clustered within groups of 4 physicians, and that the intraclass correlation coefficient was 0.01, the inflation factor would be 1.03 ([1 + (4 - 1) × 0.01]), which should have a negligible effect on the sample size required.

Analysis

We compared unadjusted percentages using χ2 tests and compared adjusted percentages using multivariate logistic regression analysis. We used generalized estimating equations to adjust for the clustering structure of the data in regression models. When comparing the 2 groups, we adjusted for patient age for each individual patient,47 physician sex,20 years since each physician’s medical school graduation,22 and Certification in Family Medicine (ie, CCFP) status26 using multivariate logistic regression analysis. Administrative data could not be used for adjustment, as these data were not available to us at the patient level. Percentage of fecal occult blood tests provided might have been different from the other services provided; the provincial provision rate was low and there was a concurrent public health campaign in 2007.48,49 As a result, we re-analyzed the 2 cohorts with this screen excluded.

As the principal investigator was also a participant in the study, a sensitivity analysis was performed by excluding her practice data.

We performed analyses with SAS software, version 9.1. All tests were 2-sided using an α level of .05. The study was approved by the University of Toronto’s Research Ethics Board; the Sunnybrook Research Institute’s Research Ethics Board approved the use of ICES data. All participating physicians provided written informed consent.

RESULTS

Characteristics of the EMR and non-EMR physicians and their practice populations are presented in Table 2 and Table 3,22, 3443 respectively. Physician characteristics between the 2 cohorts were similar. Patients in the EMR cohort were slightly younger and had lower levels of morbidity and comorbidity.

Table 2.
Characteristics of physicians in the EMR and non-EMR cohorts
Table 3.
Comparison of patient characteristics between the EMR and non-EMR cohort practice populations as of August 31, 2007

Results for individual services are presented in Table 4. Nine hundred charts were audited for each service for each year in the EMR cohort and 450 charts were audited in the non-EMR cohort. The number of Pap smears and mammograms increased more in the EMR cohort, while the rate of influenza vaccinations and fecal occult blood testing increased more in the non-EMR cohort. Composite process scores are shown in Table 5.

Table 4.
Service provision for individual services in the EMR and non-EMR cohorts before and after EMR implementation
Table 5.
Changes in composite process scores for the provision of preventive services between EMR and non-EMR cohorts

Female physician sex and younger patient age were positively correlated with the likelihood of receiving a service. There was no correlation with years since medical school graduation or Certification status. There was no significant difference in the change in service provision between the 2 groups; differences and confidence intervals were less than 5%. The results were not affected by the exclusion of fecal occult blood screening data.

Physicians in the EMR cohort mailed letters to 23 patients overdue for services in 2005, 265 patients in 2006, and 677 patients in 2007. Physicians in the non-EMR cohort mailed 1 letter in 2006 and none in 2007.

The results were unaffected by the exclusion of the principal investigator’s data. The κ statistic for the 10% of charts that were re-audited was 0.954, consistent with an acceptable level of agreement.

DISCUSSION

We found no statistically significant or clinically important difference in the change in preventive service provision between physicians implementing EMRs and those continuing to use paper records. Other studies have found no difference in the change in care of diabetes between EMR- and paper-based practices11 and no improvement when additional experience with EMR accrues over time.11,13,15

The EMR system we studied included elements that have been associated with an increase in the provision of preventive services, such as point-of-care alerts9,50,51 and the ability to generate reminder letters to patients.52 However, studies have shown that alerts or patient reminders for overdue services, even if available, might not be consistently implemented in practice.11,15,53,54 We found evidence of reminder letter mailings in the EMR cohort, but this did not appear to have had an effect on service uptake.

To frame and explain our results, a qualitative study presented in an accompanying article55 explores the context of this EMR implementation. Similar studies could help to determine what aspects of EMR implementation are associated with improved quality of care.

Limitations

This study was limited to a group of selected physicians in Toronto. However, all physicians in this study were practising in community-based settings, similar to most family physician settings in Ontario.56 Physicians in our cohorts were similar to their colleagues in capitated and reformed fee–for-service models in Ontario urban centres.33

Nineteen percent of physicians using paper records responded to the study invitation; we do not know if their characteristics differed from those of the nonresponders. The participating physicians provided preventive services to a large proportion of their patients. Increases might have been limited by ceiling effects. Also, we studied a single EMR system; results might differ for physicians using other EMR systems.

This was an observational cohort study, and is therefore subject to both measured and unmeasured confounders. In this study, we measured confounders that have been reported in the literature to affect preventive services2232,47,57,58 and used statistical adjustments to adjust for these factors. There were differences between our 2 cohorts in terms of physician funding mechanisms. However, a recent study using administrative data did not find any consistent difference in the provision of preventive services between Ontario physicians in capitated or reformed fee-for-service groups.59

Conclusion

Among the family physician practices studied, the first 2 years of EMR implementation were not associated with an increase in the provision of preventive services targeted by Ontario’s pay-for-performance program when compared with the continued use of paper records. It should not be assumed that EMR implementation improves care.

Acknowledgments

This study was funded by a research grant from the Ontario Ministry of Health and Long-Term Care Health System Strategy Division (grant no. 020709) and was supported by the Institute for Clinical Evaluative Sciences. The opinions, results, and conclusions reported in this paper are those of the authors and do not imply endorsement by the Ministry of Health and Long-Term Care or the Institute for Clinical Evaluative Sciences.

Notes

EDITOR’S KEY POINTS

  • One of the drivers of electronic medical record (EMR) implementation is the hope that EMRs can improve quality of care for patients.
  • The authors examined preventive services with pay-for-performance incentives that should have improved with EMR use owing to electronic tools that assist in timely service provision (eg, alerts, automated reminders).
  • There was no difference in service provision between physicians using EMRs and those continuing to use paper records (in the first 2 years of EMR implementation).
  • It should not be assumed that EMR implementation improves care.

POINTS DE REPÈRE DU RÉDACTEUR

  • Une des raisons qui incitent à adopter les dossiers médicaux électroniques (DMÉ) est l’espoir qu’ils peuvent améliorer la qualité des soins.
  • Les auteurs ont examiné les services préventifs qui, avec une rémunération au rendement comme mesure incitative, auraient dû s’améliorer avec les DMÉ grâce aux outils électroniques qui facilitent la prestation de ces services en temps opportun (p. ex. signaux d’alerte, rappels automatisés).
  • Il n’y avait pas de différence de prestation des services entre les médecins utilisant les DMÉ et ceux qui continuaient d’utiliser les dossiers papiers (durant les 2 ans suivant l’introduction des DMÉ).
  • On ne devrait pas présumer que la mise en œuvre des DMÉ améliore les soins.

Footnotes

This article has been peer reviewed.

Cet article a fait l’objet d’une révision par des pairs.

Contributors

All authors contributed to the concept and design of the study. Drs Greiver and Barnsley contributed to the data gathering. All authors contributed to the analysis and interpretation of the results and the preparation of the manuscript for publication.

Competing interests

None declared

References

1. Canada Health Infoway Annual report 2007–2008. The evolution of health care: making a difference. Toronto, ON: Canada Health Infoway; 2008. Available from: www2.infoway-inforoute.ca/Documents/Infoway_Annual_Report_2007-2008_Eng.pdf. Accessed 2011 Sep 9.
2. American recovery and reinvestment act of 2009, Pub L No. 111-5, 123 Stat 115-531. Available from: www.gpo.gov/fdsys/pkg/PLAW-111publ5/pdf/PLAW-111publ5.pdf. Accessed 2011 Sep 9.
3. The health of Canadians—the federal role. Final report. Volume 6: recommendations for reform. Ottawa, ON: Standing Senate Committee on Social Affairs, Science and Technology; 2002. Available from: www.parl.gc.ca/Content/SEN/Committee/372/soci/rep/repoct02vol6-e.htm. Accessed 2011 Sep 9.
4. Romanow RJ. Building on values: the future of health care in Canada. Ottawa, ON: Commission of the Future of Health Care in Canada; 2002. Available from: http://dsp-psd.pwgsc.gc.ca/Collection/CP32-85-2002E.pdf. Accessed 2011 Sep 9.
5. Ministry of Health and Long-Term Care Family Health Network agreement. Toronto, ON: Ministry of Health and Long-Term Care; 2005. IT appendix. Physician IT program.
6. CanadianEMR [website] Provincial programs. Vancouver, BC: CanadianEMR; 2010. Available from www.canadianemr.ca/index.aspx?PID=50. Accessed 2011 Sep 9.
7. Toronto, ON: OntarioMD Inc; 2011. EMR Advisor [website] Available from: www.emradvisor.ca. Accessed 2011 Sep 12.
8. Kilo CM. Transforming care: medical practice design and information technology. Health Aff (Millwood) 2005;24(5):1296–301. [PubMed]
9. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–38. [PubMed]
10. Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, et al. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002;325(7370):941. [PMC free article] [PubMed]
11. O’Connor PJ, Crain AL, Rush WA, Sperl-Hillen JM, Gutenkauf JJ, Duncan JE. Impact of an electronic medical record on diabetes quality of care. Ann Fam Med. 2005;3(4):300–6. [PubMed]
12. Linder JA, Ma J, Bates DW, Middleton B, Stafford RS. Electronic health record use and the quality of ambulatory care in the United States. Arch Intern Med. 2007;167(13):1400–5. [PubMed]
13. Zhou L, Soran CS, Jenter CA, Volk LA, Orav EJ, Bates DW, et al. The relationship between electronic health record use and quality of care over time. J Am Med Inform Assoc. 2009;16(4):457–64. Epub 2009 Apr 23. [PMC free article] [PubMed]
14. Gordon JR, Wahls T, Carlos RC, Pipinos II, Rosenthal GE, Cram P. Failure to recognize newly identified aortic dilations in a health care system with an advanced electronic medical record. Ann Intern Med. 2009;151(1):21–7. W5. [PubMed]
15. Crosson JC, Ohman-Strickland PA, Hahn KA, DiCicco-Bloom B, Shaw E, Orzano AJ, et al. Electronic medical records and diabetes quality of care: results from a sample of family medicine practices. Ann Fam Med. 2007;5(3):209–15. [PubMed]
16. Mitchell E, Sullivan F. A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980–97. BMJ. 2001;322(7281):279–82. [PMC free article] [PubMed]
17. Institute of Medicine Crossing the quality chasm. A new health system for the 21st century. Washington, DC: National Academy Press; 2001. Available from: www.nap.edu/html/quality_chasm/reportbrief.pdf. Accessed 2011 Sep 9. [PubMed]
18. Family Health Network, Ministry of Health and Long Term Care General blended payment template. Family Health Network agreement. Toronto, ON: Ministry of Health and Long Term Care; 2002.
19. Family Health Network, Ministry of Health and Long Term Care General blended payment template. Family Health Network agreement. Toronto, ON: Ministry of Health and Long Term Care; 2006.
20. Kasperski J, Greiver M, Barnsley J, Rachlis V. Measuring quality improvements in preventive care services in the first two family health networks in the Greater Toronto Area. Toronto, ON: Ontario College of Family Physicians; 2006. Grant no. G03-05567. Available from: www.ocfp.on.ca/docs/research-projects/2011/04/18/preventive-care-services-in-fhns-project.pdf. Accessed 2011 Sep 14.
21. Atlanta, GA: Centers for Disease Control and Prevention; 2005. Epi Info [computer program]. Version 3.32. Available from: wwwn.cdc.gov/epiinfo. Accessed 2011 Sep 12.
22. Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: the relationship between clinical experience and quality of health care. Ann Intern Med. 2005;142(4):260–73. [PubMed]
23. Franks P, Clancy CM. Physician gender bias in clinical decisionmaking: screening for cancer in primary care. Med Care. 1993;31(3):213–8. [PubMed]
24. Lurie N, Slater J, McGovern P, Ekstrum J, Quam L, Margolis K. Preventive care for women. Does the sex of the physician matter? N Engl J Med. 1993;329(7):478–82. [PubMed]
25. Thind A, Feightner J, Stewart M, Thorpe C, Burt A. Who delivers preventive care as recommended? Analysis of physician and practice characteristics. Can Fam Physician. 2008;54:1574–5.e1-5. Available from: www.cfp.ca/content/54/11/1574.full.pdf+html. Accessed 2011 Sep 15. [PMC free article] [PubMed]
26. Norton PG, Dunn EV, Soberman L. What factors affect quality of care? Using the Peer Assessment Program in Ontario family practices. Can Fam Physician. 1997;43:1739–44. [PMC free article] [PubMed]
27. Zyzanski SJ, Stange KC, Langa D, Flocke SA. Trade-offs in high-volume primary care practice. J Fam Pract. 1998;46(5):397–402. [PubMed]
28. Parkerton PH, Wagner EH, Smith DG, Straley HL. Effect of part-time practice on patient outcomes. J Gen Intern Med. 2003;18(9):717–24. [PMC free article] [PubMed]
29. McIsaac WJ, Fuller-Thomson E, Talbot Y. Does having regular care by a family physician improve preventive care? Can Fam Physician. 2001;47:70–6. [PMC free article] [PubMed]
30. Finkelstein MM. Preventive screening. What factors influence testing? Can Fam Physician. 2002;48:1494–501. [PMC free article] [PubMed]
31. Lofters A, Glazier RH, Agha MM, Creatore MI, Moineddin R. Inadequacy of cervical cancer screening among urban recent immigrants: a population-based study of physician and laboratory claims in Toronto, Canada. Prev Med. 2007;44(6):536–42. Epub 2007 Mar 21. [PubMed]
32. Walter LC, Lindquist K, Nugent S, Schult T, Lee SJ, Casadei MA, et al. Impact of age and comorbidity on colorectal cancer screening among older veterans. Ann Intern Med. 2009;150(7):465–73. [PMC free article] [PubMed]
33. Glazier RH, Klein-Geltink J, Kopp A, Sibley LM. Capitation and enhanced fee-for-service models for primary care reform: a population-based evaluation. CMAJ. 2009;180(11):E72–81. [PMC free article] [PubMed]
34. Shortell SM. Continuity of medical care: conceptualization and measurement. Med Care. 1976;14(5):377–91. [PubMed]
35. The Johns Hopkins University ACG Case-Mix Adjustment System [computer program] Baltimore, MD: Johns Hopkins University School of Hygiene and Public Health; 1997.
36. The Johns Hopkins University ACG System About the ACG system. Baltimore, MD: Johns Hopkins University School of Hygiene and Public Health; 2009. Available from: www.acg.jhsph.org/index.php?option=com_content&view=article&id=46&Itemid=61. Accessed 2011 Sep 15.
37. Hux JE, Ivis F, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3):512–6. [PubMed]
38. Tu K, Campbell NR, Chen ZL, Cauch-Dudek KJ, McAlister FA. Accuracy of administrative databases in identifying patients with hypertension. Open Med. 2007;1(1):e18–26. [PMC free article] [PubMed]
39. Jackevicius CA, Li P, Tu JV. Prevalence, predictors, and outcomes of primary nonadherence after acute myocardial infarction. Circulation. 2008;117(8):1028–36. [PubMed]
40. Ko DT, Alter DA, Austin PC, You JJ, Lee DS, Qiu F, et al. Life expectancy after an index hospitalization for patients with heart failure: a population-based study. Am Heart J. 2008;155(2):324–31. [PubMed]
41. Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying patients with physician-diagnosed asthma in health administrative databases. Can Respir J. 2009;16(6):183–8. [PMC free article] [PubMed]
42. Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying individuals with physician diagnosed COPD in health administrative databases. COPD. 2009;6(5):388–94. [PubMed]
43. Steele LS, Glazier RH, Lin E, Evans M. Using administrative data to measure ambulatory mental health service provision in primary care. Med Care. 2004;42(10):960–5. [PubMed]
44. Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, et al. Public reporting and pay for performance in hospital quality improvement. N Engl J Med. 2007;356(5):486–96. Epub 2007 Jan 26. [PubMed]
45. Donner A, Klar N. Design and analysis of cluster randomization trials in health research. Hoboken, NJ: John Wiley and Sons, Inc; 2010.
46. Ukoumunne OC, Gulliford MC, Chinn S, Sterne JA, Burney PG, Donner A. Methods in health service research. Evaluation of health interventions at area and organisation level. BMJ. 1999;319(7206):376–9. [PMC free article] [PubMed]
47. Asch SM, Kerr EA, Keesey J, Adams JL, Setodji CM, Malik S, et al. Who is at greatest risk for receiving poor-quality health care? N Engl J Med. 2006;354(11):1147–56. [PubMed]
48. Wilkins K, Shields M. Colorectal cancer testing in Canada—2008. Health Rep. 2009;20(3):21–30. [PubMed]
49. ColonCancerCheck [website] Toronto, ON: Ministry of Health and Long Term Care of Ontario; 2007. Available from: www.health.gov.on.ca/en/public/programs/coloncancercheck. Accessed 2011 Sep 12.
50. Balas EA, Weingarten S, Garb CT, Blumenthal D, Boren SA, Brown GD. Improving preventive care by prompting physicians. Arch Intern Med. 2000;160(3):301–8. [PubMed]
51. Craig TJ, Perlin JB, Fleming BB. Self-reported performance improvement strategies of highly successful Veterans Health Administration facilities. Am J Med Qual. 2007;22(6):438–44. [PubMed]
52. Szilagyi PG, Bordley C, Vann JC, Chelminski A, Kraus RM, Margolis PA, et al. Effect of patient reminder/recall interventions on immunization rates: a review. JAMA. 2000;284(14):1820–7. [PubMed]
53. Griffin S, Kinmonth AL. Diabetes care: the effectiveness of systems for routine surveillance for people with diabetes. Cochrane Database Syst Rev. 2000;(2):CD000541. [PubMed]
54. Welch WP, Bazarko D, Ritten K, Burgess Y, Harmon R, Sandy LG. Electronic health records in four community physician practices: impact on quality and cost of care. J Am Med Inform Assoc. 2007;14(3):320–8. Epub 2007 Feb 28. [PMC free article] [PubMed]
55. Greiver M, Barnsley J, Glazier RH, Moineddin R, Harvey BJ. Implementation of electronic medical records. Theory-informed qualitative study. Can Fam Physician. 2011;57:e390–7. [PMC free article] [PubMed]
56. Schultz SE, Guttmann A, Jaakkimainen L. Chapter 11. Characteristics of primary care practice. In: Jaakkimainen LUR, Klein-Geltink JE, Leong A, Maaten S, Schultz SE, Wang L, editors. Primary care in Ontario: ICES atlas. Toronto, ON: Institute for Clinical Evaluative Sciences; 2006. Available from: www.ices.on.ca/file/PC_atlas_chapter11.pdf. Accessed 2011 Sep 12.
57. Snider J, Beauvais J, Levy I, Villeneuve P, Pennock J. Trends in mammography and Pap smear utilization in Canada. Chronic Dis Can. 1996;17(3–4):108–17. [PubMed]
58. Glazier RH, Creatore MI, Gozdyra P, Matheson FI, Steele LS, Boyle E, et al. Geographic methods for understanding and responding to disparities in mammography use in Toronto, Canada. J Gen Intern Med. 2004;19(9):952–61. [PMC free article] [PubMed]
59. Jaakkimainen L, Barnsley J, Klein-Geltink J, Kopp A, Glazier R, Sibley L. Performance measures amongst primary care groups in Ontario. Toronto, ON: Institute for Clinical Evaluative Sciences; 2009. Available from: www.f2fe.com/CAHSPR/2009/docs/A6/a6c%20Liisa%20Jaakkimainen.pdf. Accessed 2011 Sep 12.

Articles from Canadian Family Physician are provided here courtesy of College of Family Physicians of Canada