EMR-driven performance measurement linked to payment incentives is going to be part of the clinical landscape in the coming years and perhaps beyond. The Meaningful Use Incentive Program alone dictates that quality measurement will be a high priority topic for the next three to five years. Additional long term policy initiatives such as the accountable care model will likely necessitate use of electronic quality measurement to meet program mandates or to support operational needs.
Our study used electronic data to calculate physician performance on prescribing guideline recommended antibiotics for CAP. We focused on use of the measure in the ED setting, since other studies have evaluated the measure in an ambulatory environment.14
We strove to study the measure as it is calculated and reported in an ongoing program, CMS’s Physician Quality Reporting System. Performance measurement based on known best practices is assumed, but not proven, to improve processes of care and patient outcomes.
ED physician performance for all 316 patients seen in calendar year 2009 was moderate, 70.6%. Almost all (97.8%) measure failures were patients admitted to the hospital from the ED with only a beta-lactam administered or prescribed; the remaining 2.2% were discharged home without appropriate antibiotics prescribed or administered. Performance at our institution was similar to that seen in other studies that looked at empiric antibiotic treatment of CAP.15
The measure successfully identified variations of care in our institution. Measure compliant cases had significantly shorter length of stays, less visits to the ICU, and lower direct and total costs. Mortality and 30 day all-cause readmissions were lower in the measure compliant group, but this finding was not significant. Our results suggest this measure identifies cases where there is an opportunity for quality improvement and potential cost savings. Quality measures are often criticized for measuring processes, rather than outcomes, of care. For this quality measure, completing the process being measured resulted in improved outcomes. We were also able to automate performance calculation using EMR data without additional measurement-specific documentation, an important consideration since quality measurement can be burdensome.
Interpreting our results requires an understanding of limitations of the performance measure, particularly the use of ICD-9-CM codes to identify the denominator cases. ICD-9-CM coding is known to be inaccurate at correctly identifying pneumonia cases.5
Additionally, the IDSA guidelines referenced in the measure make different recommendations depending on the severity of illness.11
The measure does not attempt to replicate this complex logic. Finally, ICD-9-CM codes only capture the high level concept of pneumonia; codes do not exist to distinguish CAP from hospital acquired pneumonia, which requires different antibiotic therapy. Use of a comprehensive standardized vocabulary such as the Systematized Nomenclature of Medicine--Clinical Terms (SNOMED-CT) in the measure specifications may allow more accurate identification of eligible cases of CAP.
Besides use of recommended antibiotics, another plausible explanation for differences in outcomes is that the non-compliant patients were more acutely ill. ICD-9-CM diagnosis codes are not detailed enough to establish pneumonia severity, so we cannot rule this out. However, the compliance and failure groups had similar Elixhauser severity of illness scores and demographic factors. Additionally, 57.1% of measure failures had their antibiotic regimen changed to be measure compliant within 48 hours. This finding suggests that measure compliant antibiotic treatment was, in fact, appropriate in the majority of measure failures. Patients may have benefited from earlier administration of measure compliant antibiotic therapy, regardless of the severity of pneumonia.
During the course of our work, several opportunities for refining the measure became apparent. With the availability of EMR data rather than just claims data, the measure specifications could be expanded to better address the nuances of treating pneumonia. For example, risk of nosocomial infection could be ascertained by looking for recent hospital admissions, history of dialysis, or admission from a long term care facility. Severity of pneumonia could be determined from EMR data or diagnoses coded with standardized medical vocabularies like SNOMED-CT. Performance could then be more accurately assessed based on guideline recommendations for both the severity of pneumonia and likelihood of facility versus community acquired disease.
Our findings are restricted to this specific measure and cannot be generalized to other measures since each quality measure is different. However, our results demonstrate that other measures can be similarly successful. Additional investigation is necessary to validate that our findings would hold across other EMR products and institutions; a multi-center study would be beneficial. Finally, more research would be beneficial to understand the variation of care noted at our institution, including the contribution of severity of pneumonia to differences in outcomes.