Effective, quick measurement of clinical care is an important area of research for today’s quality improvement efforts [16
]. In this study, we assessed the proportion of patients who were up to date with CDC and ACR guidelines on pneumococcal vaccination with the ultimate goal of spurring quality improvement initiatives within our outpatient rheumatology practice. The use of multiple data sources, such as administrative data sets (billing) and EHRs, allow for rich data to be routinely collected and used for quality measurement as compared with single data sources [17
]. We used methodology that combines billing and clinical data to create a metric for pneumococcal vaccination process improvement. We found that in our outpatient rheumatology practice, 54% of patients on immunosuppressive medications therapy and 45% of patients newly started on immunosuppressive medications therapy were up to date with pneumococcal vaccination.
Our vaccination rates are below the desired goals but slightly higher than those previously published among rheumatology outpatients in the UK (finding 20–35% adherence) and USA (19–41% adherence) [8–11
]. Some of these prior studies included small sample sizes (n
200) and non-electronic modalities of gathering data; thus, measurement cannot be repeated as easily over time to improve care. A single-centre USA study found lower rates of pneumococcal vaccination (19%) among patients taking immunosuppressive medications. That centre was able to increase the rate to 41% through alerts in the EHR, providing valuable support for the use of electronic reminders to improve quality for pneumococcal vaccination; however, not all practices will readily be able to integrate new clinical reminders into existing EHRs [10
In this study, we report on the use of available administrative and clinical data to facilitate repeated measurement of the process of care. This allows for practices to continuously measure performance to measure the effectiveness of quality improvement interventions. Since our initial measurement, we have been tracking our pneumococcal vaccination Measures 1 and 2 for over 1 year and have seen a gradual steady improvement over time (). The use of electronic data sources allows for us to tabulate Measures 1 and 2 monthly, follow trends over time and provide feedback to our clinic. Our methodology also permits for capture of patient and physician characteristics related to pneumococcal vaccination status. The advantages of our methodology are that manual chart review is not required, data collection is current, and performance can be measured regularly to provide direct feedback.
Fig. 2 Pneumococcal vaccination over time—Measures 1 and 2. Measure 1 is the percentage of patients up-to-date with vaccine while on immunosuppressive therapy, and Measure 2 is the percentage of patients up-to-date with vaccine before starting immunosuppressive (more ...)
Quality indicators (QIs) are defined as the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge [20
]. QIs can be utilized to identify potential areas for improvement, to track changes over time and to improve patient care [21
]. QIs are being developed to measure the quality of care provided to rheumatological patients [23
]. It is possible that Measures 1 and 2 could potentially be used as QIs. A QI should have meaning, be evidence based, generalizable and interpretable [24
]. It is essential to use strict definitions to clearly define the numerator and denominator for each measure to increase the face validity and content validity of the metric [26
]. The process of defining Measures 1 and 2 took over 6 months to develop using a collaborative team approach. For example, we are modifying the definitions in our quality metric to capture patients on prednisone therapy at dosages >10
mg for >6 months duration to reflect immunosuppression to ensure that we are capturing rheumatology patients who are on chronic immunosuppression.
There are several factors that may explain the relatively low adherence to established guidelines for pneumococcal vaccination. When starting new immunosuppressive medications, a provider addresses an overview of the risks and benefits, a discussion of the need for routine lab monitoring and associated logistics if labs are to be obtained off-site, as well as tuberculosis exposure history. In our rheumatology clinic, we do not currently have a standardized approach to ensure that all patients newly started on immunosuppressive medications receive a pneumococcal vaccination. The degree of variability between different rheumatologists within our same practice may reflect this lack of a standardized process for documenting pneumococcal vaccination status or clinical performance.
We explored the wide variability in performance further. Rheumatologists C and E had higher performance than any of the other rheumatologists. Informal discussions with these two providers revealed that Rheumatologist C has been integrally involved in the development and implementation of this pneumococcal vaccination project and was a former primary care provider for many patients. Rheumatologist E feels that pneumococcal vaccination is an essential aspect of the process of starting an immunosuppressive medication and has developed a habit of vaccinating routinely.
There may be other plausible aetiologies to explain the relatively low adherence to CDC guidelines for pneumococcal vaccination that we observed as well. Knowledge or acceptance by clinicians that vaccinating patients on immunosuppressive medication is the right thing to do may not be universal. Studies in patients with rheumatological conditions suggest that although some patients may not mount a robust immune response to pneumococcal vaccination, most patients do respond [27
] The evidence base for the antibody response to patients on immunosuppressive medications such as MTX or biologics has been conflicting [27
]. However, the summary of existing evidence and current treatment guidelines call for pneumococcal vaccination for chronic rheumatic disease patients on immunosuppressive therapy [5–7
In addition, the actual measurement of pneumococcal vaccination may not be comprehensive. Electronic data sources may not fully capture the numerator of patients who are up to date with their vaccines. One concern is off-site vaccinations (i.e., vaccine administered at a site other than our institution and not documented in the measured section of the EHR). Our chart review found that 14% of the time rheumatologists documented pneumococcal vaccine in their notes and electronic measurement does not capture these patients.
The limitations of our data analysis are several. Presently, we are unable to fully capture patients on i.v. medications administered in our ambulatory infusion centre, as these data are captured by a different billing system than that used in our analysis. By our estimation, ~9% of our RA patients are on infliximab, abatacept or rituximab. We expect to obtain data on these i.v. medications in the coming year and we are also encouraging our rheumatologists to document the use of these medications in the EHR medication list to increase the accuracy of our measurement. We also do not have access to the electronic medication administration record for inpatient pneumococcal vaccination. It is quite possible that rheumatological patients hospitalized for other medical issues may have received the pneumococcal vaccine as inpatients. We anticipate having access to the electronic medication administration record within the next 6 months.
The work described in this study has led to the use of a DMARD checklist for patients starting immunosuppressive therapy and a process of flagging patients who are overdue for pneumococcal vaccination when they present for their routine rheumatology appointments. At the present time, we are focusing our quality improvement efforts on patients prescribed immunosuppressive medications by rheumatologists in our practice, but we hope to expand our efforts in the future to include patients prescribed immunosuppressives by non-rheumatologists. In order to improve the quality of care for immunosuppressed patients overall, we must ensure that all physicians take responsibility for the care of patients seen in their practice.
In our first step of quality improvement, we report on the pneumococcal vaccination performance in our practice over time. In our defined cohort of rheumatology outpatients, we found 54% of patients on immunosuppressive medications were up to date on pneumococcal vaccination and 45% of patients were up to date on vaccination at the time immunosuppressive therapy was initiated. We can now generate routine reports to assess current practice performance and patterns. Our new approach can be used in a prospective manner to produce periodic assessments of adherence to pneumococcal vaccination for eligible patients. Now that we have set the foundation, our next steps are to improve over time through practice redesign principles and repeated measurement [10