|Home | About | Journals | Submit | Contact Us | Français|
Objective. The Centers for Disease Control (CDC) recommends pneumococcal vaccination for immunocompromised patients. Data suggest that rates of vaccination in this population are not optimal. To support continuous quality improvement efforts, we electronically measured vaccination status among rheumatology outpatients over time.
Methods. Using data from administrative (billing) and electronic health record sources, we identified rheumatology clinic patients seen between 1 February 2008 and 31 January 2010 and prescribed an immunosuppressive medication. CDC recommendations for pneumococcal vaccination were applied. We calculated the proportion of eligible patients who were up-to-date with pneumococcal vaccination: (i) while on an immunosuppressive medication and (ii) before newly starting an immunosuppressive medication in the last 12 months.
Results. We identified 2763 rheumatology clinic patients on immunosuppressive medications, with 568 initiated in the last 12 months. The mean age was 57 years, 75% were female and 77% were Caucasian. The most frequent disease was RA (50%) and the most common immunosuppressive medication was MTX (59%). Of patients on immunosuppressive medications, 1491/2763 (54%) were up to date with pneumococcal vaccination. Among new initiators of immunosuppressive medications, 258/568 (45%) were vaccinated before starting the immunosuppressive medication. Patients treated by rheumatologists in practice for ≤10 years were more likely to be up to date with pneumococcal vaccination (72%) than those with providers in practice >10 years (52%, P<0.001).
Conclusion. The proportion of patients who are up to date with documented pneumococcal vaccination was suboptimal in our rheumatology practice. The ability to continuously repeat electronic measurement permits us to initiate continuous quality improvement efforts.
Recent evidence demonstrates a gap between typical care and optimal care in the management of chronic diseases, including many rheumatological conditions [1–4]. Patients with rheumatological conditions often receive immunosuppressive medications as part of their treatment. The resulting immunocompromised state puts patients at increased risk for a wide range of infections, including invasive Streptococcus pneumoniae . Accordingly, guidelines from the Centers for Disease Control (CDC) and guidelines in the management of RA from the ACR recommend pneumococcal vaccination for patients receiving immunosuppressive therapy [6, 7]. Yet rates of pneumococcal vaccination appear to remain suboptimal [8–10].
The explanation for low pneumococcal vaccination rates is multifactorial. Physicians and patients may have decreased awareness of the importance of pneumococcal vaccination . There is often a delay in the translation of new recommendations and guidelines into clinical practice, which can result from a lack of reminder systems at the point of care . Busy, complex outpatient practices may not have the resources, either workflow or staffing, to integrate routine vaccination into clinical care. Lastly, there may be problems with accurate documentation of pneumococcal vaccination.
At our hospital, we have begun to develop new measures, using administrative and clinical data, to assess and improve the quality of care provided to rheumatological patients. Our initial effort seeks to measure and improve pneumococcal vaccination in patients on immunosuppressive medications. The objective of this analysis is, first to report the adherence to pneumococcal vaccination guidelines for rheumatology outpatients on immunosuppressive medications. In our effort to guide improvement efforts, we then examine determinants of pneumococcal vaccination.
In this analysis, we evaluated Brigham and Women’s Hospital Rheumatology clinic patients that were prescribed immunosuppressive medications and were seen in a 24-month period, 1 February 2008 to 31 January 2010, for pneumococcal vaccination status. A combination of administrative data (diagnosis and billing codes) and electronic health record (EHR) clinical data (vaccination dates, history of allergy to pneumococcal vaccination, medications) was acquired from our hospital’s information systems’ Quality Data Warehouse (QDW) . The QDW is a database that aggregates information from a variety of data sources (including the EHR) and can generate reports for specific clinical queries. We used these data to calculate the percentage of patients in our outpatient rheumatology clinic that were up to date with pneumococcal vaccination while on an immunosuppressive medication (Measure 1), and before initiating immunosuppressive treatment, if started in the previous 12 months (Measure 2).
Institutional review board approval for study of our quality improvement effort was obtained from the Partners Human Research Committee. The requirement for written informed consent was waived.
To capture patients seen regularly (as opposed to seen for consultation only) and managed in our rheumatology practice, patients were included in our evaluation only if they had at least two visits meeting one of the following time criteria: (i) having at least one visit within the first 12 months of the 24-month measurement period, and at least one visit within the second 12 months of the measurement period; or (ii) for patients with visits only during the second 12 months of the measurement period, the first and last visits were separated by a minimum of 90 days. Common procedural terminology, version 4 (CPT-4) codes for new patient visits (CPT-4 codes 99201–99205), established patient visits (CPT-4 codes 99211–99215) and consultations (CPT-4 codes 99241–99245) defined eligible rheumatology clinic visits.
The use of an immunosuppressive medication was determined by the presence of an active prescription initiated or renewed in the past 36 months in the electronic health record (EHR), for at least one non-biologic or biologic DMARD or corticosteroid (CS). DMARDs included SSZ, MTX, LEF, AZA, ciclosporin, MMF, CYC, etanercept and adalimumab. The intravenous (i.v.) immunosuppressive medications—infliximab, abatacept, rituximab and CYC—were also included. However, outpatient infusion centre data indicating which patients received these medications were not complete. Thus, unless a provider had listed an active prescription for an i.v. medication in our EHR medication list, patients on i.v. immunosuppressive agents alone were not included in the denominator.
Dosage thresholds were created for CSs to account for the relative immunosuppressive effect of higher doses of CSs; patients on chronic regimens of >10mg/day of prednisone (or equivalent CS dose) were included in the study regardless of the duration of therapy; patients prescribed a short course CS taper, such as a methylprednisolone dose pack, were not included. Patients on immunosuppressive medications prescribed solely by non-rheumatologists were also not included in the sample.
The criteria for being up-to-date with pneumococcal vaccination for Measures 1 and 2 were defined according to CDC guidelines (Table 1). The goal of Measure 1 was to assess whether patients were currently up to date with pneumococcal vaccination while on immunosuppressive medications, irrespective of when the medication was first started. The goal of Measure 2 was to calculate the percentage of patients who were up to date with pneumococcal vaccination at initiation of immunosuppressive medication in the last 12 months.
Pneumococcal vaccination status was determined through documentation in structured data fields in the immunization or health maintenance monitoring fields of the EHR. Typically, when a member of the health care team administers the vaccine, he or she documents the date of administration in this structured data field in the EHR. If the pneumococcal vaccination was documented only in the text of a rheumatologist’s note, this was not counted. Patients who had an allergy or an adverse reaction to pneumococcal vaccination documented in the health maintenance or allergy field of the EHR were excluded from the studied population.
We collected additional patient data on age, race and gender, and rheumatic disease diagnosis from billing codes and EHR data. A descriptive analysis was used to characterize the study population. Measures 1 and 2 were proportions, defined as the number of subjects meeting the metric (numerator) divided by the total number of eligible subjects (denominator). For each proportion, 95% CIs were calculated using the Wilson score method without continuity correction . Chi-squared tests were conducted to test the difference between two proportions . All patients were assigned a primary attending rheumatologist with whom they had the greatest number of billed visits (and in the case of a tie, the most recent visit); patients were not assigned to residents and fellows. Data for Measures 1 and 2 were stratified based on the primary rheumatologist’s number of years in practice, patient’s rheumatological condition and prescribed immunosuppressive medication.
To evaluate our measurement methods, manual chart review was conducted on a randomly selected subset of charts. The primary goal of this chart review was to compare the electronic measurement of the numerator (up to date with pneumococcal vaccination) and the denominator (on immunosuppressive therapy) with the data obtained through manual chart review of the EHR.
In our analysis, we identified 2763 rheumatology clinic patients on at least one immunosuppressive medication, of which 568 patients were newly started. Among the rheumatology outpatients eligible to be in the denominator, 75% were female, 77% were Caucasian and the mean age was 57 years (Table 2). Patients were on a broad range of immunosuppressive medications, but the most commonly prescribed one was MTX (59%). The distribution of rheumatic diseases revealed that the most frequent diagnosis was RA (50%), followed by other inflammatory arthropathies (10%), PsA (10%) and SLE (9%).
The proportion of patients on immunosuppressive medication who were up to date with pneumococcal vaccination (Measure 1) was 54% (95% CI 52, 56). The proportion of patients newly starting immunosuppressive medication in the past 12 months who were up to date with pneumococcal vaccination at the time of initiation (Measure 2) was 45% (95% CI 41, 50). We then assessed for patient determinants of pneumococcal vaccination administration, including age, race and rheumatic disease. We did not find any statistically significant differences in the percentage of patients who are up to date with vaccination when analysed by these patient factors (Table 3).
Individual rheumatologist-level data demonstrated considerable variability in the proportion of patients who are up to date with pneumococcal vaccination for Measures 1 and 2 (Fig. 1). In analyses stratified by years in practice of the rheumatologist, those who had been in practice for ≤10 years had higher proportions of patients who are up to date with pneumococcal vaccination both for Measure 1 (72 vs 52%, P<0.001) and Measure 2 (69 vs 42%, P<0.001) compared with those who were in practice for >10 years (Table 4).
Results from our manual chart review performed to assess the accuracy of our methodology showed that if the electronic measurement defined a patient as immunosuppressed, this was confirmed by chart review to be correct 100% of the time (n=25 charts reviewed). If the electronic measurement defined a patient as not immunosuppressed and the patient was excluded from the denominator, then this was substantiated 88% (95% CI 70, 96%) of the time by chart review (n=25). The 12% (95% CI 4, 30%) of patients missed by the electronic measurement were due to inaccurate capture of immunosuppressive medications prescribed as ‘free-text’ rather than as coded data elements.
If the electronic measurement defined a patient as up to date with pneumococcal vaccination, this was correct 100% of the time (n=25 charts reviewed). Lastly, if the electronic measurement defined a patient as not up to date with pneumococcal vaccination, this was validated 86% (95% CI 74, 93%) of the time (n=50 charts reviewed). The 14% (95% CI 7, 26%) of patients who were missed by the electronic measurement had their pneumococcal vaccination status documented in the text of the rheumatologists’ notes.
Effective, quick measurement of clinical care is an important area of research for today’s quality improvement efforts . In this study, we assessed the proportion of patients who were up to date with CDC and ACR guidelines on pneumococcal vaccination with the ultimate goal of spurring quality improvement initiatives within our outpatient rheumatology practice. The use of multiple data sources, such as administrative data sets (billing) and EHRs, allow for rich data to be routinely collected and used for quality measurement as compared with single data sources [17, 18]. We used methodology that combines billing and clinical data to create a metric for pneumococcal vaccination process improvement. We found that in our outpatient rheumatology practice, 54% of patients on immunosuppressive medications therapy and 45% of patients newly started on immunosuppressive medications therapy were up to date with pneumococcal vaccination.
Our vaccination rates are below the desired goals but slightly higher than those previously published among rheumatology outpatients in the UK (finding 20–35% adherence) and USA (19–41% adherence) [8–11, 19]. Some of these prior studies included small sample sizes (n<200) and non-electronic modalities of gathering data; thus, measurement cannot be repeated as easily over time to improve care. A single-centre USA study found lower rates of pneumococcal vaccination (19%) among patients taking immunosuppressive medications. That centre was able to increase the rate to 41% through alerts in the EHR, providing valuable support for the use of electronic reminders to improve quality for pneumococcal vaccination; however, not all practices will readily be able to integrate new clinical reminders into existing EHRs .
In this study, we report on the use of available administrative and clinical data to facilitate repeated measurement of the process of care. This allows for practices to continuously measure performance to measure the effectiveness of quality improvement interventions. Since our initial measurement, we have been tracking our pneumococcal vaccination Measures 1 and 2 for over 1 year and have seen a gradual steady improvement over time (Fig. 2). The use of electronic data sources allows for us to tabulate Measures 1 and 2 monthly, follow trends over time and provide feedback to our clinic. Our methodology also permits for capture of patient and physician characteristics related to pneumococcal vaccination status. The advantages of our methodology are that manual chart review is not required, data collection is current, and performance can be measured regularly to provide direct feedback.
Quality indicators (QIs) are defined as the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge . QIs can be utilized to identify potential areas for improvement, to track changes over time and to improve patient care [21, 22]. QIs are being developed to measure the quality of care provided to rheumatological patients . It is possible that Measures 1 and 2 could potentially be used as QIs. A QI should have meaning, be evidence based, generalizable and interpretable [24, 25]. It is essential to use strict definitions to clearly define the numerator and denominator for each measure to increase the face validity and content validity of the metric . The process of defining Measures 1 and 2 took over 6 months to develop using a collaborative team approach. For example, we are modifying the definitions in our quality metric to capture patients on prednisone therapy at dosages >10mg for >6 months duration to reflect immunosuppression to ensure that we are capturing rheumatology patients who are on chronic immunosuppression.
There are several factors that may explain the relatively low adherence to established guidelines for pneumococcal vaccination. When starting new immunosuppressive medications, a provider addresses an overview of the risks and benefits, a discussion of the need for routine lab monitoring and associated logistics if labs are to be obtained off-site, as well as tuberculosis exposure history. In our rheumatology clinic, we do not currently have a standardized approach to ensure that all patients newly started on immunosuppressive medications receive a pneumococcal vaccination. The degree of variability between different rheumatologists within our same practice may reflect this lack of a standardized process for documenting pneumococcal vaccination status or clinical performance.
We explored the wide variability in performance further. Rheumatologists C and E had higher performance than any of the other rheumatologists. Informal discussions with these two providers revealed that Rheumatologist C has been integrally involved in the development and implementation of this pneumococcal vaccination project and was a former primary care provider for many patients. Rheumatologist E feels that pneumococcal vaccination is an essential aspect of the process of starting an immunosuppressive medication and has developed a habit of vaccinating routinely.
There may be other plausible aetiologies to explain the relatively low adherence to CDC guidelines for pneumococcal vaccination that we observed as well. Knowledge or acceptance by clinicians that vaccinating patients on immunosuppressive medication is the right thing to do may not be universal. Studies in patients with rheumatological conditions suggest that although some patients may not mount a robust immune response to pneumococcal vaccination, most patients do respond [27, 28] The evidence base for the antibody response to patients on immunosuppressive medications such as MTX or biologics has been conflicting [27, 29–33]. However, the summary of existing evidence and current treatment guidelines call for pneumococcal vaccination for chronic rheumatic disease patients on immunosuppressive therapy [5–7].
In addition, the actual measurement of pneumococcal vaccination may not be comprehensive. Electronic data sources may not fully capture the numerator of patients who are up to date with their vaccines. One concern is off-site vaccinations (i.e., vaccine administered at a site other than our institution and not documented in the measured section of the EHR). Our chart review found that 14% of the time rheumatologists documented pneumococcal vaccine in their notes and electronic measurement does not capture these patients.
The limitations of our data analysis are several. Presently, we are unable to fully capture patients on i.v. medications administered in our ambulatory infusion centre, as these data are captured by a different billing system than that used in our analysis. By our estimation, ~9% of our RA patients are on infliximab, abatacept or rituximab. We expect to obtain data on these i.v. medications in the coming year and we are also encouraging our rheumatologists to document the use of these medications in the EHR medication list to increase the accuracy of our measurement. We also do not have access to the electronic medication administration record for inpatient pneumococcal vaccination. It is quite possible that rheumatological patients hospitalized for other medical issues may have received the pneumococcal vaccine as inpatients. We anticipate having access to the electronic medication administration record within the next 6 months.
The work described in this study has led to the use of a DMARD checklist for patients starting immunosuppressive therapy and a process of flagging patients who are overdue for pneumococcal vaccination when they present for their routine rheumatology appointments. At the present time, we are focusing our quality improvement efforts on patients prescribed immunosuppressive medications by rheumatologists in our practice, but we hope to expand our efforts in the future to include patients prescribed immunosuppressives by non-rheumatologists. In order to improve the quality of care for immunosuppressed patients overall, we must ensure that all physicians take responsibility for the care of patients seen in their practice.
In our first step of quality improvement, we report on the pneumococcal vaccination performance in our practice over time. In our defined cohort of rheumatology outpatients, we found 54% of patients on immunosuppressive medications were up to date on pneumococcal vaccination and 45% of patients were up to date on vaccination at the time immunosuppressive therapy was initiated. We can now generate routine reports to assess current practice performance and patterns. Our new approach can be used in a prospective manner to produce periodic assessments of adherence to pneumococcal vaccination for eligible patients. Now that we have set the foundation, our next steps are to improve over time through practice redesign principles and repeated measurement .
A.T. is supported by a National Institutes of Health grant (1RC1LM010460) and an Agency for Healthcare Research and Quality (AHRQ) grant (5R18HS017030). D.H.S. is supported by National Institutes of Health grants (K24 AR 055989, P60 AR 0477820, R21DE018750 and R01 AR056215) and an AHRQ DECiDE Center Award. S.P.D.'s time during this project was funded by a National Institutes of Health grant (T32 AR007530, Institutional Training Grant) and an American College of Rheumatology Research and Education Fund Physician Scientist Development Award.
Funding: The study was funded by the ACR-REF research grant and some internal division funds.
Disclosure statement: The authors have declared no conflicts of interest.