|Home | About | Journals | Submit | Contact Us | Français|
Influenza and related illness are among the leading causes of death among Americans aged 65 years and older 1, 2. The elderly are at high risks of influenza-related hospitalization3. Compared with the community dwelling elderly, nursing home residents are at even higher risks due to their frailty and because they live in an environment where infections are more likely to proliferate4-6.
Influenza vaccine is a primary way to reduce influenza related illness and mortality 7, 8. Studies have demonstrated the effectiveness of influenza vaccine in preventing acute respiratory track illness as well as reducing mortality among the elderly in nursing homes9-11. Despite the importance of influenza vaccine, however, immunization rates among nursing home residents are far from optimal and well below the goal of 90% as suggested by the Healthy People 2010 12-14.
In order to improve influenza vaccination rates in nursing homes, the Centers for Medicare and Medicaid services (CMS) developed the Nursing Home Immunization Breakthrough Initiative in 2005. In October 2005 CMS published the final rule requiring all Medicare and Medicaid participating nursing homes to ensure that residents will be offered the opportunity to get an influenza vaccine.12 The regulation also requires nursing homes to document, in the medical records, that the residents were provided with an education about the benefits of the vaccine, and their subsequent acceptance or refusal of the vaccine15. In October 2006, CMS started to publish influenza immunization rates for each nursing facility on the Nursing Home Compare website. The goal of such public reporting is to improve the quality of care in nursing homes. Studies have shown that quality report cards may alter the behavior of nursing homes and help improve the quality of care provided16, 17. However, as far as our knowledge, there have been no studies as to how nursing homes have responded to public reporting with regard to influenza vaccinations. Furthermore, it is important to understand whether changes in vaccination rates have impacted residents' health outcomes. This study will address both of these issues by focusing on two objectives:
To address the first objective, we employed the national Nursing Home Compare (NHC) database. The NHC database is collected by the CMS, and updated quarterly. It contains 19 quality indicators for all Medicaid/Medicare certified nursing homes in the country. The measurements for flu vaccination rates are collected annually during the flu season (Oct-Mar) 18, for both short-term residents (those who receive skilled nursing care or rehabilitation care) and long-term residents (those who receive custodian care and are expected to stay in nursing home for a long time). There is a time lag between the actual data collection and the time when these measurements become available in the NHC database 19. Thus, the NHC data available in the first quarter of 2007, and the first and the last quarters of 2008 were obtained to represent flu vaccination rates during the 2005-2006, 2006-2007, and 2007- 2008 flu seasons, respectively. In total, the NHC database included 16,140 Medicare/Medicaid certified nursing homes nationwide. Some facilities were excluded by CMS because their vaccination rates were not valid due to the small number of residents in the facility. Our final analytical sample contained 15,560 nursing homes.
In order to tease out the possible time trend effect on nursing home influenza vaccination rates, we also obtained data for flu immunization rates for community dwelling elderly (age 65 or older) from the Centers for Disease Control and Prevention (CDC) website for the years 2005-200720.
We used two data sources in examining this objective: the Minimum Data Set (MDS) for New York State for the years 2005-2007 and the NHC compare database for 2005-2006 and 2006-2007 flu seasons. The MDS is a federally mandated datasets containing detailed health and demographic information for each resident. We used the NHC database to obtain the information about flu vaccination rate for each facility in New York State because the MDS dataset available to us did not contain individual level vaccination records. In order to cover the whole flu season (October to May),21 we selected the MDS records from Oct 2005 through May 2006 (i.e. before public reporting) and Oct 2006 through May 2007 (i.e. after public reporting). In total, 648 nursing homes were identified. 59 facilities were excluded because the NHC database did not have influenza vaccination rates for either long-term or short-term residents during the two flu seasons. The final sample contained 589 facilities with all residents present during the two flu seasons (204,580 and 208,363 residents during 2005- 2006 and 2006-2007 flu seasons, respectively).
To investigate the effect of flu vaccinations on hospitalization rates (objective 2), we identified the following dependent and independent variables.
Facility hospitalization rate was the outcome variable of interest. It was measured as the ratio of the number of residents who had ever been hospitalized to total resident days in a facility during the flu seasons. We calculated the total number of resident days rather than simply the number of residents in a facility because the occurrence of hospitalization events could also be affected by the length of stay in the facility. We identified hospitalization events using the MDS records with discharge destination coded as ‘acute hospital’. Based on our previous study, the quality of the MDS in recording hospitalization events in New York State is quite good22. Specific reasons for hospitalizing nursing home residents can not be identified from the MDS. However, studies have shown that respiratory infections are the main reason for hospitalizations among nursing home residents23,24, 25. Therefore, we have included all-cause hospitalizations occurring during the flu season.
Based on the NHC data18, two types of influenza vaccination rates were used: vaccination rates for long-term (LT) residents and for short-term (ST) residents. In the regression model, these rates were constructed on a 0-10 scale in increments of 10% (we divided vaccination rates, which ranged from 0-100%, by 10).
We controlled for the facility-level case-mix using the average RUG-III score (version 5.12; 44 group model) for each facility. First, based on 108 MDS items from the first available assessment during the flu season, we assigned a RUG III score to each resident. We then converted the RUG classifications to case mix index (CMI) for each resident, and calculated the facility average CMI.
Other facility level characteristics included ownership, facility size, and chain membership and were obtained from the NHC database.
Facility was the unit of analysis. Since our analyses were based on the entire nursing home population (all nursing homes in the country for objective-1, and all nursing homes in New York State for objecteive-2), rather than a random sample, the effects we detected were the true effect. Thus, instead of making statistical inferences based on conventional p values, we evaluated the effect of the variables of interest by their actual effect size. Below, the specific analyses for objectives 1 and 2, are discussed.
All US nursing homes with valid influenza vaccination measures were used to compare vaccination rates before and after public reporting. An increase in vaccination rates in this time period could be caused by a secular trend rather than as a direct result of public reporting. In order to isolate the possibility of such a trend effect, we compared vaccination rates in nursing homes and in community dwelling elderly, since nursing home public reporting (starting in Oct 2006) should have had no effect on the community dwelling population.
The structural model for investigating the effect of vaccination rates on hospitalization of nursing home residents was defined as:
where yit is outcome variable; xit is a vector for facility level characteristics, including the main variables of interests; tt captures the time trend; and αi indicates unobserved facility factors that may correlate with both the outcome and the vaccination rate. In order to account for the unobserved αi, which could bias the estimate of the effect of vaccinations, we took advantage of having longitudinal data and estimated a first-differenced regression model rather than a cross-sectional model. Here we assumed the facility level unobserved αi was time invariant between the two years. We differenced the data across these two flu seasons.
The advantage of a first-differenced model is that it permits us to control for the unobserved, time-invariant facility factors, so that the unobserved factor αi is cancelled out. ‘Δ’ indicates the change from the 2005-2006 flu season to 2006-2007 flu season. The intercept 0 is the change in the intercept between two flu seasons, capturing the time trend between the two flu seasons. Δxi denotes the change in time-variant facility characteristics, such as change in vaccination rates between the two flu seasons. Thus, according to the first-differenced model, all variables are measured by facility level changes between the 2005-2006 and the 2006-2007 flu seasons. Since certain facility characteristics (i.e. facility ownership, facility size) are time-invariant, they do not enter the first-differenced model.
The effect of changes in vaccination rates may be different in facilities where baseline vaccination rates are low, compared to those with high baseline rates. Therefore, we stratified the sample into two subgroups according to the distribution of the flu vaccination rates for short-term residents in 2005-2006 flu season. The group with a high baseline vaccination rate (top 25% percentile) included 437 facilities, and the group with a low baseline vaccination rate (bottom 25% percentile) contained 152 facilities. We used vaccination rates for short-term residents to stratify facilities in this manner, because vaccination rates for long-term care residents were very high (mean=91%) in New York State.
The average flu vaccination rates across states are presented in Table 1. For short-term nursing home residents, average influenza vaccination rates in the 2005-2006 flu season ranged from 60.87% in Florida to 87.73% in North Dakota, with a nationwide mean of 74.64 % (SD=6.63). During the 2006-2007 flu season, the state average vaccination rates ranged from 61.38% in Minnesota to 89.53% in North Dakota, with a nationwide mean of 76.99% (SD=6.06). The rates ranged from 63.14% in Minnesota to 89.50% in North Dakota during the 2007-2008 flu season, with a nationwide mean of 80.10% (SD=5.56). Over the three seasons, 38 states experienced a continuous improvement in statewide flu vaccination rates. Influenza vaccination rates for short-term nursing home residents, over the three flu seasons, are presented in Figure-1.
For long-term nursing home residents, state average influenza vaccination rates during the 2005-2006 flu season ranged from 75.42% in Florida to 94.17% in Alaska with a nationwide mean of 87.15% (SD=4.58). State average influenza vaccination rates during the 2006-2007 flu season ranged from 76.50% in Nevada to 94.17% in Alaska with a nationwide mean of 87.88% (SD=3.79). During the 2007-2008 flu season, state average influenza vaccination rates ranged from 75.66% in Nevada to 95.0% in Alaska, with a nationwide mean of 88.82% (SD=3.96). Over these three flu seasons, 29 states experienced continuous improvement in statewide flu vaccination rates. State-wide trends in influenza vaccination rates for long-term residents over the three flu seasons are presented in Figure-2.
The nationwide average vaccination rates for community dwelling elderly increased from 65.64% in 2005 to 72.05% in 2007. The comparison of the national average vaccination rates for nursing home and community dwelling populations is shown in Table-1. On average, the increase in influenza vaccination rates among the short-term nursing home residents was not greater (5.46%) than that among the community dwelling elderly (6.41%). Among long-term care nursing home residents, influenza rates changed less than 2% over the three flue seasons.
In New York State, the average flu vaccination rate for short-term residents was 79.57% (SD=15.23) during the 2005-2006 flu season and 81.43% (SD =14.49) during the 2006-2007 flu season (Table 2). For long-term residents, the influenza vaccination rate was 90.99% (SD=7.87) during the 2005-2006 flu season and 90.50% (SD=9.43) during the 2006-2007 flu season. Compared with the national mean, the average vaccination rates in NYS are quite high, for example, the vaccination rate for short-stay residents is 79.6 % in NYS versus 74.6 % nationwide during 2005-2006 flu season. The vaccination rates for long-term care residents in NYS have already reached the Healthy People 2010 objective of 90%.
As presented in Table-2, in the 2005-06 flu season, immediately preceding public reporting, the difference in flu vaccination rates between facilities with low and high baseline rates were substantial, 58.53% versus 86.90% respectively, for short-term residents. The change in vaccination rates in facilities with the low baseline rate increased also substantially, from 58.53% to 70.22%, between the two flu seasons (before and after public reporting). We did not observe a similar improvement in the subgroup of facilities with high baseline rates either for short or long-term residents.
Facility characteristics for the two subgroups of nursing homes are also depicted in Table 2. Facilities with low baseline vaccination rates tended to be larger than facilities with high baseline vaccination group.
Table-3 presents the results from the regression model. Because we estimated a log-linear model, parameters were interpreted as the average percentage change in the outcome variables with respect to a unit change in the independent variable. Thus, for facilities with low baseline flu vaccination rates, a 10% increase in vaccination rate for long-term residents was associated with a 6.2% decrease of baseline hospitalization rates. However, in these facilities, we also found a 4.6% increase in hospitalization rates for each 10% increase in vaccination rates for short-term residents. In facilities with high baseline flu vaccination rates, a 10% increase in vaccination rates for short-term residents was associated with 0.9% decrease in baseline hospitalization rates. Similarly, a 10% increase in vaccination rates for long-term residents was correlated with 0.7 % decrease of baseline hospitalization rates.
While nursing home flu vaccination rate seemed somewhat higher following the implementation of public reporting, it was not possible to attribute this change to the effect of public reporting, as increases in flu vaccination rates among community dwelling elderly also occurred during this time. We were unable to differentiate the effect of time trend from public reporting itself. Perhaps, one reason we were not able to detect larger changes in vaccination rates before and after public reporting, is that we only had one year of data available for the period prior to the implementation of public reporting. CMS started to require nursing home to provide vaccination opportunity to their residents beginning with the 2005-06 flu season (Oct 2005-Mar 2006), although vaccination rates were not published on NHC website till Oct 2006. It is also possible that quality report cards may not provide an additional impetus for nursing homes to offer flu vaccines. Researchers examining the impact of nursing home report cards have reported similar findings. For example, Mukamel and colleagues found that nursing homes' response to quality report cards were limited17. Nursing homes may respond to such report cards by improving some, but not necessarily all dimensions of quality. Furthermore, it is possible that by focusing on average across state changes in vaccination rates, we were not able to pick up significant changes that may have occurred in response to public reporting in some subgroups of facilities. For example, while there was a slight decline in immunizations for short-term residents in facilities with high baseline vaccination rates, we observed 12% increases in immunizations for short-term residents in facilities with low baseline rates.
We also examined the impact of influenza vaccination on hospitalization events and found mixed results. In facilities with high baseline vaccination rates, increases in immunizations for short and long-term residents were both associated with marginal reductions in hospitalizations. We found a considerably larger effect of long-term residents' vaccination rates on reducing hospitalizations in facilities with low baseline rates. However, we detected an increase in hospitalizations associated with increasing vaccination rates for short-term residents in these facilities.
We offer several reasons for these mixed findings. First, this part of our analysis is based on one state (NYS) with relatively high overall vaccination rates and small changes in vaccination rates between the two flu seasons. Therefore, there may not be much room for improvements in vaccination rates. This may explain the small effects on hospitalizations demonstrated in nursing homes with high baseline vaccination rates in NYS. Second, vaccinations in nursing homes may not help prevent hospitalization events until their rates reach a certain level. In facilities with low baseline vaccination rates, incremental increases in immunizations may not be sufficient to positively impact resident outcomes. Third, as suggested in other studies, nursing home staff also needs to be vaccinated to prevent negative events among nursing home residents4, 26. Unfortunately we did not have information about vaccination rates among staff. Finally, there may be potential benefits of vaccinations that are not captured by this study. As these residents return to the community, facility-obtained vaccinations may reduce their risk of influenza and subsequent hospitalizations. This may be particularly true with regard to short-term residents.
Several study limitations should be mentioned. First we did not have individual level information about vaccinations events and we did not have the reasons for hospitalizations, which precluded us from conducting a more in-depth analysis about the effect of vaccination on related health outcomes. Second, the analysis of the effect of vaccination on hospitalization events was based on a state with relatively high vaccination rate. Therefore, the results based on New York State may not be generalized to other states.
In conclusion, we did not find clear evidence that public reporting can improve flu vaccination rates in nursing homes. The findings about the effect of flu on preventing hospitalizations in nursing homes were mixed. Further research with more longitudinal data and information about individual-level vaccinations is warranted.
We gratefully acknowledge financial supporting from the National Institute on Aging, Grant R01 AG23077.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.