Immunization rates within a health plan that implemented a robust piece-rate P4P program rose at a significantly higher rate than among health plans that did not, despite robust secular trends. The piece-rate P4P incentives did not negatively impact children with chronic conditions or exacerbate disparities. This finding is significant in light of the concern that P4P may induce providers to avoid medically or socially complex patients—those with chronic health conditions or with minority racial/ethnic backgrounds. Of note, while the piece-rate P4P program did not exacerbate existing disparities within a Medicaid population, it did not mitigate them either.
This study allowed us to examine, but did not shed significant light on, the mechanisms by which improvements were achieved. Our exploratory analyses of patient-level claims data for continuous enrollees showed that patients actually experienced no significant change in visit number, number of shots per visit, immunization rates, or preexisting disparities. The lack of a significant finding may be because we could not assess visit rates change for children who are not continuously enrolled. It is possible that gains in immunization rates were concentrated among children who recently obtained Medicaid insurance. The increasing number of combination vaccines may also mask more substantial changes in immunization adherence (e.g., a single shot can bring a child with up to three delayed immunizations into compliance).
Our study demonstrates many strengths relative to the existing literature. We observed a robust piece-rate P4P intervention with substantial administrative supports (e.g., the provision of patient lists to practices) over a 4-year period, and we obtained within-state comparison health plan quality data, thus minimizing artifacts from state-to-state variations in Medicaid eligibility, coverage, and reimbursement.
There are, however, three key study limitations. We only have aggregate data for our comparison group. Although our concerns about immunization rates being affected by variations in state Medicaid laws or practices are mitigated, we are unable to pool the intervention and comparison group data and conduct a patient-level analysis to assess potential changes in the patients or practices that may affect immunization rates within New York over the study period. Secondly, our evaluation starts at the end of the same year that the intervention begins. We were not able to obtain health plan immunization rates before 2003 because plans were being benchmarked according to different immunization guidelines (e.g., the 4 : 3 : 1 : 3 or 4 : 3 : 1 : 3 : 3 series rather than the 4 : 3 : 1 : 3 : 3 : 1 series instituted in 2003). Data from 2003 are the best baseline possible given data constraints and because large-scale interventions like this take time to implement. We were not able to measure, however, a preintervention trend and look for a deviation from prior trends. Lastly, the data used for this study were not amenable to differentiating between better documentation and more complete immunization of the target population as has been noted in previous studies (Fairbrother et al. 1999
). Differentiating between quality documentation and quality of care is an important distinction, but we have reason to believe that the overall quality of immunization documentation is more accurate and reliable than 15 years ago when previous studies noted that documentation and underlying care were dissociated. By 2003, proper immunization documentation was required and enforced to such a degree that under-documentation could be considered tantamount to poorer quality care. A short list of the substantial and sustained state and federal immunization efforts that have been implemented since the early P4P studies include the following: (a) Vaccines for Children (VFC) program; (b) the coordinated effort to update the recommended childhood immunization schedule by three main professional societies (i.e., Advisory Committee on Immunization Practices [ACIP], American Academy of Pediatrics [AAP], and American Academy of Family Physicians [AAFP]); (c) the passing of state laws requiring immunization documentation before school entry in all 50 states; and (d) advances in quality reporting within state departments of health (e.g., New York's QARR program). All of these programs require, verify, or enforce proper immunization documentation and most would agree that clinicians should base their treatment decisions on what is actually documented (i.e., if it was not documented, it was not done).
Aligning incentives with care quality is a key component of health reform (PPACA) generally, and child-oriented legislation (CHIPRA) specifically, and empirical evaluations may help identify effective payment strategies. This study adds to the existing literature that supports the effectiveness of piece-rate P4P programs but also underscores another pervasive theme: P4P alone is not a panacea due to the many patient and provider factors that affect performance. Ultimately, health plans must decide which interventions may be most cost-effective for their organization, the practices with which they contract, and their enrollees. Notably, given the modest but significant success of its P4P program, Hudson has chosen to continue in this vein and supplement it with patient-directed incentives.