|Home | About | Journals | Submit | Contact Us | Français|
The purpose of this study was to examine the extent to which measures of health plan clinical performance and measures of patient perceptions of care are associated with health plan organizational characteristics, including the percentage of care provided based on a group or staff model delivery system, for-profit (tax) status, and affiliation with a national managed care firm.
Data describing health plans on region, age of health plan, for-profit status, affiliation with a national managed care firm, percentage of Medicare business, total enrollment, ratio of primary care physicians to specialists, HMO penetration, and form of health care delivery system (e.g., IPA, network, mixed, staff, group) were obtained from InterStudy. Clinical performance measures for women's health screening rates, child and adolescent immunization rates, heart disease screening rates, diabetes screening rates, and smoking cessation were developed from HEDIS® data. Measures of patient perceptions of care are obtained from CAHPS® survey data submitted as Healthplan Employer Data and Information Set, Consumer Assessment of Health Plans 2.0 H.
Multivariate regression cross-sectional analysis of 272 health plans was used to evaluate the relationship of health plan characteristics with measures of clinical performance and patient perceptions of care.
The form of delivery system, measured by percent of care delivered by staff and group model systems, is significantly related (p ≤ .05) with four of the five clinical performance indices but none of the three satisfaction performance indices. Other variables significantly associated with performance were being geographically located in the Northeast, having nonprofit status, and for patient satisfaction, not being part of a larger insurance company.
These comparative results provide evidence suggesting that the type of delivery system used by health plans is related to many clinical performance measures but is not related to patient perceptions of care. These findings underscore the importance of the form of the delivery system and the need for further inquiry that examines the relationship between organizational form and performance.
Evidence of the quality gap between best practice and the current reality of everyday medical care is widely documented and acknowledged (Institute of Medicine [IOM] 2001a, b, 2002, Leatherman and McCarthy 2002; McGlynn et al. 2003). Two IOM reports, Crossing the Quality Chasm (2002) and Leadership by Example (2001a), link the defects in quality largely to system problems rather than individual errors or actions. These reports have helped to focus attention on the need to identify the characteristics that differentiate high-performing care delivery systems from those that do less well.
Studies to assess variations in care processes, costs, outcomes, and patient perceptions of care across organizations and delivery system types have not been conclusive or consistent in determining whether one type of delivery system or care delivery organization delivers higher quality or lower cost care than others (Scitovsky and McCall 1980; Nobrega et al. 1982; Miller 1992; Himmelstein et al. 1999; Miller and Luft 1994, 2002; Singh and Kalavar 2004). Shortell and Schmittdiel (2004) suggested that organized delivery systems, especially large, multispecialty practices, characterized by patient-care teams, defined patient populations, aligned financial and payment incentives, partnership between medicine and management, information technology, and accountability, have the potential to provide superior performance in terms of clinical quality and safety although they concluded that studies have yet to demonstrate superiority in the quality, efficiency, or costs of care. Casalino et al. (2003a) demonstrated that physician organizations with strong external incentives, clinical information technology, substantial health maintenance organization (HMO) penetration, a high percentage of patients with utilization management delegated to the group, and owned or affiliated with a hospital, health system or health plan used more recommended care management processes (CMPs), which have been shown to be linked to higher quality care (Wagner et al. 2001). Shortell et al. (2005) found that high performing physician organizations were significantly more likely than low-performing physician organizations to engage in formally organized quality improvement initiatives and external reporting of quality data. Chuang, Luft, and Dudley (2004) posited that health plans affiliated with group- or staff-model delivery systems deliver higher quality care than other plans because of greater integration across specialties and sites of care; decreased conflict among clinical protocols; more consistency of incentives and goals; and larger scale and more stable enrollment populations. Other studies (Levin 2001, Casalino et al. 2003b) indicated that group practices provide more recommended treatments for chronic disease and have lower mortality rates from congestive heart failure and other disease. On the other hand, a study (Baker et al. 2004) of health plans in California concluded that the impact on Healthplan Employer Data and Information Set (HEDIS®)-based quality scores could be owing to more efficient administrative and data reporting systems than to what the physicians themselves did. The study was not conclusive about the impact of the physician group on quality scores.
Similarly, existing evidence comparing patient perceptions of care in health maintenance organizations (HMOs) with those in fee-for-service settings is mixed (Miller and Luft 2002, Roohan et al. 2003; Lin, Xirasagar, and Laditka 2004). It should be noted that most existing research on patient perceptions of care does not separate HMOs by the type of delivery system used to deliver care.
In the present study, we examine whether the extent to which a health plan utilizes a staff or group model of care delivery is associated with better clinical performance and patient satisfaction. We also examine other organizational characteristics, including geographic location of the plan, affiliation with a national managed care firm, and for-profit status, that may be associated with the performance of health plans (Himmelstein et al. 1999, NCQA 2004).
The central thesis is that disease screening and prevention practices, as one set of measures of quality of care, are more likely to be carried out in more organized practices such as staff (i.e., salaried relationships) and group (i.e., contractual relationship between a health plan and an organized multispecialty practice) models. There are three reasons for this. First are shared goals that exist in the group and staff model practices and use of teams that facilitate implementation of quality improvement practices (Shortell and Schmittdiel 2004, Wagner 2004). Second is the greater presence of clinical electronic information technology and related tools to improve care and track patients over time in group and staff models (Casalino et al. 2003a, Miller, Hillman, and Given 2004a; Miller and Sim 2004b; Simon, Rundall, and Shortell 2005). These tools also increase the practice's ability to transfer knowledge from one care provider to another and increase the practice's learning capacity. Third is the existence of more aligned financial incentives for quality improvement that exist in group and staff model practices than in looser forms of practice. As a result, group and staff models have built greater capabilities than more loosely organized delivery forms to screen their patients for breast and cervical cancer, heart disease, diabetes, and immunizations; to target people who smoke for cessation programs; and to implement improvement strategies. The potential for more tightly organized practice models such as group and staff to provide higher quality has been widely described in the literature dating back to the seminal report of the Committee on the Cost of Medical Care (1932) to recent examinations of multispecialty group practices (Enthoven and Tollen 2004, 2005). However, systematic empirical evidence for this relationship is largely lacking. At the same time, past research has suggested that staff and group models may be associated with lower patient satisfaction with care, perhaps owing to more “bureaucratic” features of such organized practice models which make it more difficult for patients to access and navigate within the system (Miller and Luft 1994). Thus, if anything, one might expect a negative relationship between the prevalence of the health plans with patients receiving care from staff and group practice models and patient satisfaction.
The relationship between delivery models and selected quality performance measures may also be influenced by various demand and supply factors. Examples of such factors are education levels, income, and insurance coverage. Individuals with greater education, higher income, and more expansive insurance coverage may have greater demand for quality services and more choice of providers. Supply-side variables may also influence the relationship. These include both the number of providers in an area and composition in terms of specialty mix, level of training, support staff, and related factors.
We were able to examine these relationships by linking data from several different sources. The National Committee for Quality Assurance (NCQA) HEDIS, which includes a version of the Consumer Assessment of Health Plans (CAHPS®) patient satisfaction survey, provides comparative information on performance (NCQA 2003b). In addition, the InterStudy Competitive Edge database (InterStudy 2003) provides organizational variables for health plans. The InterStudy database included information for 454 plans, the clinical NCQA data set, 316 cases, and the patient satisfaction dataset, 307 cases. As these three datasets did not include a universal primary key, they were merged using a matching algorithm employing a fourth dataset, with 330 cases, containing “primary key” case-level identifier variables. This cross-walk between the NCQA and InterStudy data was obtained from the AHRQ Markets and Managed Care Research Center (MMCRC) at the University of California at San Francisco and the University of Michigan (Scanlon and Chernew 2004). The merged dataset contained organizational information on 410 plans. However, only 272 plans had data for both the form of delivery system and at least one performance index, either clinical and/or satisfaction, and it is these 272 plans that this study examined.
As a proxy measure of demand for quality, we examine the percentage of Medicare patients associated with the plan. While the data examined deals with the commercial population, it is possible that a greater percentage of Medicare patients associated with a given plan and its associated physicians may stimulate greater attention to various quality and patient satisfaction metrics for all patients. Although the data examined in this paper are from 2003 (the latest year for which complete data were available), we would expect this to be increasingly true in the current environment in which Medicare is experimenting with different forms of financial rewards for achieving predetermined quality targets.
We also examine a number of supply-side variables including HMO penetration, region of the country, age of the health plan, for-profit status, relationship to national office, size, and ratio of primary care physicians to specialists. HMO penetration, serving as a measure of competition, may act as a stimulus for greater quality independent of the health plan delivery form relationship. We include region of the country in the analysis in order to recognize the widely documented variations in quality and satisfaction scores across the country (Fisher et al. 2003a, b; NCQA 2004). Age of plan is included to recognize the possibility that more established plans may have greater experience in working with various types of physician delivery forms that may account for observed quality results. For-profit ownership may provide incentives or disincentives for disease prevention practices depending on whether or not the plan has sufficient stability of membership to make such investments worthwhile in the long run. Affiliation with a national managed care firm might be associated with more positive disease prevention measures as a function of greater standardization of such measures and directives for implementing such measures. Greater size in terms of enrollment may also be positively associated with more quality performance measurement. Finally, health plans working with physician practices that have a higher ratio of primary care physicians to specialists may score higher on disease prevention quality measures given the greater orientation of primary care physicians for providing such care. In sum, each of the above-noted demand- and supply-side variables represents potential competing explanations for any relationship that might be found between the percentage of health plan enrollees receiving care from staff and group models and the quality of care and patient satisfaction.
Table 1 presents descriptive information on the 272 plans and associated variables. Definitions of and more detail on these measures are provided in Appendix A. All regions of the country were represented, with the greatest percentage coming from the East North Central and the South Atlantic, and the least from the East South Central. Only a very small percentage was pure “group” (2.2 percent) or pure “staff” (0.4 percent) delivery systems, although another 19 percent were a mixed form that included either a group or staff (or both) component. About 17 percent of the health plans were pure networks. For those plans that were in part or totally group or staff, the percentage of their delivery system that was characterized as group or staff ranged from a low of one percent up to 100 percent, with the median percent equal to 49 percent. The plans were predominantly for-profit (71.3 percent) and about equally split between those affiliated with national managed care firms (54.4 percent) and not so affiliated.
The 272 plans used in the present study were not substantially different from the 138 health plans that did not have clinical and satisfaction data available in regard to their for-profit status, ratio of primary care physicians to specialists, average HMO penetration, percent of Medicare enrollment, and the form of the primary delivery system (e.g., group, staff, IPA, network, mixed), including the percent of the delivery system that is group and staff. However, they were significantly more likely than the 138 health plans without the performance data to be from the Northeast (8.5 percent versus 3.0 percent) or East North Central regions (21.3 percent versus 13.8 percent) of the U.S. as well as to be older (18.2 years versus 11.9 years), affiliated with a national managed care firm (54.4 percent versus 44.2 percent), larger total enrollment (406,254 versus 120,540), and more geographic coverage (number of metropolitan statistical areas, or MSAs, that the plan operates in, seven MSAs versus four MSAs).
The clinical and patient satisfaction data used in this study were from HEDIS and CAHPS 2.0 H 2003 datasets, respectively. Only plans that allowed public reporting (83 percent) were included and data only for non-Medicare commercial enrollees were included. The HEDIS measures selected were chosen as those that most reflected medical decision making (as opposed to administrative factors, such as claims processing speed). The HEDIS and CAHPS measures were used to create five indices of clinical quality (women's health screening rates, child and adolescent immunization rates, heart disease screening rates, diabetes screening rates, and smoking cessation) and three indices of patient perceptions of care (satisfaction with medical group, physician, and health plan). The component measures for each index and the descriptive statistics for each index are listed in Table 2 (for more detail on the indices see online Appendix B). With the exception of smoking cessation, these indices are composed of multiple measures. For indices with multiple component measures, the index was computed as the average of the standardized component measures, that is, each component measure was standardized by adjusting each observation by the mean of the measure and dividing the resulting value by the standard deviation of the measure. An index value for each observation was then computed as the average of the valid (i.e., nonmissing) component measures for that observation. In order for an index score to be computed, a health plan was required to have valid data for at least half of the component measures of that index. Plans missing data on more than half of the relevant component measures for a specific index were scored as missing on that index and thus not included in that regression analysis.
We used multivariate ordinary least squares (OLS) regression analysis (SPSS v.13.0), employing the mean substitution option for handling the few cases of missing data on the independent variables, to examine the association of the form of delivery system with the performance of the health plan on each of the eight clinical and satisfaction indices. Other approaches for handling missing data on the independent variables were also examined; only very minor differences were found and the significance patterns were not affected. Analyses examined the extent to which each of these performance indices was associated with the form of delivery system, measured as the percent of delivery system that is group and/or staff, plan region (eight regions plus the Pacific region, the referent), percent Medicare membership, age of plan, for-profit status, affiliation with a national managed care firm, total enrollment, primary care physician–specialist physician ratio, and HMO penetration.
The mean levels for each of the performance indices are presented for various categories of organizational characteristics in Table 3. The correlations between interval-level organizational characteristics and the eight performance measures are shown in Table 4. The results of the eight multivariate regression analyses are presented in Table 5. These regression results show that with the exception of smoking cessation rate, the form of delivery system is significantly related (p ≤ .05) to clinical performance, that is, the greater the extent to which a plan's delivery system is characterized as either a group or staff model, the higher that plan's performance for women's health screening, immunization rates, heart disease screening, and diabetes screening. On the other hand, the form of delivery system is significantly related to none of the three satisfaction measures.
Other factors are also significantly related with some of the performance measures. The Northeast region scores higher on all but one of the performance measures. For-profit status is frequently negatively related to performance, especially with satisfaction with the health plan. A greater percentage of Medicare patients have generally positive associations. Being affiliated with a national managed care organization is significantly and inversely related to two of the three satisfaction indices.
The results of this study provide evidence suggesting that health plans that rely more on organized physician groups or internal (staff) physician groups perform at a higher level on many clinical measures than plans without this form of delivery system. Others have documented that organized medical groups are more likely to have higher implementation of CMPs, which the other research has suggested is linked to improved quality of care (Wagner et al. 2001; Casalino et al. 2003a). The greater ability to implement CMPs may be one reason for the observed association between structure and clinical performance. Such groups are also more likely to have clinical electronic information technology that may make it easier for them to implement CMPs. On the other hand, a significant relationship between the form of delivery system and patient perceptions of care was not observed for any of three composite satisfaction measures. A summary of recent research on this issue by Chuang, Luft, and Dudley (2004) also found mixed or no consistent relationship between delivery organization form and patient satisfaction.
A potential design weakness of this study is that only plans consenting to public release of their HEDIS 2003 results were included. “Public reporters” are about 66 percent of the HMOs that report and represent about 85 percent of national HMO enrollment. Thompson et al. (2003), McCormick et al. (2002), and NCQA (2003a) have repeatedly found that “public reporters” performed at a higher level on nearly all HEDIS clinical performance and HEDIS CAHPS 2.0 H survey items than those who do not allow public reporting of their results. In the cohort studied, only 54 plans (16.7 percent of total reporting HEDIS) did not allow public reporting and were not included. Inclusion of the nonpublic reporting plans, which tend not only to be lower scoring but also to be newer, smaller and nongroup or staff (NCQA 2003a), would most likely either not impact or, actually, further enhance the results.
The argument could also be made that case-mix adjustment is needed to assess accurately the quality of care across different types of delivery systems. However, nearly all of the HEDIS clinical care measures included in this analysis are process measures constructed so as to reduce or eliminate the need for case-mix adjustment by carefully defining groups of patients for whom clinical guidelines indicate that all patients within the defined denominator should be treated. In addition, a number of the measures, like screening and immunization measures, apply regardless of health status. Published studies of the impact of case-mix adjustment on HEDIS or CAHPS results have shown a very small effect (Landon et al. 2004).
It is also important to note that the analyses presented here demonstrate a cross-sectional association between organization characteristics such as the form of the delivery system and performance, but they cannot show causality. Other unmeasured factors may be impacting both.
Finally, the measure of the form of delivery system is likely to be only a crude indicator of the degree to which care is being provided in a more organized, systems-related manner. Although staff and group models of care delivery are often more highly structured than IPA or network models, there may be some instances where this does not hold true.
These findings should be considered as exploratory, providing information for further research that is able to measure specific dimensions of delivery organization form and relate these to both process and outcome measures of care. The difference in performance suggested in this study and the plausible explanation for these results offered by Chuang, Luft, and Dudley (2004), Shortell and Schmittdiel (2004), and Wagner et al. (2001) (e.g., shared goals and use of teams, aligned financial incentives, greater information technology capabilities, and greater use of organized CMPs) can provide a model for building systems of care that are designed to meet the complex needs of large populations of patients. If further research validates the current findings and establishes relationships with clinical outcomes of care, policy makers and payors would have a growing evidence base needed to encourage movement toward more organized delivery systems and to avoid policies that promote greater fragmentation. Public reporting of performance scores by type of delivery organization will also give consumers the information needed to make more informed choices. Given the growing prevalence of chronic illness, developing greater knowledge of the impact of different types of delivery systems on patients is of great importance.
At the very least, these differences suggest that policy makers and funding agencies should avoid public or private policies that would further fragment the financing and delivery system and instead should encourage more organized models and other approaches that demonstrate higher performance. Further study of the organizational variables related to superior performance and the implementation of those mechanisms in other delivery systems might offer the potential for improved performance at all levels of the health care system in this country.
The following supplementary material for this article is available online:
Performance Measures Definitionsn.
The authors would like to thank Dennis P. Scanlon and Michael E. Chernew for allowing the use of their “crosswalk” dataset as well as providing insights regarding the InterStudy data. The authors would also like to thank Jay Crosson, M.D., Sarah Shih, Jack Ebeler, Michael Mustille, M.D., Dan Styf, Walter Meyers, Oliver Goldsmith, M.D., and Nancy Taylor for comments on earlier drafts of the manuscript. Dr. Gillies and Dr. Shortell were supported by Robert Wood Johnson Foundation grant # 038690.
|Variable||Definition [t.=table, v.=variable]||N†|
|Region||Census Bureau region for the primary service state (t. INFO132, v. REGION)||272|
|Northeast, N (%)||CT ME MA NH RI VT|
|Mid-Atlantic, N (%)||NJ NY PA|
|South Atlantic, N (%)||DE DC GA FL MD NC PR SC VA WV|
|East South Central, N (%)||AL KY MS TN|
|West South Central, N (%)||AR LA OK TX|
|East North Central, N (%)||IL IN MI OH WI|
|West North Central, N (%)||IA KS MN MO NE ND SD|
|Mountain, N (%)||AZ CO ID MT NV NM UT WY|
|Pacific, N (%)‡||AK CA GU HI OR WA|
|Age of health plan, mean (SD)||Age of plan, or years in operation, as of January 1, 2003 (in years) (t. INFO132, v. AGE132)||272|
|Affiliation with national managed care firm, N (%)||Whether a health plan is affiliated with a national managed care firm (t. INFO132, v. CLASS). If class type is 1 or 2, affiliation is 1, if class type 3 or 4, affiliation is 0||272|
|Medicare, mean % (SD)||Medicare enrollment as of January 1, 2003 (t. INFO132, v. ECARE103) divided by total HMO enrollment as of January 1, 2003 (t. INFO132, v. ETOTL103)||272|
|Total enrollment, mean (SD)||Total HMO enrollment as of January 1, 2003 (t. INFO132, v. ETOTL103)||271|
|For-profit status, N (%)||Tax status of the HMO (t. INFO132, v. TAXSTAT) where 1 is for-profit, 0 is not-for-profit||272|
|PCP/specialist ratio, mean (SD)||Primary care physicians contracted with as of July 1, 2002 (t. INFO132, v. PCPS702) divided by speciality care physicians contracted with as of July 1, 2002 (t. INFO132, v. SPEC702)||267|
|HMO penetration, mean (SD)||January 1, 2003 estimated HMO penetration in MSA averaged over all MSAs HMO operated in (t. MSASUB 132, v. PEN)||271|
|Number of MSAs in which HMO operates‡||Count of the MSAs an HMO operates in (t. MSASUB 132, v. MSACODE)||271|
|Primary delivery system‡||Model type of HMO, defined by the relationship between the health plan and its providers (t. INFO132, v. MODEL)||272|
|Group, N (%)||An HMO that contracts with one independent group practice to provide health services|
|IPA, N (%)||An HMO that contracts directly with physicians in independent practices and/or an HMO that contracts with one or more multispecialty group practices, but is predominantly organized around solo/single specialty practices|
|Network, N (%)||An HMO that contracts with two or more independent group practices, possibly including a staff group, to provide health services|
|Staff, N (%)||An HMO that delivers health services through a physician group that is employed by the HMO|
|Mixed, including group and/or staff, N (%)||An HMO that uses a combination of the contracting model types that includes either group and/or staff|
|Mixed, other not including either group or staff, N (%)||An HMO that uses a combination of the contracting model types that does not include either group or staff|
|Percentage group and staff, mean (SD)||The extent to which health plan utilizes a staff or group model of care delivery. If a health plan is either “pure” group model or staff model (see above, t. INFO132, v. MODEL), percentage group and staff equals 100%. If health plan is a mixed model, percentage group and staff is the sum of the mixed enrollment that is group (t. INFO132, v. MIXGRPP) and the mixed enrollment that is staff (t. INFO132, v. MIXSTAFP)||272|
Citation: InterStudy, The InterStudy Competitive Edge 13.2: The CE 132 Database, St. Paul MN 55104, 2003