Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Prim Care Community Health. Author manuscript; available in PMC 2012 December 11.
Published in final edited form as:
PMCID: PMC3519391

Organizational and Environmental Correlates to Preventive Quality of Care in US Rural Health Clinics

Abiy Agiro, PhD,1 Thomas T. H. Wan, PhD,2 and Judith Ortiz, PhD2



To identify organizational and environmental correlates to rural health clinics’ preventive quality of care in the United States.


A retrospective observational cohort study design was applied under Donabedian’s Structure-Process-Outcome framework. Three structure measures of care (proportion of nonphysicians, absence of physicians, and provider affiliation) and three measures of process (total clinical visits, prevention use for congestive heart failure and diabetes) were used as explanatory variables. Five environmental correlates were included. The Centers for Medicare and Medicaid Services National Medicare Chronic Care Condition Data Warehouse for 2007 was used to obtain clinical data. Preventive quality of care outcomes were measured through Agency for Healthcare Research and Quality prevention quality indicators. The indicators were risk adjusted for age, sex, race, severity, and comorbidity of patients.


Structural equation modeling with maximum likelihood estimation was used.


Provider affiliation (P = .03), absence of physicians (P = .007), and higher proportion of nonphysicians (P = .007) were negatively related to preventive quality of care. Lower cause-specific mortality rate at the county level as compared to the United States average (P = .05) and rural location (P = .001) were positively related to quality of care.


The results of the study showed the need to attract and retain physicians in rural health clinics. The positive relationship between rural location and quality of care reflects more on the limited access to hospitals in remote areas.

Keywords: quality of health care, outcome assessment, ambulatory care facilities, preventable hospitalizations, health services for the elderly


Quality of preventive care in rural areas is a significant issue and will likely become even more critical in the coming few years.1 In the United States, the Patient Protection and Affordable Care Act of 2010 promised to expand rural Medicaid and Medicare coverage to hitherto uninsured and vulnerable populations. However, not all delivery of care approaches could cope with the anticipated expansion of coverage. Consequently, identifying correlates that impede or enhance quality is of importance. This study makes a contribution by identifying organizational and environmental correlates to rural health clinics’ (RHCs) preventive quality of care.

Rural health clinics provide services for residents in areas with a shortage of health professionals and/or medically underserved areas. The RHC Services Act of 1977 defines an RHC as a primary care facility where a midlevel practitioner such as a physician assistant or nurse practitioner is available 50% or more of the time the clinic is open.2 In other words, RHCs are primarily designed to be nonphysician facilities with periodic physician oversight.3

For purposes of this study, we defined quality of preventive care as having lower preventable hospitalization rates as measured by Agency for Healthcare Research and Quality (AHRQ) prevention quality indicators (PQIs).4 Prevention quality indicators assess conditions for which good outpatient care can potentially prevent the need for hospitalization, or for which early intervention can prevent complications or more severe disease.4 Prevention quality indicators measure the outcomes of preventive care for both acute illness and chronic conditions, reflecting 2 important components of the quality of preventive care—effectiveness and timeliness.4

Several studies have explored the effectiveness or quality of care in RHCs.57 Zhang et al and Probst et al found positive relationships between the presence of RHCs in counties and county-level ambulatory care sensitive conditions (ACSC) rates.5,6 Both studies have several limitations. First, Zhang et al did not use risk adjustment. Probst et al used population-level risk adjustments. Without individual-level risk adjustments, it is hard to ascertain whether the actual provisions of outpatient care by RHCs contributed to reduced ACSC rates. Second, given the ecological focus of both studies, the impact of organizational variables on ACSC rates was not examined. Third, both studies have limited generalizeability. Zhang et al studied RHCs in Nebraska; Probst et al studied RHCs from 8 states.

Ortiz et al conducted a national survey of 402 RHCs focusing on efficiency and effectiveness.7 Using prevention of diabetes (DIAB) as a measure of effectiveness, provider-based RHCs and use of technology were found to be positively related with effectiveness. Use of disease management programs, use of interdisciplinary teams, process efficiency, participation in integrated health systems, poverty rates, percentage of elderly, and geographic location were not related to effectiveness.7

The findings of our study built on previous works as follows. First, the results of our study validated a measurement model of quality of care based on multiple PQIs. A second improvement relates to risk adjustments of PQI rates for sex, age, race, severity, and comorbidity of patients. A third improvement stems from the use of Donabedian’s Structure-Process-Outcome (SPO) quality of care framework.8 The framework incorporates organizational characteristics that could affect quality of care. The aims of this study are to identify organizational and environmental correlates to United States (US) rural health clinics’ preventive quality of care.


Study Design

The study design is a nonexperimental and correlational research design that employed structural equation modeling as a statistical procedure. Specifically, a retrospective observational cohort study design was used.

Study Period and Sampling

The study period is from January 1, 2007 to December 31, 2007. To obtain stable PQI rates, we excluded RHCs that did not have at least 1 expected or 2 observed preventable hospitalizations for 1 or more of the PQIs. Therefore, our study included 25.4% (N = 960) of all RHCs in 2007 (N = 3781 RHCs).

Data Sources

The data came from 7 secondary data sources that were assembled for a recently completed national study.9 Environmental variables concerning the location of RHCs were obtained from the Area Resource File (ARF).10 The ARF, assembled by the US Bureau of Health Professions, is a publicly available data source that provides environmental data related to health care. Cause-specific mortality rates, physicians per 1000 county population, and percentage of Medicare-eligible county population were based on US Census 2000 data. Area median income per county was based on 2004 data. Labor and patient visit data for provider-based RHCs were obtained from the Center for Medicare and Medicaid Services (CMS) Medicare cost reports for 2007. Center for Medicare and Medicaid Services Medicare cost reports are publicly available data sources that provide aggregate data on provider level use and cost measures. Labor and patient visit data for free-standing RHCs were obtained from 4 fiscal intermediaries contracted by CMS. The 4 fiscal intermediaries provide proprietary data that is similar in content to Medicare cost reports. Clinical data for RHCs were obtained from the CMS’s National Chronic Care Condition Data Warehouse (CCW) for 2007.11 As of 2005, CCW provides outpatient and inpatient data on 100% of Medicare beneficiaries (Chronic Condition Data Warehouse, version 1.4, Buccaneer Computer Systems and Services, Silver Spring, MD, 2008). The CCW is a proprietary data source with patient-level information. All analyses were conducted on de-identified patient-level data. Therefore, there are no known ethical disclosure issues.

Sample Size and Power

Using the guideline of Wan and Ullman for 15 observations for each free parameter,12,13 the proposed study model has 22 free parameters (14 structure coefficients, 4 measurement error covariances, 1 prediction error variance, and 3 covariances). Therefore, the minimum sample size needed is 22 × 15, which yielded 330 clinics. The study had 960 RHCs. Therefore, there is sufficient statistical power to conduct the study.

Statistical Analyses

Structural equation modeling (SEM) followed the theoretical and methodological guidelines elucidated by Wan and Ullman.12,13 Structural equation modeling combines a number of factor analyses with a set of multiple regression analyses. When the phenomena under study are complex and multidimensional, SEM is the analysis tool that allows complete and simultaneous tests of all hypothesized relationships.13 In addition, the measurement of quality of care will need to address both systematic and random measurement errors. Regression equations require the absence of systematic measurement errors. That assumption is often untenable. Structural equation modeling is the only procedure capable of accounting for both random and systematic measurement errors.12

Structural equation modeling was conducted using 2 key steps. First, confirmatory factor analysis (CFA) was performed on the measurement model of quality of care. The CFA establishes the construct validity of the model through goodness-of-fit indices. Next, SEM was fitted by introducing organizational and environmental variables to the CFA model. Maximum likelihood estimation was used. AMOS (version 18, IBM SPSS Inc, July 30, 2009) was used to conduct SEM. SAS (version 9.1, SAS Institute Inc, Cary, NC, USA. February 10, 2006) was used for data management and statistical analyses.

The fit of SEM models was assessed mainly through goodness-of-fit indices.13 For the power aspect of models, root mean square error of approximation (RMSEA) is taken as the best alternative. Values ≤ 0.05 are excellent. For comparison of nested models, comparative fit index (CFI) is the more desirable index. An excellent fit is indicated by values ≥ 0.95. For a focus on the variance explained by models, goodness of fit index (GFI) and adjusted goodness of fit index (AGFI) are recommended. Adjusted goodness of fit index and GFI are considered analogous to coefficient of determination (R2) in multiple regressions.13 Adjusted goodness of fit index and GFI ≥ 0.90 indicate excellent fit.

Quality of Care Measures

The unadjusted PQI rates for DIAB were computed as a ratio of RHC patients with DIAB who later received inpatient services with admission diagnosis related to diabetic sentinel conditions. Sentinel admissions to any US hospital were considered. Prevention quality indicators rates for congestive heart failure (CHF), chronic obstructive pulmonary disease (COPD), and bacterial pneumonia (PNEU) were computed similarly. The ICD9 diagnoses codes for sentinel conditions were obtained from PQI technical specifications.14

The facility-level risk-adjusted PQI rates for DIAB were computed as the difference between the total number of observed sentinel diabetic admissions and total number of expected sentinel diabetic admissions divided by the total number of expected sentinel diabetic admissions. Risk-adjusted PQI rates for CHF, COPD, and PNEU were computed in a similar manner. Multiple admissions of the same patient for the same condition were counted only once.

The risk-adjusted PQIs took into account age, sex, race, severity, and comorbidity of RHC outpatients. Severity levels were computed through software offered by AHRQ (Version 4.1 Inpatient Quality Indicators Download 3M APR-DRG Limited License Grouper, Agency for Healthcare Research and Quality December 2009). Comorbidity scores were computed through the Healthcare Cost and Utilization Project (HCUP) comorbidity software (HCUP Comorbidity Software, Version 3.6. November 9, 2010). Logistic regression was used to implement the risk adjustment.


Descriptive Analysis Results

Table 1 presents a summary of RHC characteristics. Nearly 54% of the clinics do not have physicians. Nearly 44% of RHCs were provider based (PROV). Provider-based RHCs are owned by hospitals, skilling nursing facilities, or home health agencies. PUDIAB and PUCHF were the prevention use level for DIAB and CHF, respectively. The codes for preventive services were obtained from the Healthcare Effectiveness Data and Information Set (HEDIS) reference guide, which is a publication of the National Committee for Quality Assurance.15 Attempts were made to compute similar variables for PNEU and COPD. However, a large majority of RHCs appeared to have not provided preventive services related to PNEU and COPD.

Table 1
Rural Health Clinic Characteristics (N = 960)

The Rural Urban Commuting Area (RUCA) zip code approximation of rural classifications was used to group location of RHCs (RURAL).16 Nearly 71% of RHCs were located in counties that are closer to urban-focused areas or large rural towns. About 29% of the clinics were located in counties that are closer to small rural towns or isolated rural areas.

Cause-specific mortality rate (CMR) was computed as the sum of mortality rates for the 4 leading causes of death in each county divided by the national average mortality rate for the 4 leading causes of death. Centers for Disease Control and Prevention (CDC) WISQARS online database reported that heart diseases, malignant neoplasms, cerebrovascular disease, and chronic low respiratory disease were the top 4 cause-specific mortalities for all races, all ages, and both sexes from 1999–2007.17 Cause-specific mortality rates serve as a proxy measure for health risk differences at population level.18

The average risk-adjusted PQI rates for CHF, COPD, DIAB, and PNEU were all positive values. The presence of more observed than expected preventable admissions is a proxy sign of lower quality of care. In other words, negative values of risk-adjusted rates indicate better quality of care, whereas the reverse is true for positive values.

Confirmatory Factor Analysis Results

A confirmatory factor analysis was conducted on the hypothesized measurement model of quality of care and support for it was found (RMSEA = 0.00, CFI = 1.00, GFI = 0.999, AGFI = 0.997). The model took 2 iterations to fit. Post-hoc modifications were not needed. All four risk-adjusted PQIs were significant. The model had an excellent fit, which indicates construct validity.

Structural Equation Modeling Results

Figure 1 presents the path diagram depicting the original (proposed) SPO model for RHC’s quality of care. Structure measures [Non Physicians (NONPHY), Providers (PROV), and Physicians (PHY)] and process measures [Total Utilization Visits (TOTUTN), Preventive Utilization for Diabetes (PUDIAB), Preventive Utilization for Congestive Heart Failure (PUCHF)]) were hypothesized to have direct effects on the quality of care measurement model. Environmental variables [Area Median Income (AMI), Medicare Population (MEDICARE), Physicians per 1000 population (PHYPOP), Cause-Specific Mortality Rates (CMR), and Rural Classification (RURAL)] served as control variables on RHC’s quality of care.

Figure 1
Original Structure-Process-Outcome model for rural health clinics’ quality of care.

The original or proposed model as shown in Figure 1 was fitted to raw data. All nonsignificant relationships were removed 1 step at a time. Post hoc model modifications were performed in an attempt to develop a better fitting and a more parsimonious model. On the bases of modification indices and theoretical relevance, 1 new path was introduced. RURAL and PROV were allowed to correlate. The correlation was sensible, since providers like hospitals are least likely to open RHCs in isolated areas or frontier places. The revised model attained good fit (RMSEA = 0.04, CFI = 0.98, GFI = 0.99, AGFI = 0.97).

Since the revised model was nested within the original model, a chi-square test of difference was conducted. The revised model was a substantial improvement over the original model (P < .001). Table 2 depicts further results for the revised model.

Table 2
Parameter Estimates for Revised Model of Rural Health Clinics’ Quality of Care

Since having less observed than expected preventable hospitalization is a proxy sign of higher quality of care, variables that have a negative association with the quality of care measurement model have a positive relation with quality of care. Provider-based status was negatively related to quality of care (P < .03). Absence of physicians in RHCs was negatively related to quality of care (P < .007). Rural health clinics with a higher proportion of nonphysicians were negatively related to quality of care (P < .007). Provider-based status was negatively related with rural location (P < .001). Lower cause-specific mortality rate at county level as compared to US average was positively related with RHCs’ quality of care (P < .05). Rural health clinics located in isolated rural areas or small rural towns were positively related with quality of care (P < .001). The significant variables in the revised model accounted for 11% of variation in quality of care.


All 3 structure measures in the study were negatively related to RHCs’ quality of care. The absence of physicians in RHCs was detrimental to quality of care. This result was also corroborated by the negative relationship between higher proportion of nonphysicians and quality of care. Generally, provider-based RHCs, by virtue of their placement in larger systems, were assumed to access more resources that could enable them to attain quality of care in ways not possible for independent (free-standing) RHCs.19 However, the empirical results in our study were to the contrary. Knott and Travers’s evaluation of quality assessment and performance improvement programs in RHCs indicated that provider-based RHCs lagged behind freestanding RHCs in assessment of clinical effectiveness.20 Therefore, the negative association between provider-based RHCs and quality of preventive care should not be surprising.

None of the 3 process measures in our study was related to quality of care. Since a large majority of RHCs appeared to have not provided preventive services related to pneumonia and COPD, we were not able to include prevention use variables for these conditions. A focus group discussion with RHC practitioners revealed that a large majority of preventive vaccinations for pneumonia were coded as nurse practitioner (NP) visits rather than using the relevant HEIDS codes. Many RHCs do not have the practice of reporting preventive services to COPD. The equipments needed for COPD preventive tests may not be readily available for a majority of RHCs. In any case, the secondary data sources in use in this study lend only to use aspects of process measures. Adequate measures of process require not only detailed data sources like chart reviews, but also substantial qualitative assessment of patient-provider interactions. Reflecting on the volume of SPO-related literature since the 1960s, Donabedian concedes that it is still difficult to establish the relationship between process and outcome measures.8

Environmental factors were also included in the SPO model. Population-level risk differences, as measured by cause-specific mortality rates, were positively related to RHC’s quality of care. In other words, areas with cause-specific mortality rates lower than the US average tended to have RHCs with higher quality of care (lower rates of preventable hospitalizations). The apparent positive relationship between rural location and quality of care probably reflects more on the limited access to hospitals in remote areas.

There are several key limitations in our study. First, quality of care is not the only dimension of care measured by PQIs. The dual nature of PQIs, as measures of access to care and quality of care, was indicated by the positive relationship between rural location and quality of care. Residents in remote regions have limited access to hospitals, which could lead to lower number of sentinel admissions. Lower number of sentinel admissions could then be misconstrued as “better” quality of care.

Second, a correlational research design could not rule out alternative explanations. For example, hospitalization rates for sentinel events could depend on the prevalence of the underlying chronic illnesses and the socioeconomic status of patients. In addition, case-mix adjustments of PQIs do not account for socioeconomic, genetic, and other differences among outpatients.

Third, quality of care is related to additional measures of structure and process. The use of technology (eg, electronic medical records) could have a positive impact on quality of care. In addition, organizational strategies (eg, disease management programs) could have an impact on quality of care in RHCs. The limited access to data precluded the inclusion of such variables.

Last but not least, the genearalizeability of the study will be limited. Although, the CCW data warehouse provides researchers with “100%” Medicare beneficiaries linked across the continuum of care, the claims and assessment data are primarily limited to the fee-for-service population. In addition, the CCW and other data sources used in the study face anomalies associated with large, national, administrative datasets. In particular, we were not able to generate stable PQI rates for all conditions listed by the AHRQ. For instance, hypertension was listed as one of the most common chronic conditions among rural Medicare beneficiaries.21 However, most RHCs were associated with no preventable hospitalizations related to hypertension. Therefore, this study included only 7 of the 13 PQIs related to adult populations. In order to have stable rates of DIAB, this study summed up the 4 separate AHRQ PQIs for DIAB into a single PQI for DIAB.

Future research could improve on the following areas. This study was not able to include 6 more PQIs related to hypertension, dehydration, perforated appendix, angina, urinary tract infection, and asthma. The majority of RHCs in this study had less than 1 observed preventable hospitalization for these PQIs. Zero-inflated Poisson latent variable modeling could be used to accommodate PQIs with high prevalence of zero observed preventable hospitalizations.

Figure 2
Revised Structure-Process-Outcome model for rural health clinics’ quality of care.



The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a target grant from Health Resources and Services Administration (HRSA) [R04-RH10661-01-00].



Abiy Agiro, MHS, PhD is a health outcomes research analyst with HealthCore.


Thomas TH Wan, MHS, PhD is a professor and associate dean for research at the College of Health and Public Affairs, University of Central Florida.


Judith Ortiz, MBA, PhD is a research associate at the College of Health and Public Affairs, University of Central Florida.


Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: HealthCore, the company Dr Agiro works for, was not involved in the research, authorship, and publication of this article. The research was conducted during the tenure of Dr Agiro as a presidential doctoral fellow at the University of Central Florida.


1. Mueller KJ. The Patient Protection and Affordable Care Act: A summary of provisions important to rural health care delivery. RUPRI Center for Rural Health Policy Analysis; 2010. [Accessed January 8, 2012]. Report no. 2010-4.
2. Phillips DM, Kruse J. Economic viability of a model rural family practice. Fam Med. 1995;27:194–199. [PubMed]
3. Cheh V, Thompson R. Rural Health Clinics: Improved Access at a Cost. Princeton, NJ: Mathematica Policy Research; 1997.
4. Agency for Healthcare Research and Quality. Guide to Prevention Quality Indicators: Hospital admission to Ambulatory Care Sensitive Conditions (version 3.1) Rockville, MD: US Department of Health and Human Services; 2007. Mar, [Accessed February 9, 2011].
5. Zhang WQ, Mueller KJ, Chen LW, Conway K. The role of rural health clinics in hospitalization due to ambulatory care sensitive conditions: a study in Nebraska. J Rural Health. 2006;22:220–223. [PubMed]
6. Probst JC, Laditka JN, Laditka SB. Association between community health center and rural health clinic presence and county-level hospitalization rates for ambulatory care sensitive conditions: an analysis across eight US states. BMC Health Serv Res. 2009;9:11. [PMC free article] [PubMed]
7. Ortiz JN, Meemon N, Tang CY, Wan TTH, Paek S. Rural health clinic efficiency and effectiveness: insights from a nationwide survey. J Med Syst. 2009 Nov 16; doi: 10.1007/s10916-009-9404-4. [PubMed] [Cross Ref]
8. Donabedian A. An Introduction to Quality Assurance in Health Care. New York, NY: Oxford University Press; 2003.
9. Breen GM, Ortiz J, Wan TTH, Meemon N, Paek S, Tang CY, Agiro A. Rural health clinics: contributors to efficiency and effectiveness. [Accessed February 9, 2011];Online J Rural Nurs Health Care. 2009 9:15–22.
10. Bureau of Health Professions. Area Resource File (ARF) Access System. Rockville, MD: US Department of Health and Human Services, Health Resources and Services Administration; 2007.
11. Centers for Medicare and Medicaid Services. Chronic Care Condition Data Warehouse. Baltimore, MD: US Department of Health and Human Services, Centers for Medicare and Medicaid Services; 2007.
12. Wan TTH. Evidence-based Health Care Management: Multivariate Modeling Approaches. Boston, MA: Kluwer Academic Publishers; 2002.
13. Ullman JB. Structural equation modeling. In: Tabachnick BG, Fidell LS, editors. Using Multivariate Statistics. New York, NY: HarperCollins; 2007. pp. 676–780.
14. Agency for Healthcare Research and Quality. Prevention Quality Indicators technical documentation, Version 4.2. [cited 2011 Feb 9]. Available from:
15. National Committee for Quality Assurance. HEDIS 2010 volume 2: Technical specifications. Washington, DC: National Committee for Quality Assurance; 2010.
16. WWAMI. [Accessed February 9, 2011];Rural Health Research Center: using RUCA data.
17. Centers for Disease Control and Prevention. [Accessed February 9, 2011];WISQARS leading causes of death. 1999–2007
18. Kelley MF, Linthicum L. Mortality in jails and corrections. In: Puisis M, editor. Clinical Practice in Correctional Medicine. Chicago, IL: Mosby Elsevier; 2006. pp. 401–415.
19. Edwards JB, Tudiver T. Women’s preventive screening in rural health clinics. Womens Health Issues. 2008;18:155–166. [PubMed]
20. Knott A, Travers K. Implementing quality assessment and performance improvement systems in rural health clinics: clinic and state agency responses. Minneapolis, MN: University of Minnesota, Rural Health Research Center; 2002. [Accessed February 9, 2011]. Report no. 42.
21. Dennis LK, Pallotta SL. Chronic disease in rural health. In: Loue S, Quill BE, editors. Handbook of Rural Health. New York, NY: Kluwer Academic Publishers; 2001. pp. 189–207.