The procedures described above identified 23,756 surveys mailed between 1/1/2002 and 12/31/2005. These surveys were mailed to 17,387 consumers, with 13,311 surveyed a single time, 2713 surveyed two times, and 1363 surveyed three or more times. Mailed surveys concerned 131 providers practicing at seven facilities. The number of surveys per provider ranged from 20 to 436 (mean number of mailed surveys per provider = 181.3, median = 173).
8029 completed surveys were returned (33.8% of those mailed). Surveys were returned by 6588 consumers, with 5506 returning one survey, 828 returning two, and 254 returning three or more. Returned surveys concerned 127 providers, including 24 psychiatrists and 123 non-physician psychotherapists. The number of completed surveys per provider ranged from 5 to 186 (mean number of completed surveys per provider = 63.2, median = 56). Across the seven facilities, the number of providers per facility ranged from 7 to 24.
Unadjusted results are shown in the left portion of . The proportion responding to the mailed survey appeared higher among women, those aged 50 or more, consumers with longer enrollment in the health plan, those ensured by Medicare (vs. other insurance types), and those making return visits. Response rate appeared lower among those receiving a diagnosis of bipolar or psychotic disorder at the index visit. displays regression-based estimates of the relationship between covariates and survey response, decomposed into between-provider effects (column 2) and within-provider effects (column 5). For example: the within-provider odds ratio associated with gender estimates the relative odds of survey response for women versus men while adjusting for other consumer and provider characteristics, while the between-provider odds ratio associated with gender estimates the relative odds of survey response for individuals seen by a hypothetical provider who treats only women relative to another who treats only men. The means and standard deviations in columns 3 and 4 indicate the level of variability between providers. For example, the average proportion of consumers aged 50 or older was 31% for all providers with a standard deviation of 8%.
Unadjusted results for response rate and proportion of returned surveys with an Excellent satisfaction rating
Adjusted predictors of survey response based on a logistic regression model estimated by generalized estimating equations to account for clustering of consumers within providers and providers within facilities.
Between providers, having a higher proportion of return visitors was significantly associated with higher response rates (OR=1.99, p<0.05). No other between-provider effects reached statistical significance.
Within providers, returning a survey was significantly associated with female sex, older age, longer enrollment in the health plan, being a return visitor, and insurance through Medicare.
Among returned surveys with valid responses, 49.9% gave an “Excellent” rating regarding “How well this practitioner understood your concerns”. Unadjusted results are shown in the right portion of . The proportion of consumers giving an Excellent rating appeared higher among women, those aged 50 or older, those with longer enrollment in the health plan, those insured by Medicare, and those making a return visit. displays regression-based estimates of the relationship between covariates and an “Excellent” response, decomposed into between-provider effects (column 2) and within-provider effects (column 6).
Adjusted predictors of excellent satisfaction rating based on a hierarchical logistic regression model accounting for clustering of consumers within providers and providers within facilities.
Between providers, a higher proportion of return visitors was significantly associated with higher satisfaction ratings. Again, none of the other between-provider effects were significantly associated with consumer satisfaction.
Within the practice of any provider, higher satisfaction ratings were significantly associated with female sex, older age, longer enrollment in the health plan, and being a return visitor.
illustrates how adjustment affects comparison of average satisfaction ratings across providers. These analyses are restricted to 122 providers with distinct estimates of provider and facility random effects. Adjusting for sample size accounted for a moderate portion of the observed variability between providers (i.e. in the top section, lines shrink significantly toward the mean value). Adjusting for case-mix had a modest effect and changed the relative position of some providers (i.e. some lines cross in the middle section). Adjusting for facility differences has minimal effect on either between-provider variation the position of individual providers. Qualitative impressions are consistent with estimated random effect variance terms: patient standard deviation was large (sd=2.25, 95% CI=1.94, 2.57); provider standard deviation was moderate (sd=0.66, 95% CI =0.08, 0.83); and facility standard deviation was small (sd=0.18 95% CI=0.004, 0.47).
Estimated excellent ratings for each provider based on a hierarchical logistic regression model that includes consumer, provider, and site-level random effects.