Overall distribution rates were incomplete for handout surveys (74 percent), whereas mail surveys were distributed to the entire selected sample. Distribution rates for handout surveys declined over the 3-week study period and varied by each site (). Surveys distributed by mail attained a higher total response rate (58 percent; 1,602 completed/2,745 eligible) than handout surveys (40 percent; 1,160 completed/2,903 eligible). Cooperation rates, calculated as the proportion of completes among those who received a questionnaire, were similar for surveys distributed by hand (54 percent; 1,160 completed/2,140 distributed) and mail (58 percent; 1,602 completed/2,745 distributed). Among patients who were handed a survey, only 25 (2.4 percent) returned the survey by drop box versus mailing it in. Lag time between survey distribution and receipt was an average of 8.4 days shorter for surveys distributed by hand (). This lag time difference was expected as respondents receiving surveys under the handout mode did not have to wait for their questionnaires to arrive in the mail. Survey respondents were less likely to be under 54 years of age and more likely to be female compared with nonrespondents (). Respondents for both modes of distribution were older and more likely to be female than nonrespondents, but respondent characteristics did not vary across the modes of distribution ().
Handout Distribution Rate by Office and Week
Self-Reported Respondent Characteristics by Mode of Distribution*
Respondents to surveys distributed by hand reported significantly better experiences on all but three survey items. On average, scores were 2.1 points higher among those who received the survey by handout versus mail (). In the random intercepts model, there was a significant difference between modes for 19 of the 29 items and scales, with all items comprising the Physician Community Quality composite showing significant mode effects (). In addition, the percent of respondents choosing the highest response option on items was significantly greater among handout than mail respondents, with particularly large effects observed for physician provision of clear instructions (75.2 percent handout, 66.8 percent mail; χ2=16.5, p<.001) and thoroughness of physical exams (71 percent handout, 64.3 percent mail; χ2=11.8, p<.001) ().
Survey Items and Composite Scores by Mode of Survey Distribution
Composite measures for which physicians' relative standings were most strongly correlated between modes were as follows: Physician Communication Quality (ρ=.78), Care Coordination (ρ=.68), Access to Care (ρ=.85), and Office Staff Quality (ρ=.76) (). Relative standings among physicians for the Shared Decision Making (ρ=.31) and Quality of Physical Exam (ρ=.11) composite measures were not significantly correlated between modes (). The most discordant relationship was observed for an item measuring physician provision of comfort during physical exams (ρ=−.28).
Correlations in Physician Relative Standing (i.e., “rank”) between Mail and Handout Modes
Several interaction effects between survey mode and individual physicians were statistically significant (), indicating divergent relative standing of physicians across modes. These included the following: the physical exam composite [F(14, 2210)=2.2, p<.01] as well as both of its constituent items—thoroughness of exam [F(14, 2388)=1.8, p<.05] and attention to physical comfort during the physical examination [F(14, 2210)=1.9, p<.05], and the interpersonal item measuring how often the physicians respects what the patient had to say [F(14, 2394)=1.7, p<.05]. For each significant interaction, physicians with lower scores on mail surveys showed substantially elevated scores on handout surveys compared with physicians with higher performance on mail surveys. The use of 15 physicians limited our ability to thoroughly examine interaction effects, particularly with regard to determining the confidence intervals surrounding correlations between random intercepts and slopes in the mixed effects models. However, only negative correlations were detected between intercepts and slopes. Stable confidence intervals were detected for the thoroughness of physical exam composite measure, where the correlation between the intercept and slope was −0.87 (95 percent CI −0.98 to −0.28).
Finally, an interaction effect was detected at the site level for the Access to Care composite score [F(2, 2199)=3.1, p=.04]. Here, the site with the lowest score in the mail mode showed the largest discrepancy in scores between modes.