The results suggest that using stamped return envelopes rather than pre-paid replies does not significantly change the response rate to reminders for patient surveys. However, this study had 80% power to detect a difference of only 12.5% or more between the two groups; differences smaller than 12.5% would not be identified as significant in this study. Thus we need to treat our finding with some caution.
Edwards et al's Cochrane review found that stamped envelopes increase the odds of response by 1.29 (95% CI: 1.18–1.42) compared to pre-paid [6
]. The response to the pre-paid envelopes in the present study (26.9%) would therefore be expected to increase to 32.2% according to the results of this review. Hence, our actual result, an increase in response to 31.8%, is consistent with the existing evidence.
Although this suggests that, as with other types of surveys, stamped envelopes increase responses to patient surveys compared to pre-paid envelops, this assumption needs to be tested in further RCTs of patient surveys with sufficient participants to detect a response gain of approximately 5% or more at 80% power. However, this would require large sample sizes. For example, the present study would have needed 1305 participants in each group to detect a difference of 5% (27%–32%).
An alternative approach would be undertake meta-analyses combining the results of smaller studies of patient returns to mail surveys, using similar methodology to the Cochrane review [6
]. However, this review contained only one small study (n = 138) of a patient survey, therefore more trials of returns to patient surveys need to be undertaken before this sub analysis is viable. The lack of available evidence concerning strategies to improve returns to patient surveys has been highlighted in a recent review. Nakash et al combine 15 trials of strategies to improve responses to patient surveys and demonstrate that shorter questionnaires and more intensive follow up strategies can improve response [10
]. However, no trials of stamped versus prepaid envelopes were identified and the authors stress the need to test other such strategies specifically within patient populations.
Our study suggests that the stamped envelope strategy is cheaper per returned item compared to pre-paid returns if set-up costs are included. Previous evidence suggests that the use of pre-paid envelopes is usually more expensive per item returned because of the additional return yielded by first class post [12
] under these circumstances. However, if we omit the set up costs of the licensing fee on the basis that this is a fixed cost for an institution, the pre-paid envelope strategy is cheaper per item returned. This suggests that in an organisation undertaking a large number of surveys covered by one pre-prepaid licence, using a pre-paid system of survey returns may provide the cheaper option compared to using stamps.
Only women aged ≥65 years where eligible to receive the survey in this study. As response rates to mail surveys differ by both age and gender [3
] the generalisability of the results to younger people and men may be limited. Differential response rates to return post methods may also exist between socio demographic groups. These could not be investigated in our study as we surveyed only older women. However, Shiono et al, found a significant difference for men (+5.9%) but not women [19
]. Conversely, Harrison et al, found no difference in response rates to stamps versus pre-paid by age or gender [20
]. Further studies of responses to these postal methods between various socio-demographic groups are needed.
This study investigated responses to a second mailing, primarily due to methodological constraints of randomising at first mailing. Although this may limit the generalisability of our findings to first and final response rates, as the Cochrane review found both first and final response rates are increased with stamps compared to pre-paid envelopes, this seems unlikely [6
]. However, as this review selected studies randomised at first mailing conclusions specific to second or subsequent mailings can not be drawn. For example, the study undertaken by Shiono et al demonstrated a significantly higher returns for stamps versus pre-paid (P > 0.05) for second (39.1% versus 35.1%) but not first (52.8 versus 51%) mailings [19
]. However, as randomisation was undertaken at first mailing the impact of confounders on responses to the second mailing may have biased this result. Studies randomising at second mailing such as the present study are needed to confirm if strategies to increase response rates demonstrated for first and final response rates in the Cochrane review are also true for responses to second mailings.