In any clinical research involving questionnaire data, problems may result from incomplete or illegible questionnaires, as well as costs for data entry and for printing and distributing a paper questionnaire. In addition, the data entry process can introduce error. Electronic questionnaires ^6–10 give the investigator immediate and direct access to data entered by the subject, thus decreasing costs, permitting more rapid analysis of collected data, and, in theory, improving the accuracy of collected data. A web-based approach is appealing because, in addition to the advantages of electronic questionnaires, web-based questionnaire administration also allows data entry from any convenient internet-accessible location and at any time convenient to the respondent. This has obvious applications in multi-center clinical trials and epidemiological studies.

This was a randomized crossover study. Order of questionnaire completion was randomly assigned (through opaque randomization envelopes). If paper administration was the first mode assigned, the questionnaire was mailed to the participant. If the WBQ was the first mode assigned, the woman was provided with a link to the web site, a unique user name, and a unique password. Women were asked to complete the first questionnaire as soon as possible. The second questionnaire was completed upon arrival at the clinical site. The interval between completion of the first and second questionnaire varied, but was between 2 days and 6 weeks for all participants.

The WBQ and paper questionnaire were identical in content but there were three differences in format. First, subjects were not permitted to skip a question in the WBQ, although subjects could return to view and alter prior responses if desired. Second, the WBQ presented one item at a time. Finally, a branching feature was built in to the WBQ. Regarding the branching feature, women who did not endorse a symptom on the PFDI-20 were not asked to rate the severity of that symptom. In addition, participants who did not endorse any items for an individual scale of the PFDI-20 (e.g., the Urinary Distress Inventory-6) did not complete the corresponding scale of the PFIQ-7 (e.g., the Urinary Impact Questionnaire-7) and were assigned a score of 0 for this PFIQ-7 scale.

After submitting the second questionnaire, each participant completed a preference questionnaire. We surveyed women regarding their preference for mode of administration and we recorded descriptive information about computer and internet use for each subject. Additional demographic information was also obtained for each participant.

Of the 52 women who participated in this research, 24 were assigned to complete the WBQ first and 28 were assigned to complete the paper questionnaire first (table I). The completion rates were similar for the two modes of administration: 92% for the WBQ and 90% for the paper questionnaire (p=.94). Among the 43 women who completed both versions of the questionnaire, the mean age was 52±13 years. The majority was White (73%), had a computer at home (95%), and used the Internet (93%).

Comparing the summary scores and individual scale scores between the two modes of administration (table II), we found no significant differences for the summary scores or any individual scales of the PFDI-20 or PFIQ-7. Post-hoc power analysis revealed that these data provide 80% power to exclude a difference in mean PFDI score of ±10 points. Also, we noted a high degree of correspondence between scores with the two modes of administration. Specifically, the ICC was 0.91 for the PFDI-20 summary score, 0.81 for the PFIQ-7 summary score, and 0.66 to 0.88 for each of the scales (p<.001 for each). No effect from the order of administration was detected.

There was no missing data from the WBQ because subjects were not allowed to skip items in this format. For the paper questionnaires, 2.8% of items were left blank or were illegible. Ten of the 47 paper questionnaires (21%) included at least one item that was left blank or was illegible. There was no discernible pattern of items left blank.

In research involving sensitive or embarrassing information, self- administered paper and pencil questionnaires or electronic questionnaires are preferred to direct interviews. Interviews are less sensitive than self-administered questionnaires because subjects are less likely to volunteer potentially-embarrassing information.^{13, 14} Moreover, when comparing modes of self-administration, prior research has shown that electronic questionnaires may be more sensitive than paper questionnaires,^{12, 14} possibly because patients respond to the apparent “legitimacy” of the format¹⁴ or because electronic questionnaires are perceived by research subjects as more anonymous.¹³ In this study, we did not find that the WBQ was more sensitive than the paper-and-pencil questionnaire. Scores and subscale scores were not significantly different between modes of administration. This is reassuring, since researchers can conclude that scores obtained through one mode of administration are equivalent to scores obtained with the other mode.

Electronic questionnaires have numerous benefits to researchers. These include the ability to incorporate complex branching patterns, reduction of errors (including multiple responses to single-option items), and elimination of the need for separate data entry.^11–13 While branching patterns can be incorporated into both paper questionnaires and electronic questionnaires, complex skip patterns or extensive customization may create confusion, leading to unnecessary responses to some items and/or incomplete responses to other items. Also, electronic questionnaires can be designed with automated date and time stamps,¹⁵ allowing accurate representation of when timed questionnaires or diaries are completed. The on-screen format can be designed to present questions individually, minimizing the impact of questionnaire length on subject response. We found that 21% of paper questionnaires had at least one missing or uninterpretable item, with less than 3% of items missing overall. Other researchers have reported 5–10% of missing items on self-administered paper questionnaires¹⁶ and an increasing proportion of missing items with increasing questionnaire length.¹⁷ Because subjects were not allowed to skip items on the WBQ, this format eliminated the problem of missing data for this mode of administration. With an electronic questionnaire, the problem of missing data can be eliminated if the electronic format either does not permit subjects to skip individual items or if skipped items are presented a second time at questionnaire completion.⁸ For research on sensitive topics, participants should be given an option to “decline” any specific item if skipped items are not allowed by the questionnaire format.

We speculated that age, education and computer experience might impact questionnaire results, but we were not able to identify any significant differences in scores or subscale scores. The relatively high level of educational attainment in this population may have limited our power to detect differences. However, other researchers have found similar results. Specifically, age, level of formal education, and familiarity with computers do not seem to significantly influence the accuracy of electronic questionnaire results.^{8, 11, 13}

A limitation of our research is the sample size. This may have limited the power of this study to detect differences in some of the scores considered. Also, the women who agreed to participate in this research may not reflect the population of women typically presenting for clinical care. Nevertheless, they are likely to reflect populations that participate in research and therefore we believe our results are generalizable to subjects enrolled in clinical research. Finally, the interval between completion of the first and second versions of the questionnaire may have been up to 6 weeks. It’s possible that symptoms may have changed over time. However, since the PFDI-20 and PFIQ-7 both specifically address symptoms “over the last 3 months”,⁵ we don’t believe an interval of 6 weeks would have a substantial impact on participants’ responses.