To our knowledge, this is the first study to compare the reliability of time-based and activity-based sun exposure questionnaires whether self-administered (as in this case) or interviewer administered. The retest questionnaires were sent 4–9 months after the baseline (test) questionnaire. The activity-based questionnaire produced higher estimated time spent outdoors, especially in adult years and during lifetime. We also wanted to learn whether cumulative estimates of time spent outdoors could be improved, especially for women, using an activity-based questionnaire instead of the more usual time-based questionnaire. While there was little apparent difference between the two questionnaire types in reliability of age-group specific categorical estimates of time spent outdoors on weekdays and weekend days, as assessed by kappa statistics (), the activity-based approach showed significantly higher ICCs for continuous estimates of average hours outdoors in adult years and over the lifetime than the time-based questionnaire (). This difference was more evident in females than in males and was not evident for childhood and adolescence exposure.
One concern is the large difference between the estimated numbers of hours spent outdoors based on the two questionnaire approaches. A key potential strength for the activity-based questionnaire is the opportunity to trigger respondent’s memory to recall the full spectrum of outdoor activities. This is particularly true for adult years, during which participants perform various types of outdoor activities that may have not been captured using the traditional time-based questionnaire. For instance, one activity category that is likely not captured in the time-based approach is “driving”, which has been shown to account for 39% of total time spent outdoors in our study population during the assessment of short-term sun exposure (17
). Despite the potential advantage discussed above, the activity-based approach also allows a greater chance of inaccuracies because of the number of questions asked. It is likely that the activity-based questionnaire approach gave much higher estimates in time spent outdoors than the time-based questionnaire approach due to the combination of these factors.
Other studies of reliability using time-based questionnaires have observed similar or higher reliability than the activity-based questionnaires in this study. These previous studies were all interviewer-administered (12–15), and thus could presumably have used cues to help elicit responses. In an Australian population of 62 cases with basal cell carcinoma and 162 controls (84 females and 80 males, aged 40–64 years at the first interview), English et al. reported an ICC of 0.77 for lifetime hours spent outdoors using face-to-face interviews 5 years apart. In another study in Australia of 52 cases with multiple sclerosis and 52 controls (69 females and 35 males, mean age = 43.6 for cases and 44.7 for controls) who were interviewed 11 weeks apart, the authors reported weighted kappa values ranging from 0.43 to 0.74, according to four age intervals and two seasons and disease status, using face-to-face interviews (15
). In a multi-center European study of 115 cases of basal cell and squamous cell carcinoma and 119 controls (106 females and 130 males, mean age = 61.2 for cases and 62.4 for controls), Rosso et al. obtained ICC values of 0.68, 0.79, and 0.56 for weighted outdoor hours in a lifetime for work, vacation, and sports, respectively, using face-to-face interviews 18–26 months apart (14
For the current investigation, the test-retest reliability for the time-based questionnaire was lower in females than males (). Some, but not all, studies have also reported gender differences in time spent outdoors and types of outdoor activities (16
). Compared with males, estimating lifetime cumulative sun exposure in females during adult years has been reported to be particularly difficult, possibly because of females’ frequently combined work and family roles (13). Kricker et al, evaluating reliability of a shortened telephone questionnaire against its longer face-to-face version 9–20 months apart in an Australian population of 15 ocular melanoma cases and 45 controls (34 males and 26 females), also reported lower reliability of cumulative sun exposure among females (ICC = 0.54) than in males (ICC = 0.73) (13
). The same study also reported that females had lower reliability of working day sun exposure than males (ICC = 0.48 for females and 0.71 for males), whereas females had higher reliability of nonworking day exposure than males (ICC = 0.67 for females and 0.53 for males) (13
). As a second component of their study, Kricker et al compared the correlations between occupational exposure and total exposure (occupational + recreational) in males and females. The correlation was lower in females than in males (R
= 0.54 and 0.78, respectively) (13
). In contrast, other studies reported no differences in reliability by gender but did not give gender-specific estimates of reliability (12
). Lastly, in the USRT cohort, using self-administered, mailed questionnaires, test-retest responses to the time-based questionnaire approach was evaluated in a previous study of 481 U.S. radiologic technologists (ages 44 to 80 years old, including 238 males and 243 females from throughout the U.S.) who volunteered to complete the same questionnaire about lifetime UVR solar exposures at two times approximately 9 months apart. In this larger nationwide population, reliability of responses about lifetime sun exposure was fair to good for both women (ICC = 0.62, 95% CI 0.53–0.70) and men (ICC = 0.53, 95% CI 0.42–0.62) (Thomas Fears, personal communication). These mixed findings suggest that the potential gender difference in test-retest reliability is still an evolving area. Additional studies are needed to examine the potential gender differences in the reliability of cumulative sun exposure, by properly accounting for various types of activities.
Our study has several strengths, including high participation rates, a wide range of ages, substantial numbers of participants in both genders, inclusion of participants from two very different latitudes, and systematic assessment of sun exposure at specified intervals. For the activity-based approach, we provided a comprehensive list of activities in our questionnaire to potentially improve participants’ recall.
The study also has some limitations. Most importantly, when answering the activity-based questionnaire, participants over-estimated the total hours spent outdoors when the number of hours spent in a given activity were summed, raising questions about the validity of the activity-based questionnaire. The degree of over-estimation (compared to the maximum number of hours of possible response) was lower during weekdays (particularly for females), and higher on weekend days. Part of the overestimation may have been due to study participants recalling their time outdoors during 9 A.M. to 5 P.M., which corresponds more closely to the working day, rather than the 9 A.M. to 3 P.M. time frame asked in the questionnaires. The over-reporting on the activity-based questionnaire may have contributed to the higher number of total outdoor hours recorded using the activity-based questionnaire ().
We cannot completely rule out chance as an alternative explanation for the higher ICC in the activity-based questionnaire, because of the multiple comparisons made. Nonetheless, given the body of literature indicating gender differences in outdoor activity patterns, we consider that the activity-based questionnaire approach provides a promising new direction to improve reliability in the questionnaire assessment of sun exposure. This position is supported by the fact that the activity-based questionnaire showed the greatest improvement in women’s recall of their outdoor hours, which is consonant with the basis for which it was designed.
In conclusion, a newly-developed activity-based questionnaire, which asks detailed contribution from each activity category, produced an increase in overall reliability of sun exposure in adult years and over the lifetime, in comparison with a time-based questionnaire, which was greater in women than men. There is, however, uncertainty about the validity of the activity-based questionnaire in light of many participants estimating daily outdoor hours that were greater than the specified time window during the day. Research is needed to improve the accuracy of the activity-based questionnaire while retaining its high reliability, for instance, by alerting participants when the total hours across activities exceed the maximum.