Our results reveal that the CT feedback was evaluated positively by all respondents, and evaluated the sunscreen CT feedback as relevant, credible, nicely arranged, instructive, complete, felt that the advices matched their answers, helpful, and that the advices stimulated them to improve sunscreen use. The overall score on a scale from 1 (“very bad”) to 10 (“excellent”) was 7.8 indicating a positive evaluation despite these comments made. Suggestions concerning improvements pertained to the fact that sometimes information could be more elaborate; but also that the advices were quite long, and sometimes confusing. In-depth process evaluations are therefore needed to find out which particular program components needed to be improved; and also how to deal with the conflict of adding information on the one hand, and to shorten overall message length on the other [12
]. The positive evaluation results are congruent with those reported earlier concerning other health behaviors [8
]. One potential reason for these positive findings may be that program development was based on a combination of approaches implying the utilization of a broad, social cognitive model and principles of communication theories [33
] and social marketing principles [35
], implying that messages were developed and piloted in the target group. Hence, as has been noted before, it is important to acknowledge that besides these positive evaluations, computer technology-based interventions have many advantages when compared to human-delivered interventions. These include lower cost to deliver, greater intervention fidelity, and greater flexibility in dissemination channels, which might include in person (for example, clinic setting), mail, Internet, cell phones, or other delivery channels [36
]. CT online interventions utilize a great variety of options for assessing individuals, creating and delivering customized health messages, equipping individuals with the tools necessary to maintain or change their behaviors, and keeping them engaged in their own self-care [5
]. Our results support that this strategy is appreciated.
An encouraging finding was that fact that, in contrast to our expectations, respondents of a low level of education evaluated the CT feedback more positively than those with a high level of education, although the latter group was also positive in their evaluation. We would not have been surprised to have seen an opposite result, because the CT feedback implied quite some reading—given the concerns expressed by others concerning Internet use by lower educated groups [15
]. Yet, the pilots of the program were aimed to identify passages that were difficult and unattractive to read, certainly for respondents with a low level of education. Hence, our results of the program evaluation suggest that this goal has been attained. In our other programs about smoking cessation, physical activity, and nutrition, we also did not find differences in evaluation between the educational groups [8
]. This suggests that CT feedback can be a very attractive and effective way to reach both low- and high-education level groups.
Patients evaluated the CT feedback slightly, but, significantly, more positively than respondents with no history, who were also very positive concerning the program evaluation. The more positive finding is most likely due to the fact that the topic is more salient to them because of their history of skin cancer. Although they may have received already more information about skin cancer and sunscreen use, they evaluated the feedback very positively. Our findings do not suggest that two separate programs for patients and non-patients are needed.
Our results need to be interpreted with a certain level of caution. First, it is not clear why the low education level group evaluated the program more positively. More qualitative research is needed to asses whether this occurred because the program provided more new information for this group and whether the language used was more adapted to this group. Yet, we did not receive complaints from the high education level group concerning too simply formulated messages. Second, it is difficult to assess how representative the online sample is for the total populations. We also had a relatively large group of highly educated respondents. Yet, our main purpose was not to obtain a representative sample, but to compare the differences in evaluation between high- and low-educated groups. Third, suggestions to shorten the messages were made by all subgroups. These suggestions are relevant to take into account, also within a context that revisits of Internet based programs have found to be quite low, also resulting in high drop-out rates in research studies up to 50% or sometimes higher [37
]. A challenge will be to find an optimal balance between length and the provision of essential feedback. Fourth, Internet based methods can use a large variety of behavior change techniques and exposure-promoting elements. In order to enhance exposure, peer and counselor support may result in a longer website visit and that email/phone contact and updates of the website result in more log-ins [3
]. Recent notions about infodemiology describe important factors related with the distribution and determinants of information in an electronic medium, such as analyzing how people search, navigate, and share information that also yield important insights into health-related behavior [40
]. Hence, it is recommended to further extend interventions with these elements. Lastly, an effect evaluation is still needed using a randomized control trial to assess ultimate behavioral effects, as well as analysis may be needed to assess potential differences in program evaluation due to seasonal changes.
The most important lesson learned from this study is that CT programs do not necessarily have to be less attractive for lower educated groups. This is encouraging, since this may imply that Internet based interventions can also reach lower educated groups; provided that these interventions are tailored towards the needs and characteristics of lower educated groups. A second lesson learned was that the results of our program evaluation did not suggest a need for the development of separate programs for patients and non-patients.