Our study found that the on-line scientific writing group performed significantly better than the standard writing guidance group in terms of writing quality, also reporting greater overall satisfaction. The on-line group also reported a greater number of participant-mentor communication events in the post hoc analysis. Because this was not the primary objective of this study, it should be investigated further to understand the effects of on-line learning on collaboration and group instruction. A significantly lower error rate occurred among participants with previous scientific writing experience, but reported satisfaction was unaffected by writing experience. Inter-observer reliability for the results of this study was high. The use of Google Docs clearly enhanced participants' familiarity with an increasingly popular method of collaboration, as well as improving the mentors' efficiency.
Web-based education has become increasingly popular over the past decade. Multiple studies have evaluated the effectiveness of Web-based and other computer-assisted teaching methods. Although they have mainly addressed distance education, [23
] broad comparisons to this study are possible, as distance education often includes Web-based or other computer-assisted education. Significant advantages of on-line teaching methods over traditional classroom methods have been previously demonstrated, [28
] on-line instruction producing enhanced performance, [30
] cognitive gains and higher satisfaction, [32
] and improved test scores. [31
] For example, in one study [31
] students who were provided with Web-based educational materials obtained higher scores than those who were not provided with similar materials. It has also been shown that computer environments are conducive to the presentation of visual material that tends to benefit students. [29
] Nurses have shown a greater willingness to adopt Web-based methodologies, [37
] and they consider the benefits to far outweigh their disadvantages.48
Although studies favoring traditional teaching over on-line methods do exist, [23
] studies yielding neutral results are more prevalent. A comparison of the two methods in graduate-level courses in learning disabilities and other curricula evaluated quality, test scores, cognitive gains, and student performance, success and satisfaction, finding no significant differences between distance education using on-line materials and traditional classroom-based instruction. [10
] Since on-line learning is becoming increasingly prevalent due to its easy scalability and flexibility in scheduling, even despite its current drawbacks, on-line teaching methods will likely improve with the emergence of new Web-based technologies and thus need constant re-evaluation.
Student satisfaction is an important consideration when implementing new teaching methods and the present study finds greater satisfaction with on-line workshops. Carr, [39
] however, found that student satisfaction was lower in Web-based distance education despite higher rates of success. The author attributed this result to technical difficulties, including problems with internet connection and computer problems requiring the assistance of engineers, which ultimately resulted in greater time expenditure by students in the distance learning environment. Finally, on-line courses can also automate processes such as test score collation, saving time by reducing faculty workload and thus providing students with faster feedback. [40
Student-mentor communication was higher in the Web-based group, indicating a more open atmosphere for comments and criticism. Earlier studies evaluating students' reactions toward distance education have documented feelings of loneliness and frustration, and have cautioned against a shift away from traditional modes of group learning. [41
] However, this trend is likely to change as students that have grown up surrounded by Web applications as part of their life enter distance learning programs. These students are more likely to offer critical suggestions, facilitating beneficial changes to the curriculum. [42
Mixed results for performance advantages and decision making have been noted on a variety of tasks in computer mediated versus face-to-face learning environments. [44
] Contributors to these mixed results included factors such as participants' prior experience with on-line courses, their grades, [34
] computer competency, [45
] and idiosyncratic interactions with a given system of instruction as well as instructor skills, [46
] the presence or absence of supervision, [47
] and the relatively slow diffusion of Web-based technology as described by Karl Pajo and Catherine Wallace. [25
] Web-based learning is a maturing technology, and early studies may have suffered from poor program design and students' lack of familiarity with on-line environments, as shown in a study by Spooner. [29
] This study analyzed student response to the two types of instruction, and, interestingly, results for one parameter (class organization) were contradictory: Traditional methods were rated superior in one of the two courses studied, while distance education was favored in the other.
The use of Blackboard™ http://www.blackboard.com/
, a Web-based learning system, has been proven effective in helping students write research papers. [48
] However, in this system, access is limited to registered candidates or institutions, preventing collaboration with outside researchers who might be working on the same project. Also, while the system is useful for tracking students' use of course materials and monitoring their progress, it is not open-source and can be costly to implement. The present study used Google Docs http://docs.google.com/
, which is a freely available interface for sharing, editing, and tracking on-line documents. We are not aware of any existing previous study making use of Google Docs for the purposes of scientific writing. Scientific writing demands collaboration, in the form of back-and-forth communication with collaborators, peers, mentors and outside researchers. Given this requirement, Google Docs is an excellent solution that precludes the time-consuming compiling of multiple iterations of text that occurs with the exchange of documents via email or hard copies.
Despite significant advances in relation to the previous literature in the field, our study has limitations. First, the number of potential confounding factors we could track was limited, and did not include subjects' prior participation in on-line courses, their grades, [34
] proficiency with computers, [45
] and interaction with the system of instruction, as well as instructor skills [46
] and the presence or absence of supervision. [47
] Examining these factors was beyond the scope of this study. Second, our mentors were not blinded, which may have resulted in bias. Because it was not possible to maintain blinding in the study's early stages, we ensured that the final statistical analysis was blinded for an unbiased interpretation of the results. Third, our study did not evaluate participants' perceptions of their own performance. Several earlier studies have demonstrated that, although the two methods are comparable in terms of performance, [[9
], and [49
]] the same consistency is not seen in the participants' perceptions of their own performance. [50
] While the present study did not include a parameter for self-assessment, reports of satisfaction provide a rough correspondence. Last, our current study used the SSQS scale for measuring manuscript quality without this scale having a formal cross-cultural validation. As we did not have a large enough sample to conduct a stratified analysis, it is unclear whether language might have affected our outcomes.
Future studies should address the problems associated with using imperfect measurement scales such as SSQS, which is prone to subjective bias, and tools should be developed to objectively measure writing quality. The lack of tools to objectively evaluate manuscript quality and participants' self-assessment makes it difficult to interpret the results of this type of study. Thus, developing tools that consistently evaluate participants' perceptions of their own performance, as well as objective measurements of the quality of scientific writing, could prove beneficial. Several other factors are also found to influence the outcome of this type of study, including participants' previous experience with on-line courses, their grades, [34
] computer competency, [45
] and interaction with the system in question, as well as instructor skill [46
] and the presence or absence of supervision. [47
] Future studies should enroll participants in sufficient numbers to stratify comparison groups according to these factors.