|Home | About | Journals | Submit | Contact Us | Français|
Recent reports calling for change in undergraduate biology education have resulted in the redesign of many introductory biology courses. Reports on one common change to course structure, the active-learning environment, have placed an emphasis on student preparation, noting that the positive outcomes of active learning in the classroom depend greatly on how well the student prepares before class. As a possible preparatory resource, we test the efficacy of a learning module developed for the Virtual Cell Animation Collection. This module presents the concepts of meiosis in an interactive, dynamic environment that has previously been shown to facilitate learning in introductory biology students. Participants (n = 534) were enrolled in an introductory biology course and were presented the concepts of meiosis in one of two treatments: the interactive-learning module or a traditional lecture session. Analysis of student achievement shows that students who viewed the learning module as their only means of conceptual presentation scored significantly higher (d = 0.40, p < 0.001) than students who only attended a traditional lecture on the topic. Our results show the animation-based learning module effectively conveyed meiosis conceptual understanding, which suggests that it may facilitate student learning outside the classroom. Moreover, these results have implications for instructors seeking to expand their arsenal of tools for “flipping” undergraduate biology courses.
Recent reports calling for reform in undergraduate biology education (American Association for the Advancement of Science [AAAS], 2011 ; President’s Council of Advisors on Science and Technology [PCAST], 2012 ) have identified the active engagement of students in the learning process as a key factor in improving students’ conceptual understanding. Indeed, the implementation of active-learning strategies has consistently been shown to increase student achievement and concept retention in the classroom setting. The results of a recent meta-analysis (k = 225) found that science, technology, engineering, and mathematics (STEM) students in traditional classrooms had a 55% higher failure rate than those in active-learning settings (Freeman et al., 2014 ). In addition, active-learning classrooms were found to provide an improvement of almost half a standard deviation in learning outcomes (Z = 9.78, p < 0.001). In response, university instructors are increasingly redesigning courses to introduce students to content outside class, thereby freeing up in-class time for active learning (Gross et al., 2015 ; Jensen et al., 2015 ).
Recent research indicates that not all active-learning classrooms are created equal; proper preclass preparation is critical for successful implementation of active-learning strategies. For example, Andrews et al. (2011) examined active learning and student achievement at 77 institutions nationwide, yet they found no significant differences in basic introductory biology learning outcomes between classes that used active-learning strategies and those that used traditional techniques. The authors also noted that reported success in active learning could be a result of well-trained instructors effectively preparing their students before (i.e., outside) class. Similarly, “highly structured” course designs, wherein student preclass preparation requires them to interact intimately with the content outside the classroom, have demonstrated significant learning gains in active-learning classrooms (Freeman et al., 2011 ; Haak et al., 2011 ; Gross et al., 2015 ). Collectively, these findings underscore the importance of characterizing the types of out-of-class learning experiences that can provide appropriate levels of preparation for students to benefit from active-learning pedagogies in class.
If proper preparation is the key to increasing achievement in the active-learning classroom, it becomes imperative that we bridge the gap between how students are introduced to content outside the classroom and how they interact with it during face-to-face meeting times. Identifying and characterizing the diverse ways in which students learn outside the classroom will allow us to provide students with learning opportunities that provide the solid base of understanding needed to achieve the goals of in-class, active-learning activities. Instructors have commonly required students to complete textbook readings or preclass worksheets as preparatory activities (Moravec et al., 2010 ; Freeman et al., 2011 ; Haak et al., 2011 ). While the benefits of these methods are shown in a highly structured classroom setting with proper guidance from the instructor (Moravec et al., 2010 ; Freeman et al., 2011 ), it has been noted that not all students are equally motivated to read before class (Boekaerts, 2001 ; Marek and Christopher, 2011 ; Aagaard et al., 2014 ). In addition, simply assigning textbook readings without holding students accountable has been shown to likely result in poor participation rates (Vafeas, 2013 ; Aagaard et al., 2014 ). One increasingly popular alternative to textbook and writing assignments is the use of online multimedia learning resources outside the classroom (Zappe et al., 2009 ; Crampton et al., 2012 ; Pierce and Fox, 2012 ; Fung, 2015 ).
Well-developed multimedia resources provide instructors one option to allow students to process conceptual information in a short period of time (Kraidy, 2002 ; McClean et al., 2005 ; O’Day, 2010 ). By leveraging effective multimedia learning materials, instructors can provide students with effective instruction before class, thereby allowing for classroom time to be used for active-learning activities rather than traditional lecture (Gross et al., 2015 ; Jensen et al., 2015 ; DeLozier and Rhodes, 2016 ). Within the realm of molecular and cellular biology, one such collection of materials—the Virtual Cell (VCell) Animation Collection—has been widely available since 2004. These animations outline the basic introductory concepts of a variety of molecular and cellular biology topics (Reindl et al., 2015 ). Recently, these animations have been incorporated into online-learning modules that can be implemented throughout an undergraduate biology course or as a stand-alone learning tool available to students. Learning modules can augment a hybrid or flipped classroom setting by providing instructors a means of structured online content presentation that can be implemented outside the classroom. In addition, these learning modules aim to answer the call (AAAS, 2011 ; PCAST, 2012 ) to engage STEM students outside the classroom while preparing them for in-class active-learning activities.
The VCell Animation team has completed the production of two online-learning modules focused on concepts generally covered in introductory biology: biological energy flow and meiosis. One additional module has also been developed for upper-level cell biology and covers the concepts of insulin signaling. The guiding principle of this effort was to develop stand-alone learning tools that provide instructors a reliable resource to deliver biology concepts to students outside the classroom. In this study, we aimed to investigate the effectiveness of one of these learning modules (meiosis) in the introductory biology course (Biol101) at a large public university in the southeast United States. We focused our efforts on a comparison of this online-learning module with a traditional classroom lecture to determine whether the two approaches were similarly effective at reinforcing the introductory concepts of meiosis to students. Our choice of traditional lecture as a control group was based on reports that instructors in STEM fields are, on average, more resistant than non-STEM instructors to adopting flipped-class methods (Eagan et al., 2014 ; Kuiper et al., 2015 ). This study aims to answer the question “To what extent does the VCell meiosis online-learning module reinforce meiosis concepts compared with a traditional classroom lecture?” The online module in this study is designed to be a personal, self-paced, interactive-learning experience. We feel that the distinct interactive environment of the online-learning module provides an experience that cannot be accommodated in a traditional lecture setting. As a result, we hypothesized that the online-learning module would perform at a level equal to or better than that of a traditional classroom lecture. If our hypothesis was correct, we would show that students exposed to the basic concepts of meiosis through the meiosis online-learning module are at minimum equivalently prepared with the conceptual understanding of meiosis compared with if they were presented by lecture alone. In addition, we thought we would be able to provide evidence that meiosis concept presentation via the learning module is on a level at least equivalent to the traditional lecture style that some STEM instructors have been hesitant to relinquish. Empirical evidence demonstrating the efficacy of learning modules at teaching meiosis concepts might create an entry point for traditional lecturers to make a foray into active learning; instructors could assign the module before lecture, thereby freeing up time for more student-centered activities targeting resilient meiosis misconceptions.
The topic of meiosis is a common source of misunderstanding among many undergraduate introductory biology students (Brown, 1990 ; Kindfield, 1991 , 1994 ; Newman et al., 2012 ). The K–12 science framework outlined by the National Research Council (2012) suggests that, by completion of grade 12, students should have an understanding of the cell cycle, sexual reproduction, DNA replication, chromosomal structure, and genetic variability. The process of connecting these underlying concepts is a critical component of understanding the mechanisms involved in meiosis. However, evidence suggests that many undergraduate introductory biology students do not make these connections (Newman et al., 2012 ; Kalas et al., 2013 ). For example, undergraduate students commonly misrepresent chromosomes throughout the stages of meiosis, including inaccurate depictions of sister chromatids and improper interactions between chromosomes (Kindfield, 1991 , 1994 ; Dikmenli, 2010 ; Newman et al., 2012 ). Yet, even if we assume that all students enter their undergraduate studies equipped with all of the prior conceptual understanding outlined in the K–12 standards, not all instructional resources meant to help connect underlying concepts and convey deeper understanding are equally effective (Tversky et al., 2002 ). One example of this is the comparison of external representations depicting biological processes as part of instruction. When directly comparing dynamic representations with their static counterparts, one meta-analysis (Höffler and Leutner, 2007 ) shows that students who were presented information using dynamic representations of biological concepts have higher learning outcomes (d = 0.37). With these positive learning outcomes in other realms of science education, we focus on using these resources as an instructional aid for teaching meiosis as well. By developing a dynamic, interactive-learning module, our goal is to provide students with a visual guide that promotes the connection of concepts and, ultimately, a deeper understanding of the topic of meiosis.
Recent studies on the use of dynamic, animated multimedia have emphasized their ability to promote learning in the science classroom (Kozma et al., 2009 ; Cook, 2012 ; Eilam and Gilbert, 2014 ; McElhaney et al., 2015 ). It has been suggested that dynamic representations of scientific processes provide learners with cognitive assistance, allowing them to process information more efficiently and resulting in the formation of more accurate mental models (Williamson and Abraham, 1995 ; Höffler and Leutner, 2011 ). These benefits are especially evident when students are presented concepts associated with the small, nonobservable facets of molecular biology (McClean et al., 2005 ; Marbach-Ad et al., 2008 ; Jenkinson and McGill, 2012 ; Ryoo and Linn, 2012 ; Barber and Stark, 2014 ). It should be noted that not all forms of multimedia are created equal and that dynamic representations have not always been shown to be superior to their static counterparts (Tversky et al., 2002 ). Optimization of dynamic representations can be achieved through the application of multimedia design principles (Mayer et al., 2003 ; O’Day, 2006 , 2010 ) and following best practices for classroom implementation (Pierce and Fox, 2012 ; Hill et al., 2015 ).
The development of multimedia resources for use in an educational setting is not an uncommon practice in undergraduate education; however, finding empirically tested, free-to-use options can prove difficult. The current leaders in educational multimedia are well-funded textbook publishing companies. These companies typically produce resources that coordinate with the concepts presented in their publications and can be passed along to teachers for incorporation as they see fit (O’Day, 2010 ). Many of these packages are well developed and present concepts in a way that promotes learning for many students (Speckler, 2014 ), but they are limited in their accessibility. Typically, these publisher-produced resources are only made available to institutions that have adopted their textbooks and to students who have either paid for the books or have paid for access to their websites (McGraw-Hill Connect, 2015 ). While this practice can be a profitable business model, it provides little benefit to students who do not have access to these features.
In addition to publisher-driven content, a second category of educational multimedia consists of free-to-use videos and animations that are often posted to Internet sites such as YouTube or course-focused Web pages as part of a learning management system. These resources are typically produced by either the instructor or a group of students in hopes of promoting better understanding of certain concepts. While many of these productions may be effective, a large number introduce concepts inadequately (Azer, 2012 ; Raikos and Waidyasekara, 2014 ), which could potentially confuse students by introducing misconceptions.
The VCell Animation Collection addresses these concerns. The VCell team applied research-based principles of multimedia instructional design (Mayer and Moreno, 2002 ; Mayer, 2009 ) to develop a series of high-quality animations and learning modules. In addition, all of these resources are free to use and openly accessible to both teachers and students. The VCell team included an expert group of cellular and molecular biology researchers in order to ensure the validity of information within the videos, while following research-based design principles helping to maintain a low cognitive strain on the viewer (McClean et al., 2005 ; Reindl et al., 2015 ). The VCell Animation Collection currently consists of a catalogue of 24 animations depicting concepts of molecular and cellular biology. The collection is housed on the project’s website (http://vcell.ndsu.edu/animations), and each animation is readily available for either streaming or downloading. The appeal of the VCell animations to students and educators can be exemplified through those who have completed the optional registration process. Currently, there are ~21,000 registered users from more than 150 countries. In addition to the project website, the VCell Animation Collection also has a YouTube site (www.youtube.com/user/ndsuvirtualcell) that currently boasts ~35,000 subscribers and more than 12,000,000 views. The team has also developed a free Apple iOS application (http://itunes.apple.com/us/app/virtual-cell-animations/id427893931?mt=8) that has been downloaded ~175,000 times to date.
With such widespread appeal of VCell animations, the VCell development team has recently focused on using the animations as part of online-learning modules geared toward presenting difficult biological concepts to students in manner that is both effective in conveying the information and accessible in a setting independent of a lecture hall and instructor. It is the goal of these learning modules to provide effective resources that instructors can use to present concepts to students outside the classroom, thereby allowing time in class to be devoted to active learning and other teaching strategies that require students to exhibit higher-level thinking. To assess the module’s ability to effectively convey the relevant information, we had to develop the VCell learning modules using current research on module design and multimedia learning (Mayer and Moreno, 2002 ; Mayer and Pilegard, 2014 ) and rigorously tested in a classroom environment (Reindl et al., 2015 ). Details of development strategies and how this study aimed to investigate the effectiveness of the module in a classroom environment are outlined below in Experimental Procedures.
Research has shown that, with proper classroom implementation, online-learning modules can promote both greater conceptual understanding and retention, as compared with traditional methods of instruction (Stelzer et al., 2009 ; Khalil et al., 2010 ; Florida, 2012 ; Hill et al., 2015 ; Lancellotti et al., 2016 ). Development of learning modules with an attention to the cognitive load of the content presented can provide students with information in appropriately sized chunks that they can process and retain at their own pace (Khalil et al., 2005 ; Ayres and Paas, 2007 ; Hatsidimitris, 2012 ). To develop online-learning modules that effectively convey the biological concepts needed for introductory-level biology students, the VCell Animation team followed published multimedia design principles (Mayer and Moreno, 2002 ; Mayer and Pilegard, 2014 ) throughout the design process.
In accordance to the segmentation principle of multimedia learning, conceptual information presented by VCell learning modules are divided into three to four brief segments (Mayer, 2009 ). At the beginning of each segment, an onscreen narrator provides learners with a set of key points that they should focus on in a subsequent animation clip. The learners are given prompts as to what will be shown and what key concepts they should grasp from the animation (Figure 1). These prompts follow the pretraining principle by providing guidance and introducing key ideas, which should reduce the cognitive strain on learners as they progress through the segments of the learning module (Clark and Mayer, 2011 ; Mayer and Pilegard, 2014). Following the presentation of these concept prompts, each module segment has an embedded animation from the VCell Animation Collection that presents the biological concepts that are the focus of that module. The development of these animations as a part of the VCell Animation project follows a strict adherence to the seven principles of multimedia learning (Supplemental Material 1) presented by Mayer and Moreno (2002 ), adding strength to their design and aiming to increase their effectiveness. Research has also demonstrated that the integration of “thought questions” before and follow-up questions after periods of concept introduction further strengthen student learning and provide a means of formative assessment (Hegarty, 1992 , 2004 ; Weston and Barker, 2001 ; Huang, 2005 ). To address this, the VCell modules also provide a series of thought questions at the beginning of each segment (Figure 1). These questions provide further structure, focus the student’s attention on important ideas, and prompt higher-level thinking while the animations are being viewed. Following the viewing of the animation clips for a particular section, students are then asked to answer a number of follow-up questions on what they have viewed (Figure 2). Students are given immediate feedback on their answers and can be allowed (or not) to rewatch the previous animation before progressing if they feel that a review is necessary to understand the concepts. To conclude the module, students are given another group of summative questions meant to provide feedback on all of the concepts within the module. The goal of these cumulative questions is to bring together concepts presented in each segment of the module and provide feedback to help correct learner misconceptions. The experiment described below was designed to test the efficacy of the VCell learning module on the subject of meiosis.
To investigate the effectiveness of an online meiosis learning module as a stand-alone learning tool, we conducted an experiment using participants enrolled in the introductory biology course at a large public university in the southeast United States. Study participants (n = 534) self-enrolled in one of four sections of an introductory biology course (Biol101) offered in the Fall of 2015. Classroom sections were randomly assigned to one of two treatments. The “online-learning module” group (n = 131) consisted of two class sections that interacted only with the online meiosis learning module. The “traditional lecture” group (n = 403) consisted of two class sections that received instruction on meiosis in a traditional lecture setting. Traditional lecture instructors were aware of their participation in the experiment; however, they were asked to make no changes to their typical instructional style. Students assigned to the traditional lecture treatment attended classroom lecture as normal and were not given access to the learning module until the end of the experimental period. Variation in treatment group size was due to uncontrollable variability in student enrollment between course sections, ranging between 68 and 271. Such variation in course section size is common at this university, and instructors typically do not vary teaching strategies between sections.
Student conceptual understanding was assessed using instruments chosen by the research team for this project. Lengths of the assessment instruments were purposely designed to remain relatively short so as to prevent interfering with the instructor’s course design while simultaneously maximizing student participation. Student participation in both the pre- and posttest was 70% for the module group and 76% for the traditional group.
The pretest consisted of 25 questions that focused on students’ basic understanding of a variety of basic biological concepts. Ten questions focused on basic understanding of meiosis and were used to identify treatment outcomes, and five questions covered basic demographic information. In this study, we were concerned only with the 10 meiosis questions and the demographic information. The meiosis pretest assessment consisted of five questions from validated concept inventories produced by the Q4B: Questions for Biology (2015) team at the University of British Columbia (Kalas et al., 2013 ) and five additional, slightly modified questions from the Campbell Biology textbook (Reece et al., 2014 ). This textbook was chosen because it was used as the primary text for students in the introductory biology course in this study and represents a large market share of biology texts used nationwide. Questions selected for this instrument from the Q4B team correspond to numbers 2, 7, 12, 14, and 15 on the Meiosis Concept Inventory. Per the request of the Q4B project, access to these materials can be granted by contacting the team directly (Kalas et al., 2013 ). Modification was conducted to make questions more appropriate for introductory learners and consisted of removing confusing phrasing and images that were more representative of upper-level biology course concepts. To evaluate student improvement after treatment, the posttest contained the same meiosis concept questions as the pretest. Cronbach’s alpha was used as a measure of internal consistency of the assessment based on the presented sample (pretest α = 0.55; posttest α = 0.57).
While it is likely that introductory biology students have learned about the process of meiosis in high school, studies have shown that they may still harbor misconceptions (Kalas et al., 2013 ). Common misconceptions include an inability to decipher the number of DNA molecules present in a cell (Kindfield, 1991 ), misidentification of chromosomal elements and their interaction (Kindfield, 1991 ; Newman et al., 2012 ), and misunderstanding of the stages and timing of the cell cycle (Brown, 1990 ; Dikmenli, 2010 ). The assessment instrument implemented in this study directly measures student understanding related to each of these identified misconceptions.
As part of the pretest assessment, students were also asked the following question: “I learn best when information is presented in a visually stimulating (i.e., animations/video) manner.” On a five-point Likert scale, answers ranged from “strongly agree” to “strongly disagree.” Our follow-up analysis focused on students who self-identified as one of the two possible extremes, as these students are most likely sure of their personal preference to multimedia learning techniques. Additional demographic data were obtained from the university registrar (gender, ethnicity, year in school, major, and Scholastic Aptitude Test [SAT] score) and matched to student performance on the assessment instrument.
At the beginning of the semester, all participants were given the pretest designed to assess the students’ baseline understanding of the concepts to be introduced throughout the semester (Figure 3). During the ninth week of the semester, students from both treatment groups were presented the topic of meiosis in their introductory biology courses. The module group was assigned the meiosis online-learning module as an out-of-class activity that was to be completed by the students entirely through the Blackboard learning management system. After students completed the learning module, they were directed to complete the posttest that measured students’ understanding of the meiosis concepts presented. Students were not allowed to revisit the module once it was completed. Students in the traditional treatment attended classroom lecture as normal and were not given access to the learning module until after the experimental period. After classroom instruction, students in the traditional treatment immediately completed the posttest via the Blackboard learning management system.
All students in this study were assumed to have a previous exposure to the basic concepts of DNA and the cell cycle and a general introduction to the process of meiosis as part of their high school education (National Research Council, 2012 ; Kalas et al., 2013 ). Throughout the module development process, assumptions of prior conceptual understanding allowed several aspects of meiosis to be introduced only as review. As one example of this, students were assumed to have a basic understanding of terminology as it relates to meiosis. Terms were introduced throughout as a review, often accompanied with an onscreen visual and/or text. If students were familiar with the terms, they could flow seamlessly to the next aspect of instruction without wasting time on a more detailed explanation. As a result, the module is focused more on the division events of meiosis without additional extraneous information that may add to the cognitive strain placed on the student (Chandler and Sweller, 1991 ). Module development was also informed by literature on common meiosis misconceptions. For example, the development team used variation in color and size when creating their depictions of chromosomes because students have difficulty identifying homologous chromosomes (Kindfield, 1991 ; Newman et al., 2012 ). These design aspects and narration and dynamic onscreen movement allow learners to follow the progression of chromosomal separation throughout meiosis. The added layer of guidance provided by the dynamic nature of the embedded animation allows students to form more accurate mental models of the processes of meiosis (Williamson and Abraham, 1995 ). These design elements could also aid students in avoiding misconceptions associated with DNA count (Kindfield, 1991 ) and cell cycle progression (Brown, 1990 ; Dikmenli, 2010 ).
The instructors involved in the traditional lecture aspect of the research design were aware of the study being conducted and operated under the same assumptions of prior conceptual knowledge as did the module development team. Instructors did, however, have the ability to readdress any previous concepts as part of the lecture as they saw fit. Knowledge of misconceptions commonly associated with meiosis was determined by an instructor’s own understanding of the literature or learning outcomes from previous semesters. Conceptually, lectures included the same meiosis concepts as were presented in the online-learning module. This includes sexual reproduction, ploidy, chromosomal arrangement, cell cycle progression, cell division events, and resulting genetic variability. Content-delivery styles, however, did have some intrinsic differences. The meiosis learning module was developed to be an interactive, personal experience in which students observe processes as they happen on screen and then apply their knowledge to directed questions. Progression occurs on the students’ own time and they have the ability to review the material multiple times if needed. The traditional lecture group met in a large presentation hall where information was presented as part of projected PowerPoint slides accompanied by instruction from the class professor. Progression generally occurs as dictated by the instructor, and professors tend to vary in their tone and general delivery styles. In addition, student–teacher interaction varies depending on classroom dynamics and student attitude. We attempted to account for aspects by analyzing for a section effect, as described below in the Results section. Ultimately, while the concepts presented did not vary between the two experimental groups, the methods through which they were presented were indeed different.
For each aspect of student achievement, descriptive statistics were compiled and inferential analysis was run comparing treatment groups using the R statistical programming package (R Project for Statistical Computing, 2015 ). Normalized gain scores (G = (post score % − prescore %)/ (100 − prescore %)) were calculated for each student who completed all aspects of the study (Hake, 1998 ). Multiple linear regression analysis was used to investigate the effect of possible explanatory variables on normalized gain scores. In addition to linear regression, we looked at individual demographic variables and analyzed treatment results across each factor. Using independent t tests, we calculated p values comparing treatment groups and calculated 95% confidence intervals for improvement differences between treatments. Cohen’s d, a mean difference effect size, was reported when significant results were found. Two-way analysis of variance (ANOVA) was used to investigate possible interactions between treatment conditions and demographic variables.
To present our results in the most effective and representative manner, we implemented the use of bean plots as a graphic display of our data. As a variation on the more traditional box plot, bean plots provide the viewer with additional information regarding the distribution throughout the sample (Kampstra, 2008). In bean plots, the distributions are depicted by the width of the plots, with a wider plot representing a larger distribution for a specific value. In addition, the mean value of a sample is noted by a bold line within the plot itself. Specifically, results from this study were depicted using asymmetrical bean plots. This allowed for a more direct comparison of the target groups outlined earlier and a more accurate representation of the data presented as part of our results. For our figures, p values were also added above comparison groups to identify possible significance.
Analysis of pretest scores between the traditional lecture group (M = 3.69, SD = 1.72), and the module group (M = 3.48, SD = 1.48) showed low recollection of concepts relating to meiosis. Student achievement was measured using normalized gain scores calculated from pre/posttest performance for each condition. Students who interacted with the learning module showed significantly higher normalized gain scores than students in the traditional lecture group (t(317.03) = 4.42, p < 0.001, d = 0.40; Figure 4; Supplemental Material 2). Descriptive statistics relating to individual posttest items show that the learning module group had a higher percentage of students who answered correctly than the traditional group on all questions, except for one (Supplemental Material 3).
To investigate the treatment outcomes (Figure 4) across additional possible contributing variables, we analyzed our data using linear regression modeling. Creation of a predictive model for student normalized gain scores originally included the following factors: multimedia preference, class section, year in school, gender, ethnicity, pretest score, SAT total score, and treatment condition (module/traditional). Our resulting linear regression equation was
Regression analysis of Eq. 1 shows a significant contribution from factors that suggest a prior knowledge, a college-preparedness (pretest score and SAT score) component, and the treatment condition that the student received. The remaining factors examined in our model did not show significant contributions to normalized gain scores.
We therefore created a more parsimonious model (Eq. 2) by removing variables with low correlation to student normalized gain score: multimedia preference (r = 0.03), year in school (r = 0.002), gender (r = 0.03), and ethnicity (r = 0.02). Regression analysis again shows a significant contribution of treatment condition (t(412) = 3.28, p = 0.001, d = 0.32) to student achievement (Table 1).
Owing to the variability in both instructor and enrollment numbers across sections, we also used linear regression to test for a possible section effect on assessment scores across our study population. Initial analysis into section effect using Eq. 1 resulted in no significant effect of student section on normalized gain score (t(406) = −0.56, p = 0.58). Additionally, we refined our testing to account for section effect within treatment groups. Using Eq. 2, we substituted treatment condition for section within the specified condition to give us two models: one for the learning module group and one model for the traditional lecture group (Eqs. 3 and 4). Neither treatment group showed a significant effect due to the section in which students’ received their designated treatment: learning module: (t(99) = −0.21, p = 0.84); and traditional lecture: t(309) = −0.61, p = 0.54) (Tables 2 and and33).
Linear regression modeling resulted in no significant explanatory effects from the student demographic variables of multimedia preference, gender, year in school, or ethnicity. However, with a moderate coefficient of determination (R2 = 0.20), we decided to stratify student outcomes based on treatment condition across each of these factors. Additional analysis allows us to make inferences on the effects of treatment conditions within the spectrum of the individual variable, thus providing further evidence to answer the research question proposed for this study.
Students in the learning module treatment that self-identified as multimedia learners (“strongly agree” selectors) show significantly higher normalized gain score (t(127.18) = 2.63, p = 0.01, d = 0.39) when compared with the traditional lecture treatment (Figure 5; Supplemental Material 4). Additionally, self-identified non–multimedia learners (“strongly disagree” selectors) show no significant difference in normalized gain score (t(5.04) = −0.12, p = 0.91) when comparing module and traditional lecture treatments (Figure 5; Supplemental Material 4). Two-way ANOVA also shows no significant interaction between treatment condition and multimedia learning preference (F(1, 527) = 0.45, p = 0.50). However, we do note that the total number of students (n = 11) in the “strongly disagree” category could affect the generalizability of our results. This is in contradiction to the disproportionately large number of “strongly agree” students (n = 171). This dichotomy in multimedia learning preference could also explain why, despite contradictory results across learning preference, linear regression still showed no effect on assessment outcome based on this factor as a whole (t(406) = −1.41, p = 0.12).
Demographic information was used to examine module performance based on student gender, ethnicity, and year in school. Analysis of assessment performance stratified across student gender (Figure 6; Supplemental Material 5) showed significantly higher normalized gain scores by students in the module treatment group than those in the traditional group for both males (t(96.68) = 3.05, p = 0.003, d = 0.51) and females (t(215.03) = 3.39, p < 0.001, d = 0.37). Additionally, two-way ANOVA suggests no significant interaction between treatment condition and student gender (F(1, 527) = 0.40, p = 0.53).
Stratification by treatment condition as a function of student ethnicity was not possible for this study due to the disproportionate distribution in the ethnicity breakdown (white: 81%; African American: 9%; Asian: 7%; Hispanic: 2%; other: 1%). It should, however, be noted that results from linear regression above (Eq. 1) show no significant effect on normalized gain score based on student ethnicity (t(406) = 1.32, p = 0.18).
With regard to student year in school, due to the introductory status of this course, there was a disproportionally small sample number of senior-level students enrolled in the class. To account for this, we grouped class data into two categories: underclassmen (consisting of freshmen and sophomores) and upperclassmen (consisting of juniors and seniors). Analysis of student performance shows significantly higher normalized gain scores (t(281.33) = 4.51, p < 0.001, d = 0.33) for underclassmen who interacted with the online-learning module compared with those whose received instruction in the traditional lecture treatment (Figure 7; Supplemental Material 6). Results for upperclassmen show no significant difference in normalized gain score (t(24.23) = 0.35, p = 0.73) between treatment groups. Two-way ANOVA also suggests no interaction between treatment condition and student year in school (F(1, 527) = 0.04, p = 0.70). It should be noted that the total number of students constituting the upperclassman group was still small (n = 32), which could affect the generalizability of inferences pertaining to significance in the upperclassman comparisons. As was seen previously with learning preference, this small sample size could also possibly explain why, despite differences across categories, linear regression analysis showed no significant contribution of year in school to assessment scores (t(406) = 0.51, p = 0.19).
In this study we set out to investigate the effectiveness of a learning module that incorporated a meiosis animation developed by the VCell Animation Collection team. Our results show that students who were presented the concepts associated with meiosis by means of a stand-alone learning module performed significantly higher (p < 0.001, d = 0.40) on an assignment designed to assess understanding of meiosis than students who received instruction solely in a traditional lecture setting. The module implementation strategies in our experimental design allowed the learning module to be tested as a true out-of-class concept presentation that could act as preparation before a classroom meeting. The significantly higher achievement seen in students who were presented information in the learning module condition provides preliminary evidence that the learning module can adequately present concepts to students in settings other than the classroom itself.
Descriptive statistics relating to individual posttest items show that the learning module treatment group had a higher percentage of students answering correctly than the traditional lecture group for every question, except for one. This question was one of four on the assessment that addressed chromosomal structure, and the learning module group showed a higher percentage of correct responses of the remaining three questions on the concept. In addition, a relatively low percentage (<50%) of students in both groups answered questions 1 and 9 correctly. Interestingly, both of the questions seem to address the concept of DNA amount through the stages of meiosis, which has previously been shown to be a common misconception among introductory students (Kindfield, 1991 ). While we feel that the depth the assessment instrument used in this study does not allow us to make generalizable statements as they relate to specific concepts, a focus on such conceptual understand may be future direction for studies on the interactive-learning module tested here.
The experimental conditions of this study did not allow us to randomly assign individual participants to specific treatments. Therefore, we acknowledge that it can be difficult to determine whether the outcomes are truly due to the intervention being tested or the variation in student characteristics within the study (Theobald and Freeman, 2014 ). To help account for this, we created a linear regression model to predict student outcomes on the meiosis assessment. Our model (Eq. 1) shows no significant contribution to student scores due to the demographic or multimedia preference variables that were investigated. The model did, however, point to prior knowledge (pretest score) and SAT scores as possible contributors to student normalized gain scores. The relationship between prior knowledge and posttest scores makes sense, as students who are more familiar with the material before instruction are more likely to achieve consistent scores on assessments after instruction. With regard to SAT scores, while they have been suggested as a possible predictor of freshman college success (Hannon, 2014 ), their contribution to student meiosis scores in our sample was extremely minimal (βSAT = 0.0005). This suggests that, despite their minor influence within our sample, SAT scores may have little to no contribution to learning outcomes on the topic of meiosis. Of most importance to our study, however, was the analysis of the contribution of treatment condition (module vs. traditional lecture) to student outcomes. Linear regression analysis showed a significant effect of treatment condition on assessment scores (β = 0.12, p = 0.001). This suggests that the manner in which meiosis concepts were presented to students in our study did play a significant role in the outcomes of their meiosis assessments. Using linear regression, we were also able to show that, within treatment conditions, there was no significant effect on assessment score due to the section in which the students enrolled (Tables 2 and and3).3). Regression modeling allows us to show the outcomes demonstrated in this experiment were most explained by the instructional treatment that participants received rather than the other possible contributing variables investigated. To provide further support for this, we also decided to stratify student assessment scores across the individual factors investigated in regression analysis. Stratification provided us with a more in-depth view of treatment effects within specific demographic categories, thus furthering the conclusion that the differences in learning outcomes observed can be attributed to the treatment condition.
Previous studies have investigated the possible link between preferred student learning styles and the effective use of multimedia learning tools on a variety of different concepts. While the results have been rather mixed (Carlson, 1991 ; Ross and Lukow, 2012 ), we attempted to account for the variability in preference for multimedia learning in our sample population. Previous studies have used a variety of instruments to assess student learning styles (Kolb, 1984 ; Ross and Lukow, 2012 ); however, in our investigation, we decided upon a more simplistic approach, allowing students to self-identify their levels of multimedia preference. The participants in this study were asked to answer on a Likert scale how well they believe that they learn using multimedia resources such as animation and video. From these data, we selected the subset of students who chose one of the two extremes: strongly agree or strongly disagree. Students who self-identify as having either a strong preference or strong opposition to multimedia resources are thought to be more likely to have specific and memorable previous experiences with multimedia learning tools that could skew their achievement on the meiosis learning module. Our results show that students self-identifying as having a strong preference to multimedia learning resources scored significantly higher when they used the learning module rather than attended a traditional lecture setting (d = 0.39, p = 0.01). This outcome is no surprise, considering that these students already show a preference to this type of learning. It is, however, of note that there was no significant difference (p = 0.91) between treatment groups when students identify strong opposition to multimedia learning tools. This would suggest that even among students who self-identify as being opposed to multimedia learning, the learning outcomes are equally high. However, as noted previously, the low sample number in the “strongly disagree” category could challenge any inferences made on this group. Even if we redesigned the analysis to include both those selected “disagree” and “strongly disagree,” the sample size (n = 28) would still be disproportionate compared with the “strongly agree” group (n = 171). We also did not feel comfortable with grouping these two categories together, since they could represent wide variation in students’ perception of multimedia learning. As a whole, linear regression still showed no effect on assessment outcome based on this variable. These findings are consistent with recent studies reporting that defined learning styles such as these do not effect student learning outcomes (Rohrer and Pashler, 2012 ). In a large introductory classroom in which students from myriad of educational backgrounds come together, results such as these are important. With interactive multimedia learning tools such as these that, at a minimum, perform equivalently to a traditional lecture setting, instructors can use the learning module investigated here with confidence that they can effectively convey the material needed to a diverse cross-section of students.
Recent studies have focused on the call for both a greater overall persistence in scientific majors and an increase in numbers of students enlisting in STEM majors (AAAS, 2011 ; PCAST, 2012 ). To achieve these reform goals, it is imperative that learning take place across the demographic spectrum that is seen in today’s college lecture hall. To ensure this, we set out to investigate the performance of VCell learning modules across multiple demographic variables. Our original plan was to analyze the results of the treatment groups across student major, year in school, ethnicity, and gender in hopes of investigating achievement in a large classroom setting consisting of students with diverse backgrounds. However, with the introductory status of the course that was used in this investigation, the number of non–STEM major students enrolled in the study was too small (n = 13) for us to effectively analyze any treatment effect across student major. Disproportionate distribution also prevented stratification of treatment conditions across student ethnicity. Regression analysis, however, did not show any significant contribution of ethnicity to assessment outcome for the study presented here.
For demographic factors that we were able to investigate, when looking at module versus traditional lecture treatment by year in school, we did see that underclassmen performed significantly higher on the meiosis assessment when they received instruction solely from the learning module rather than from a traditional lecture setting (d = 0.33, p < 0.001). Upperclassmen, by contrast, showed no significant difference in scores across treatment (p = 0.73). It should be noted that the number of upperclassmen enrolled in this course was also rather low (n = 32), which could affect the results seen here. This could explain why regression analysis showed no significant contribution of year in school to student assessment outcome. In addition to analysis by student year in school, analysis of our results stratified by student gender also showed a significantly higher outcome for students in the module group regardless of gender, suggesting gender uniformity in module performance. Module performance for both males and females in this study again provides instructors confidence in assigning this stand-alone learning tool regardless of class makeup.
Ultimately, achievement scores on the meiosis assessment in the learning module group were either higher than or on par with those of the traditional lecture treatment across the demographic conditions tested here. These results suggest that the achievement outcomes attained after learning from this learning module are consistent across the demographic variables investigated in this study. This again provides instructors preliminary evidence that this learning module can be used to prepare students with concepts in a setting outside the lecture hall.
Dissemination of empirically tested learning modules that convey concepts to students despite differences in demographics or learning preference can provide instructors with powerful resources for implementation in a hybrid learning environment. These resources would provide students the preparation that is needed to reap the benefits of an active learning–centered, flipped-classroom environment (Freeman et al., 2014 ; Gross et al., 2015 ). While the results from our investigation show that student achievement was significantly higher for the learning module treatment group, we have yet to investigate its effectiveness in an actual flipped-classroom setting. In the future, we plan to expand our research on learning modules to a variety of classroom environments, including flipped classrooms. These studies will focus on the effectiveness of VCell learning modules as compared with other methods of outside instruction such as reading assignments and recorded lectures. We would also like to expand the conclusions that can be made from the results of our future studies, and therefore would redesign our assessment instrument to examine specific concepts in more depth. This would allow us to make stronger conclusions on conceptual understanding of specific aspects of meiosis when learning with interactive modules and lead to greater understanding of the strengths and weaknesses of the multimedia resources we have developed. This information can then be used to guide revisions to the modules or delineate more specifically when the module(s) may be most effective.
Additionally, we acknowledge that the quasi-experimental design in this study does have limitations. Further investigation using a true experimental design with participant randomization in a controlled environment would reduce the number of extraneous variables seen in this study and would add strength to our inferences. We plan to include this level of experimentation in future projects on the effectiveness of VCell online-learning modules. We also are creating new learning modules for use in undergraduate biology instruction that will require further investigation. From this research, we aim to develop and test an entire collection of learning modules for use in introductory-level undergraduate biology. This collection would serve as research-tested instructional tools through which instructors at any university can convey basic concepts to their students, thereby opening classroom time for active-learning activities. By making these resources available to institutions nationwide, we can provide additional learning resources that reinforce science learning as a whole in an effort to assist with STEM education reform.
The goal in the production of learning modules by the VCell Animation team is to provide high-quality online resources designed to convey biological concepts across variation in student demographics and course design. One such course design in which effective learning modules may prove most beneficial is the flipped model of active-learning classrooms, which has been shown to lead to higher student achievement in multiple studies (Haak et al., 2011 ; Freeman et al., 2014 ; Gross et al., 2015 ). However, it has been noted that students must be properly and adequately prepared before the class period to achieve the greatest learning outcomes (Gregory, 2009 ; Andrews et al., 2011 ). The results of our investigation show that students using a stand-alone learning module on the topic of meiosis achieved significantly higher outcomes on a meiosis assessment than students who received instruction in a traditional lecture setting alone. We believe that the dynamic, interactive nature of the learning module presented here provides students with cognitive assistance that may promote conceptual understanding. This, together with the ability to provide a one-on-one interaction with the material, could aid the module in providing an alternative yet effective environment for students to reinforce ideas about meiosis. These results demonstrate the potential impact of online-learning modules. However, we note that additional research is needed to investigate what features modules should have to further improve student learning, whether modules appropriately prepare students for active-learning activities in class, and how modules can be designed to most effectively prepare students for in class, active-learning activities.
Materials presented in this paper can be accessed using the VCell Animation project website (http://vcell.ndsu.edu/animations). There are no requirements for access; however, there is an optional registration prompt for individuals who choose to download the materials for personal use. In addition to the project website, the VCell Animation Collection also has a YouTube site (www.youtube.com/user/ndsuvirtualcell) and a free Apple iOS application (http://itunes.apple.com/us/app/virtual-cell-animations/id427893931?mt=8) that will provide access to VCell content. Links to the module tested in this study are currently being loaded to the project website or can be accessed by contacting the corresponding author.
Funding for the Virtual Cell Animation Collection was provided by National Science Foundation (NSF)/CCLI, NSF/TUES (NSF awards: 0086142, 0618766, and 0918955) and the U.S. Department of Education/FIPSE.