|Home | About | Journals | Submit | Contact Us | Français|
Cueing facilitates retention and transfer of multimedia learning. From the perspective of cognitive load theory (CLT), cueing has a positive effect on learning outcomes because of the reduction in total cognitive load and avoidance of cognitive overload. However, this has not been systematically evaluated. Moreover, what remains ambiguous is the direct relationship between the cue-related cognitive load and learning outcomes. A meta-analysis and two subsequent meta-regression analyses were conducted to explore these issues. Subjective total cognitive load (SCL) and scores on a retention test and transfer test were selected as dependent variables. Through a systematic literature search, 32 eligible articles encompassing 3,597 participants were included in the SCL-related meta-analysis. Among them, 25 articles containing 2,910 participants were included in the retention-related meta-analysis and the following retention-related meta-regression, while there were 29 articles containing 3,204 participants included in the transfer-related meta-analysis and the transfer-related meta-regression. The meta-analysis revealed a statistically significant cueing effect on subjective ratings of cognitive load (d = −0.11, 95% CI = [−0.19, −0.02], p < 0.05), retention performance (d = 0.27, 95% CI = [0.08, 0.46], p < 0.01), and transfer performance (d = 0.34, 95% CI = [0.12, 0.56], p < 0.01). The subsequent meta-regression analyses showed that dSCL for cueing significantly predicted dretention for cueing (β = −0.70, 95% CI = [−1.02, −0.38], p < 0.001), as well as dtransfer for cueing (β = −0.60, 95% CI = [−0.92, −0.28], p < 0.001). Thus in line with CLT, adding cues in multimedia materials can indeed reduce SCL and promote learning outcomes, and the more SCL is reduced by cues, the better retention and transfer of multimedia learning.
As one of the material-oriented interventions in multimedia learning, cueing affects learners’ cognitive processing and learning outcomes [1–3]. The term cueing refers to the non-content information (e.g., arrows, color coding, highlighting) added in learning materials to attract learners’ attention and to further promote their selection, organization, and integration of instructional elements [4, 5]. A large body of research has shown that adding cues in multimedia materials facilitated retention and transfer of learning [4, 6–10]. Two recent meta-analyses have also confirmed the robustness of the cueing effect on learning outcomes [11, 12]. Many researchers have given an explanation from the perspective of cognitive load theory (CLT) , maintaining that cues could reduce learners’ overall cognitive load and help avoid overload, thus contributing to their learning performance. Nevertheless, two crucial problems that remain to be solved are (a) whether the cues really reduce total cognitive load, and (b) how a reduction in cueing-related cognitive load may be related to learning outcomes. In the current study, a meta-analysis and two following meta-regression analyses focused on these issues.
Cognitive load is generally defined as a multidimensional construct representing the cognitive demands associated with performing a specific task [14, 15]. In the 1980s Sweller  proposed cognitive load theory, which articulated the association between cognitive resources and task demands in creating cognitive load. Learners will consume their cognitive resources as long as they are performing a task, leading to cognitive load. In this model, working memory is a cognitive resource, but is a limited one; only a small fraction of elements can be consciously handled per unit time, especially when they are novel or unfamiliar. However, long-term memory provides the ability to circumvent the limitation of working memory with the help of schemas. Schemas are cognitive constructs in which multiple elements are incorporated into a single element, which can then be processed automatically. Consequently, the construction and automation of schemas are the main goals of instruction.
In the field of educational research, cognitive load theory [13, 17] is mainly used to explain the effects of various forms of instructional design. According to this theory, intrinsic cognitive load (ICL) is not directly affected by instructional design. It is related to element interactivity in learning materials and learners’ prior knowledge. Element interactivity has been regarded as the primary, representative mechanism of ICL for quite a long time. The level of ICL of a specific task is usually assumed to depend on the level of element interactivity. An element can be anything that will be or has been presented, for example a concept or a procedure. Instructional materials with low element interactivity allow single (or several) element(s) to be processed with little or even no reference to other elements, thus resulting in a low ICL; however, high element interactivity materials contain elements that heavily interact with each other and cannot be processed separately, leading to a high ICL. Extraneous cognitive load (ECL) is concerned with the quality of instructional design. It is detrimental to the processes of schema construction and automation and thus will hinder learning. Germane cognitive load (GCL) is directly beneficial to learning. It can be imposed by the cognitive processes of active schema construction such as classifying, inferring, and organizing. The total cognitive load during information processing is the sum of the three kinds of cognitive loads. One important objective of instructional design is to ensure that the total cognitive load is within the learner’s cognitive capacity, in order to avoid cognitive overload .
Different techniques have been used to measure cognitive load, mainly including subjective rating scales, dual-task performance, and physiological measures [14, 18–20]. Pass  introduced the mental effort scale, which was a modified version of Bratfisch, Borg, and Dornic’s scale  for measuring perceived task difficulty. Pass’s 9-point mental effort scale included one item that asked learners to report how much mental effort they invested when learning the material. Since then, the mental effort or perceived difficulty scale has been widely used in research in the field of learning and instruction because it is easy to administer, is non-invasive, and has good reliability and validity .
Bratfisch, Borg and Domic’s  mental effort or perceived difficulty scale for measuring subjective cognitive load (SCL) has been modified in several ways in previous studies. This heterogeneity reflects differences in terms of scale items, number of scale units, and the timing of measurement . Concerning items, Pass  asked participants to report their invested effort on a 9-point Likert scale by translating the perceived amount of mental effort into a numerical value. Other researchers have required learners to estimate how difficult (or easy) it was for them to learn from the instruction [8, 10, 23]. Some studies combined mental effort with perceived difficulty to measure cognitive load [24–27], whereas others added items measuring something that was different than cognitive load (e.g., “It was important to me to do well at this task” ). As for number of units, these scales have ranged from 9 points to 5 points (See Table 1). In most studies the questionnaire was given to the learners after learning had taken place [4, 8, 25], whereas other studies presented it during the learning process [29, 30].
What kind of cognitive load does mental effort or perceived difficulty (SCL) represent: ICL, ECL, GCL, or total cognitive load? We assert that mental effort or perceived difficulty is more likely to represent the total cognitive load for the following reasons: (1) Mental effort is defined as “the aspect of cognitive load that refers to the cognitive capacity that is actually allocated to accommodate the demands imposed by the task” . It is influenced by three factors: task environment characteristics, learner characteristics, and their interaction. The combination of these factors is more likely to represent the total cognitive load, rather than a specific cognitive load. (2) A mental effort or perceived difficulty scale does not specify which aspect of the learning experience the participants are required to rate. Rather, participants report their overall feelings about learning. (3) This position is consistent with that of Pass and colleagues [14, 31, 32] and other researchers [33–35] who have considered all measures of mental effort or perceived difficulty to assess overall cognitive load, rather than its constituent components (i.e., intrinsic, extraneous, germane).
Multimedia learning materials usually have high element interactivity, and learners often have no idea how to quickly search and process the correct elements in the limited time. In this context, the total cognitive load that learners bear can easily exceed the limited capacity of cognitive resources. In order to avoid cognitive overload during the learning process, many researchers add some non-content information (i.e., cues) in the learning material to guide the learners’ attention and reduce their total cognitive load. For example, Jeung, Chandler, and Sweller  studied participants who were learning geometry. In the experimental group, when the participants heard any comment about a certain rectangle (e.g., “area of the rectangle MADE”), this rectangle (e.g., MADE) in the picture would flash. De Koning et al.  added visual cues in the animation presented to the experimental group. When explaining a certain element in the cardiovascular system, all elements in the animation except this certain element would be darkened, which could be thought of as a spotlight-effect. Analogously, when experimental group participants in Lin and Atkinson’s  study were learning information about the rock cycle, red arrows would appear and point to the element being discussed. However, no such cues were presented to the control groups in these studies.
According to CLT, cues should prevent cognitive overload . However, this has not yet received consistent support from empirical research. Some studies have found that cues could reduce students’ perceived amount of invested mental effort or perceived task difficulty. For instance, Kalyuga et al. (Experiment 2)  found that using color coding reduced searches for diagrammatic referents in the text and ameliorated split-attention effects, resulting in lower perceived difficulty. Berthold and Renkl  showed that participants who were provided color coding and flashing reported significantly less SCL than their counterparts without such aids. However, a larger number of studies have found that cues do not produce a significant reduction in SCL [29, 36, 37]. Ozcelik et al.  found there was no significant difference in the perceived difficulty of the instruction (chemical synapses) between the color-coded group and the conventional group. A series of studies by De Koning and colleagues also did not find any effect of cues on mental effort [4, 38–41]. In sum, there is a lack of cogent evidence about whether cues affect cognitive load.
Compared to research on cues and SCL, results concerning the impact of cues on learning outcomes can be considered to be a bit more consistent; that is, adding cues in multimedia materials can improve learning performance. Ozcelik, Karakus, Kursun and Cagiltay  discovered that color coding facilitated learners’ memory and comprehension of chemical synapses, revealing higher scores on retention and transfer tests, though the cues did not influence perceived difficulty. By now, a positive cueing effect on learning outcomes has been shown in numerous studies in various instructional domains, such as math , biology [4, 10, 39], engineering [42, 43], psychology , and physics [8, 44]. Though some studies did not find an improvement in learning outcomes due to cues [36, 38, 45], the authors of two recently published meta-analyses concluded that overall, cues indeed improved learning performance [11, 12].
The first question is whether cueing can indeed lower learners’ total cognitive load and help avoid cognitive overload, as cognitive load theory predicts. The present study used meta-analysis to assess whether cueing reduces cognitive load, as measured by SCL (mental effort, perceived difficulty, or their combination) and improves learning (retention and transfer). In addition, the present study calculated pooled effect sizes for the impact of cues on a retention test and transfer test in order to compare the results with those of two related meta-analyses [11, 12] that adopted different inclusion criteria from ours. Our first research question about effects of cueing on total cognitive load was not considered in the two recently published meta-analyses, which only looked at the effects of cueing on learning outcomes.
Supposing the findings of the meta-analysis do suggest that cueing reduces total cognitive load, then the following question is about the relationship between cueing-related cognitive load and learning outcomes. One of the solutions is to use correlation or regression analysis to explore this question. For example, a few studies have found a significant negative correlation between SCL and learning performance [23, 46]. Nevertheless, the vast majority of empirical researchers have separately analyzed the effects of cueing on cognitive load or on learning outcomes [4, 25, 29, 39], leading to a lack of evidence needed to make direct inferences about their relations. An alternative solution is to use meta-regression to examine the predictive effect of cueing-related cognitive load on learners’ performance. If total cognitive load is reduced by cues, there is every reason to predict better learning outcomes (e.g., better retention or transfer performance) according to CLT. Two meta-regression analyses were used to test this assumption. This statistical method synthesizes correlational results across studies. In this case the correlational data that were included were those describing the relation between cueing-related SCL and scores on retention or transfer tests, respectively.
According to CLT, we expected that (a) cues can reduce SCL (Hypothesis 1) and (b) promote scores on retention (Hypothesis 2a) and transfer (Hypothesis 2b) in multimedia environments, and further, (c) the more SCL is reduced by cues, the better retention (Hypothesis 3a) and transfer (Hypothesis 3b) of multimedia learning. Hypotheses 1, 2a and 2b were tested using meta-analysis; Hypotheses 3a and 3b were tested using meta-regression analyses.
The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed (S1 Checklist).
To identify relevant studies on the effects of cueing in multimedia learning, a systematic literature search was conducted by searching the electronic databases PsycINFO, Education Research Complete, Science Direct, PubMed, ProQuest, and China National Knowledge Infrastructure (CNKI). Search engines such as Google Scholar and the reference lists of the identified studies were also used. The search keywords were “cue,” “cueing,” “signaling,” and “color coding” with different combinations of “multimedia learning,” “cognitive load,” “mental effort,” “retention,” and “transfer.” The search was limited to the period between January 1995 and March 2016. To minimize the file drawer problem, we (1) tried to contact some researchers to provide the details of their unpublished studies (e.g., dissertations or conference papers) through email, and also (2) assessed publication bias statistically (see below).
Journal articles as well as dissertations and conference presentations would be selected. The following inclusion criteria for the retrieved articles were adopted for the meta-analysis. The studies were included if (a) they were based on an experimental design; (b) multimedia learning materials were used, that is, the materials simultaneously contained words (e.g., on-screen text, narration) and pictures (e.g., diagrams, videos, animations); (c) both an experimental group with cues and a control group without cues were compared; (d) sufficient quantitative data (e.g., means, standard deviations and n; t test or F test values) were reported to calculate effect size; and (e) the effect size did not go beyond three SD of the mean of all effect sizes to alleviate the effect of outliers representing extreme values , which would make mean values unrepresentative of the literature as a whole. Studies were excluded if they did not meet the inclusion criteria mentioned above.
It was important to emphasize that different studies might adopt multiple dependent variables to measure (total) cognitive load. A basic assumption of meta-analysis is the independence of effects, and the inclusion of multiple dependent variables in each study would not conform to this assumption . Thus to abide by this assumption and avoid the potential deviation due to dependencies between effect sizes introduced by multiple variates per study, the following criteria were also used:
Data with respect to SCL ratings for both experimental and control groups were extracted by two of the authors (HX, YH) and checked by another (JC). Discrepancies were resolved through discussion.
As for analyses of retention and transfer performance, the included studies reported both SCL and a learning outcome (i.e., SCL and retention test, or SCL and transfer test, or SCL and both). Because the data included in this study were continuous data with no consistent unit, we chose Cohen’s d as the standardized estimate of effect size for each article . Specifically, Cohen’s d was calculated as the mean score difference in SCL ratings or learning outcomes between cueing (experimental) and no-cueing (control) groups. When the studies reported multiple experiments or multiple conditions, the data were merged to compute one pooled effect size in order to minimize the deviation of results caused by a large number of effect sizes and disproportionate weight if not pooled . For instance, Jamet et al.  conducted a 2 (Salience: salient, non-salient) × 2 (Display: sequential, static) between-subjects design, which could have generated two effect sizes (one effect size per condition of Display), but we combined these two into a pooled study-level effect size through CMA (see below). Study-level effect sizes were then averaged to obtain an overall average effect size. The 95% confidence interval (95% CI) of each effect size was also calculated. For Cohen’s d, the direction of the effect size was negative if the SCL rating or learning performance of the cueing group was lower than that of the no-cueing group. An effect size of ±0.2 was considered to be small, ±0.5 moderate, and ±0.8 large . A random-effects model was preliminarily selected to calculate the pooled effect sizes and their 95% CIs because articles included in the present study differed in a number of variables (e.g., groups of participants, research methods), potentially resulting in a heterogeneity of results among studies.
Data were analyzed using the Comprehensive Meta-Analysis (CMA) 2.0 software (https://www.meta-analysis.com/). We calculated the Q statistic with its p value to test whether the random-effects model used in this study was reasonable. A statistically significant Q value indicates that it would be better to calculate effect sizes based on the random-effects model [48, 52]. Regarding publication bias, Egger’s linear regression test  was used. Through this test, a regression equation can be created with the standard normal deviate of each study as the dependent variable and the estimate’s precision of each study as the independent variable. The intercept of the regression equation provides a measure of publication bias. The smaller its deviation from zero the less pronounced the bias.
Results of the initial literature search and study selection are shown in Fig 1. A total of 32 articles that met the inclusion criteria were finally included and analyzed. There were 27 articles obtained from journals, 4 from dissertations, and 1 from an academic conference. Twenty-eight articles were written in English, and 4 in Chinese. Accordingly, 32 study-level effect sizes with respect to SCL were computed, involving 3,597 participants (See Table 1 and S1 Dataset). Twenty-five study-level effect sizes regarding retention containing 2,910 participants and 29 study-level effect sizes regarding transfer encompassing 3,204 participants were also computed.
Table 2 presents the results of cueing on SCL as well as learning outcomes, and forest plots of the meta-analysis with respect to each index are shown in Fig 2. Concerning SCL, the meta-analysis based on the random-effects model revealed that the overall pooled effect size was small but statistically significant (d = −0.11, 95% CI = [−0.19, −0.02], p < 0.05). Thus is in line with CLT, adding cues in multimedia materials reduced learners’ perceived total cognitive load.
Concerning learning outcomes, there were small-to-medium cueing effects for both retention (d = 0.27, 95% CI = [0.08, 0.46], p < 0.01) and transfer (d = 0.34, 95% CI = [0.12, 0.56], p < 0.01). These results are consistent with the previous two meta-analyses; adding cues in multimedia materials facilitated retention and transfer of learning.
If cues reduce SCL, would this lead to a better retention or transfer of learning as CLT would expect? To clarify this question, we conducted two parallel meta-regression analyses to investigate the potential relationships between SCL and scores for retention or transfer. In both analyses SCL (dSCL) effect sizes were used as the predictor, and retention test (dretention) and transfer test (dtransfer) effect sizes as dependent variables (Table 3; S2 Dataset). The data sources were the same as those used for retention- and transfer-related meta-analysis.
First, a meta-regression with dSCL as the predictor and dretention as the dependent variable was conducted (Table 3; Fig 3A). The result revealed that the effect of cueing represented by dSCL could significantly predict the effect of cueing represented by dretention (β = −0.70, 95% CI = [−1.02, −0.38], p < 0.001). Thus in line with CLT, the more SCL was reduced by cues, the better retention of learning. Similarly, a meta-regression with dSCL as the predictor and dtransfer as the dependent variable was conducted (Table 3; Fig 3B). The result showed that dSCL significantly predicted dtransfer (β = −0.60, 95% CI = [−0.92, −0.28], p < 0.001). Thus also in line with CLT, the more SCL was reduced by cues, the better transfer of learning.
As shown in Table 2, the p values of the Q statistic were significant (QSCL = 45.20, p < 0.05; Qretention = 147.02, p < 0.001; Qtransfer = 250.10, p < 0.001), indicating significant heterogeneity in the results. Thus, using a random-effects model in this study was appropriate. Egger’s linear regression test was not significant, suggesting that publication bias was an unlikely influence on the results.
A meta-analysis and two subsequent meta-regression analyses were conducted in the present study to test whether the addition of cues can reduce learners’ subjective cognitive load and promote learning outcomes in a multimedia environment, and further, to explore the relationship between cognitive load and learning performance in this learning context. Consistent with Hypothesis 1, learners in the cueing condition reported a lower perception of cognitive load than learners in the no-cueing condition (d = −0.11, p < 0.05), revealing that cues could reduce subjective cognitive load. Results from retention-related meta-analysis and transfer-related meta-analysis also showed that adding cues in multimedia materials facilitated retention and transfer of learning significantly, thus supporting Hypotheses 2a and 2b. Our results were the same as the previous results on the transfer test. Both Richter et al.  and Xie et al.  found a small-to-medium cueing effect (r = 0.17 in the meta-analysis by Richter et al.; g = 0.36 in the meta-analysis by Xie et al.). Regarding the retention test, Xie et al.’s  meta-analysis and the current study both found that cues improved retention, but Xie et al.  found a medium-to-large cueing effect (g = 0.53) whereas our meta-analysis revealed a small-to-medium cueing effect. The reason for this discrepancy may be that we used different criteria than Xie et al.  did for including retrieved articles. Specifically, the articles included in the present study reported both SCL and learning outcomes, whereas Xie et al.  left SCL aside. Therefore the inclusion scope in our meta-analysis was different because studies with the measurement of retention scores but no SCL were excluded from our retention-related analysis. In addition, in support of Hypotheses 3a and 3b, dSCL negatively predicted both dretention (β = −0.70, p < 0.001) and dtransfer (β = −0.60, p < 0.001), suggesting that a lower cueing-related cognitive load meant higher scores on retention and transfer tests. Overall, results from the present study provided full support for CLT.
CLT is a plausible theory to explain the cueing effect. According to CLT, a reduction of subjective total cognitive load and an avoidance of cognitive overload would be expected in conditions with cues by virtue of their favorable external design, compared to conditions with no cues . This assumption proved to be correct according to the results of our meta-analysis on the effects of cueing on SCL, retention and transfer. Thus, reducing cognitive load may be the crucial step in effective retention and comprehension in multimedia learning. However, to make a more direct inference about the cueing effect on learning outcomes based on the perspective of CLT, a further analysis of the relationship between cognitive load and scores of learning is probably requisite [23, 46]. Following CLT, the perceived total cognitive load would be negatively related to learning outcomes, such as retention and transfer performance . Two meta-regression analyses in this study found this was indeed the case, again supporting CLT. These results were analogous to those of other empirical studies. Through two experiments, Huk et al.  discovered that students’ perceived cognitive load negatively correlated with both remembering and understanding. In Paik and Schraw’s study , learners were required to make a judgment of difficulty (JOD) after learning about a flushing toilet tank. Similarly, negative correlations were found between JOD and learners’ recall, as well as transfer performance.
Though the present study confirmed a reduction of total cognitive load (defined as subjective cognitive load in the form of mental effort, perceived difficulty, or their combination) in the cueing condition, it is seemingly impossible to draw conclusions about a specific type of cognitive load, i.e., ICL, ECL or GCL. Cueing is the consequence of external instructional design, but it may be related to both ECL and GCL . For example, ECL would be decreased by reducing the unnecessary visual search when cues are provided in a given material (ICL is constant). However, it also could be that cueing increases GCL by optimizing the schema construction and automation.
It should be noted that the sensitivity of subjective ratings is critical for the measurement of cognitive load. SCL, especially mental effort, perceived difficulty, or their combination, is likely to be the most frequently used index of cognitive load [21, 38, 64, 65], but the sensitivity of measures of SCL has been questioned. After all, there are still quite a few empirical studies that have not found a reduction of SCL by cues as expected [4, 10, 36–41, 63]. There remain two possible explanations. First, there may be differences in participants’ understanding of the SCL questionnaire. Taking the question “How much effort did you have to invest to learn about the materials?” for example, some learners in the cueing condition might report that “I invested less effort because cues really worked” (as CLT would expect); others in the same condition could interpret the question in the opposite way, as “In an effort to comprehend the materials, I learned carefully” (not supporting CLT). Therefore, for the same question, learners in the cueing condition may respond differently from different viewpoints, probably decreasing the sensitivity of SCL.
Second, the effect of cueing on SCL may be moderated by external variables (e.g., dynamism of materials) that were not examined in the current analyses. For instance, cues have been found not to play a positive role in learning outcomes when the materials were dynamic . Other studies have shown that when presenting materials in a static format, learners in the cueing condition outperformed those in the no-cueing condition . A meta-analysis of cueing effects by Xie et al.  found that adding cues was beneficial for retention and transfer of knowledge when the multimedia material was static, whereas there were no significant effects on learning outcomes if the material was presented in a dynamic way. Taking CLT into consideration, these results could have been obtained because the elements in the dynamic materials overshadowed the effect of cueing on cognitive load, resulting in no learning improvement.
The current study presents the results of a meta-analysis adopting SCL as an index of cognitive load and scores on retention and transfer as indexes of learning outcomes based on CTL and, different from previous meta-analyses [11, 12], includes two meta-regression analyses focused on the relationship between SCL and learning outcomes, not only providing a more direct reference for CTL but also giving guidance on instructional design. Even so, several limitations should be acknowledged. First, we chose only SCL (specifically mental effort, perceived difficulty, or their combination) as the index of total cognitive load. Perhaps other measures would show additional unexpected but vital results. Second, we just focused on multimedia learning environments that contained both words and pictures. The effect of cueing when reading plain text (with no pictures) is also worthy of attention . Third, though we argue that both mental effort and perceived difficulty measure the total cognitive load, it is not necessarily the case. After all, there are still many studies in which mental effort and perceived difficulty were used to measure ECL or GCL, leading to doubt about the measurement of cognitive load . Moreover, differences between measurements of mental effort and measurements of perceived difficulty were ignored. It should be acknowledged that a learner’s rating of the perceived difficulty is not completely equivalent to the mental effort invested, though the two concepts are correlated. Van Gog and Paas  pointed out that mental effort was subordinate to a process, and contained more complex components than the task per se, whereas the perceived difficulty was mainly subject to the task. A potential example was that learners might not be motivated to invest sufficient mental effort when they perceived a specific task to be extremely difficult , leading to reversed scores of mental effort and the perceived difficulty scales. Thus, results concerning SCL in the present study must be treated with some caution.
All in all, in line with cognitive load theory, our meta-analysis and subsequent meta-regression analyses indicate that (a) cues can reduce subjective cognitive load, (b) cues can facilitate retention and transfer performance, and (c) the more SCL is reduced by cues, the better retention and transfer of multimedia learning. These results have clear theoretical and applied value.
This research was funded by National Natural Science Foundation of China Grant (#31300864, #31771236) and Self-Determined Research Funds of CCNU from the Colleges’ Basic Research and Operation of MOE (#CCNU16A02023) to FW.
The data files of this study are available as Supporting information.