Figure is an example of a high-scoring mind map from one of the subjects. AVD assigned this mind map a total score of 400, GPZ assigned it a total score of 337, and VGO assigned it a total score of 377. The average total score of this mind map, based on all 3 examiners, was 371.33. This mind map will be used as an example in the following discussion.
The MMAR contains 6 variables (concept-links, cross-links, hierarchies, examples, pictures, and colors) that assess the quality of a mind map and translates this into a numeric score that can enable an educator to compare one mind map from another.
The operational definitions of 4 variables were adapted from the work of West, Pomeroy, Park, Gerstenberger, and Sandoval [20
]. A concept is a perceived regularity in events or objects designated by a label [21
]. As can be observed in Figure , there are many concepts in this mind map such as succulents
and structural modifications
. A concept-link is a valid link between concepts using a line with a word or statement written above the line describing how the concepts are related [21
]. On the right side of the mind map, there is a concept-link between the central theme of the mind map, succulents and cacti
(note how the subject did not write the word "cactus" but simply represented it as a picture), and structural modifications
. This is a valid link because the text passage described the different structural modifications found in cacti and other succulent plants. A cross-link is a valid link demonstrating a relationship between different domains of knowledge [21
]. Cross-links are very important in mind maps because they demonstrate relationships between concepts. On the left side of the mind map, there are several blue cross-links between different types of succulents
. Hierarchy is indicated by the direction of the line in the concept-link and the arrangement of concepts in the mind map–that is, more general concepts are located centrally and more specific concepts are located peripherally [21
]. In the example, the link from succulents and cacti
to structural modifications
to thickened leaves
represents secondary hierarchy that is valid and unidirectional. An example is a valid word that exemplifies the concept [21
]. In Figure , the words written in purple–such as leaves
, and corms
The final 2 variables unique to mind maps that were included in the MMAR are pictures and colors, both of which facilitate the conversion of information from short- to long-term memory [26
]. A picture is a graphic representation that aids the learner in recalling the information [15
]. Pictures can be located anywhere in the mind map and the combination of the two cortical skills of words and pictures enhances intellectual power [12
]. The quality, clarity, and detail of the pictures were not a factor in assigning points. In Figure , the subject integrated many pictures into the mind map. Each color used in the mind map was assigned points so that the more colors used, the more points assigned to the mind map. Again, in Figure the subject used many colors when creating the mind map. The advantage of using different colors in a mind map is that it improves the recall of information, allows faster access to the information, and ultimately increases creativity [12
In this study, we have demonstrated a high ICC total score value (.86) among 3 examiners using the MMAR. This finding suggests that the MMAR can be reliably used to score mind maps and demonstrates its potential applications in research and education. Moreover, the addition of pictures and colors reveals strong interrater reliability (.86 and .73, respectively). This suggests that these variables can also be reliably scored and lend to the uniqueness of mind maps. However, pictures and colors were not found to strongly strengthen the overall reliability of the MMAR based on our internal consistency analyses using Cronbach α. Recall Cronbach α was .38 (95% CI, .11 to .58) for all 6 variables of the MMAR and .29 (95% CI, -.02 to .53) without pictures and colors. Cross-links (.58) and examples (.53) were moderately reliable. In this study, concept-links and hierarchies were found to have very weak reliability, and we believe that this was due to confusion as to their differences. Each cross-link in a mind map should be assigned a numeric value (10 points each). Unlike cross-links, however, only the highest level of hierarchy is scored on a mind map (5 points each). For example, the mind map in Figure has quaternary (fourth-level) hierarchy, which can be observed in the left upper quadrant of the mind map. Ultimately, there should be 20 points assigned for hierarchy in this mind map. Cronbach α for Factor 1, with the variable hierarchy excluded (ie, only cross-links and examples), was .75. This suggests that hierarchy is difficult to assess in mind maps, and that further development of this variable is necessary in order to improve its reliability. Further research should explore how different operational definitions for concept-links and hierarchy impact the validity and reliability of the MMAR.
In this study, we did not attempt to create an original mind map grading rubric. Rather, we used a valid and reliable CMA structural scoring system specifically studied in resident physicians as a framework for the MMAR [20
]. The metacognitive similarities between concept maps and mind maps allowed us to use the CMA structural scoring system as the foundation for the MMAR. Although we have established the interrater reliability of the MMAR, further research is needed to investigate its construct validity and reliability before it can be used in medical education.