Adequate dimensionality, divergent validity, and scale consistency contribute validity to the self-regulation (SR) scale and its subscales in the medical education setting. However, the analyses indicate that some of the items do not contribute optimally in the regulation scales and that external regulation (ER) and lack of regulation (LR) scales were weak regarding dimensionality in this sample.
Because psychometrics is about measuring the unobservable, there is not one single criterion against which we can assert fulfilment of the goal of trustworthy tools. Therefore, several aspects, taken together, form a basis for assessment and basis for further scale development. The internal consistency in this sample was very good in comparison with other studies using the scales. Cronbach’s alpha values for the three main scales (SR
: 0.82, ER
: 0.72, and LR
: 0.65) were exceeding those obtained in a Norwegian medical setting (SR
: 0.73, ER
: 0.69, and LR
] and generally exceeded previously reported Cronbach’s alpha
of 0.48 to 0.81 in several studies in the Netherlands
], 0.46 to 0.72 in British settings
], and 0.69 to 0.75 in a Finnish pharmacy setting
Dimensionality, as assessed by Mokken scale analysis, was higher in the dataset from the revised version, although the estimates were not ideal (Table
). The strongest regulation scale was the subscale Self-regulation process and results,
with coefficient H
0.41. Following empirically derived rules of thumb of H
-interpretation, this falls into the category of moderate dimensionality
]. Of the main scales, self-regulation
got highest value with H
0.32 which means it is a weak scale regarding dimensionality. External regulation
scale displayed lowest dimensionality (H
0.21). This finding corresponds with this dimension not being detected at all in a Finnish sample
]. Higher H
coefficient values for sub scales are explained by their items being more narrowly connected to each other, whereas the main scales approach the strategy dimensions more broadly. The implication for scales with low dimensionality is that ranking of participants with scores in a narrow range will be less accurate.
On a more detailed level, the Hi estimate contributed with insight into how single items contribute to scale dimensionality (Table
). In general, all items contributed to their respective scale, except items 22 and 28, which did not seem to contribute much to the scale (external regulation). The characteristic of item 22 is discussed below. Item 28 concerns fulfilment of assignments during the course which makes it less appropriate for a clinically (practically) oriented course. The total dimensionality coefficient (H) for that scale would increase removing these two items. However, that would remove nuances from the scale construct, and hence, from the theoretical base. Since both items score positively on the Hi, they contribute, however little, to the scale construct. The practical implications are that the scale is less accurate in distinguishing between respondents with similar scale scores than if all items contribute highly to the scale construct. When disregarding previous partitioning and exposing all items to exploratory scale partitioning, they risk being regrouped in ways that were not initially intended. However, the theoretical scale structure was broadly replicated in the data. All aspects of the established scales were represented in the six Mokken scales. The largest group is Mokken scale 1, with six of the seven self-regulation of learning process and results items represented. Scale 2 contains items from external regulation, mostly from external regulation of learning process. Scale 3 was identical to self-regulation of learning content. Scale 5 has two items from external regulation of learning results, and scales 4 and 6 correspond with lack of regulation.
With the exception of one item (22), the six exploratory Mokken scales correspond well with the theoretical partitioning established in previous research (Table
). However, in this sample, item 22 converges with self-regulation of learning process and results, although it belonged to external regulation of learning results in the established partitioning. This item also displayed low item-scale homogeneity (Hi). The item concerns the frequency with which students thoroughly apply themselves to the methods dealt with in a course. Considering the context in which respondents were situated—the clinical preparatory course—the “methods” could be interpreted as diagnostic methods, and thus a core aspect of the course. Training to apply these methods could fit well into the scope of a self-regulated goal. Consequently, this item does not function well in discriminating between self- and externally regulated strategies, but contributes to both constructs in this setting.
Regulation strategies as such are theorised as not only relating to individual preference, but also to the specific learning situation
]. Consequently, the situative aspect should be considered in data collection and interpretation. Comparisons between different overarching curricula, such as PBL and traditional ones, should expect variations in how the construct is interpreted
]. Comparisons within the same educational culture and setting, contribute to establish the construct validity of regulation strategies as measured by these scales. When comparing this study’s scale means to other traditional Scandinavian curricula, a similar distribution of pattern is discerned, dominated by external regulation,
followed by self-regulation
and lack of regulation
Regulation strategy mean scores from the Swedish setting compared with Norwegian medical education settings, using problem-based (PBL) and traditional curriculum
As the three regulation strategies are different constructs, they are not expected to interrelate; i.e. they should display divergent validity. Nevertheless, the subscales of the self-
and external regulation
scales should relate, to some extent, although covering different facets. The non-interrelation identified among the main scales contributes indirectly to their divergent validity. Interrelations between subscales were significant, although somewhat weaker than those reported elsewhere
]. Judging by their face value, one could assume that self-regulation
and external regulation
would be each other’s opposites and, hence, correlate negatively; however, in this sample, they did not. Research shows that students can successfully combine the two regulation strategies, suggesting a dynamic relationship where external regulation has modelling and scaffolding functions
Invariant item ordering (IIO) is not a claimed property of the ILS’s regulation strategy scales. Scale items contribute within the regulation strategy scales, even if not ordered in a hierarchy. Nevertheless, the test for IIO contributed additional validation data in disclosing to what extent respondents conceive the items in a similar manner. Three scales displayed adequate IIO accuracy, while the overall low IIO estimates for other scales express variation in how respondents approach the scale items (Table
Implications and further development
The gains of having an established instrument that can be used in different disciplines and in different cultural settings should be considered when choosing and assessing methods to measure regulation strategies. The continued use of these established scales allows researchers to build on prior theoretical bases. In line with the view of validity pertaining to contextualised results, complexity of the construct, and purpose of use of the results, the researcher needs to interpret results on regulation strategies in relation to the students’ study situation
Scale scores should be interpreted in conjunction with other empirical data, such as course activities or other scale scores, thereby contributing to further construct validity.
Further refinement of the wording of questionnaire items is recommended in this translation, as medical students found some items hard to deal with. This work should retain content validity in adhering to the theoretical base and use reported dimensionality coefficients (Hi),
and future content validation of the wordings with student groups, preferably from other disciplines as well. The vocational character of the participants’ study environment is currently marginally reflected in the strategy scales. Considering the contextual influence discussed by Richardson
], more accurate results would perhaps be found in medical and other professional education if the scale items were adapted toward vocational aspects. However, as item adaptation would imply deviation from the original scales, and thus, the theoretical base, a better approach would be to add a vocational oriented scale along with the others. Generalizability of findings of this study beyond third-year students in a traditional Swedish medical curriculum is restricted by the moderate sample size and single discipline context. Data-gathering from the whole cohort, cross-sectionally over four course settings, contribute to the strength of the study, providing a broad base for validation of scale results.