Just as one might conduct an exploratory analysis prior to fitting a confirmatory model, in the bifactor case it is important to conduct an exploratory bifactor model (where items are free to load on any group factor) before fitting a more restricted bifactor model (where items are forced to load on one and only one group factor). Although standard statistical software programs do not provide “bifactor rotations”, a researcher can easily conduct a Schmid-Leiman (SL; Schmid & Leiman, 1957
) orthogonalization prior to fitting a restricted bifactor model. In the present study, these analyses were performed on a tetrachoric correlation matrix using the SCHMID command in the PSYCH library (Revelle, 2009
) in the R
statistical package. Simply stated, a SL is conducted as follows. First, a correlated traits (i.e., oblique rotation) factor analysis is performed specifying a given number of "primary" factors. Second, the correlation matix among the primary factors is in turn factored extracting a single second-order factor. Finally, the SL transformation is performed as follows. An item's loading on the general factor is found by multiplying the item's loading on the primary factor by the primary factor's loading on the second-order factor. An item's loading on the group factor is found by multiplying the item's loadings on its primary factor by the square root of one minus the loading of the primary on the second-order squared. In other words, it is obtained by multiplying by that part of the primary that is not explained by the general.
We conducted SL orthogonalizations, extracting one general and two3
, three, and four group factors.4
These results were inspected for: a) identification (are there at least three items with simple loadings for each group factor?), b) substantive interpretability, c) cross-loadings (which suggest the item is a blend of two group factors and problematic for restricted bifactor models). Most importantly, based on the SL results, we identified which group factor an item loaded highest on. We then estimated restricted bifactor models using MPLUS based on specifying that each item loads on the general factor and one group factor.
Unfortunately, our results indicated that neither the two, three, nor four group factor restricted bifactor models appears promising for IRT application. There were several notable problems. First, in running MPLUS we ran into estimation problems, especially in the two group factor model. For example, when an item’s loading on a group factor was low, we obtained zero or negative parameter estimates. Beyond technical problems, we also ran into problems in identifying the group dimensions. For example, in the four group factor model, there were not enough items with relatively large and simple loadings to uniquely identify two of the four group factors. Yet, the most daunting problem was the presence of cross-loading items. Specifically, when an item cross-loads in the SL solution (i.e., loads on more than one group factor), not only does that suggest model mis-specification, but forcing that item into a restricted bifactor inflates the item’s loading on the general factor and deflates its loading on a group factor.
To illustrate the problem, in the second column of we re-display the factor loadings from a unidimensional model. In the next set of columns are the SL results for the three group factor model. This three group factor model is displayed because it was the least problematic to identify and estimate, it is the most interpretrable, and displays a good fit. Notice that the loadings on the general factor in the SL solution are lower than the unidimensional solution. This illustrates that: a) loadings in a unidimensional solution are inflated due to multidimensionality and, b) the SL structure nicely accounts for that bias by shifting the multidimensionality over to the group factors. Also, note that in the SL around a dozen items have cross-loadings, especially for group factor three.
Unidimensional, Schmid-Leiman Orthogonalization, and Confirmatory Bifactor Analysis for Three Group Factor Model
Finally, in the next set of columns are the results of the restricted ("confirmatory") bifactor model with three group factors. As noted above, each item was assigned to a group factor based on its highest loading in the SL. The general factor in the restricted bifactor accounts for around 22 percent of the variance and the three group factors represent 3.5, 5.5, and 4.2 percent (around 13%), respectively (unexplained variance is around 65%). Substantively, we interpret the three group factors to represent: a) lack of interest in others, b) lack of warm attachments, and c) preference for solitude. The statistical fit of this three-group factor model is very good: CFI=.97 and RMSEA=.03. However, a good fit indicates only that the loadings recover the original tetrachoric correlation matrix closely and not that the parameters are correctly estimated. In fact, a major cause for concern with this model is the potential distortion in the parameter estimates caused by forcing cross-loading items into a highly constrained model.
For example, unlike the SL results, in the restricted model several items load higher on the general factor than their loadings in the unidimensional (or SL) solution – implying that controlling for multidimensionality via the restricted bifactor makes the item a better measure of the common trait! By inspecting the SL results, it is clear that this phenomenon occurs when items have significant cross-loadings. For example, item #19 has loadings of .28, .32, and .10 on the three orthogonal group factors in the SL. When this high communality item is forced into a one general and a single group factor model, the restricted bifactor model assigns that item’s common variance mostly to the general factor.
In short, the constrained bifactor model does not (mathematically) know what to do with an item with high communality and cross-loadings on two or more dimensions. Thus, when a restricted bifactor model is estimated, the model assumes that the item is a strong measure of the general factor. In turn, if the loading on the general factor is biased high, then there is less communality left for the item to load highly on a group factor. Most importantly in terms of practice, not only are the item parameters wrong, but an item could (artificially) look like a great indicator of the general trait and a poor indictor of a group factor (e.g., see items #37 and #11 in ).
In sum, our conclusion is that despite the good statistical fit, we simply cannot trust the parameters of this or any of the other restricted bifactor models considered. Furthermore, the bifactor analyses indicate that the factor loadings in the unidimensional analyses conducted in the prior section are artificially inflated by multidimensionality, which distorts IRT parameter estimates (biased high). Thus, we conclude that the 40 item SAS does not conform well to either Model A or Model B, and thus does not have a coherent latent structure – i.e., Model C.