Agreeing with Crosby et al. [7
], who advocate a combination of anchor-based and distribution based approaches, we not only combine the results of the two approaches, but also integrate them. We call this method anchor-based MIC distribution
. Using an anchor, we divide a population into three groups: importantly improved, not importantly changed, and importantly deteriorated. We then plot the distribution of the change in scores on the health status instrument (Figure ). We assess the MIC for improvement and for deterioration separately, as these can differ [7
]. Next, we choose the cut-off point for an MIC. Here we will consider two cut-off points: the Receiver Operating Characteristic (ROC) cut-off point and the 95% limit cut-off point.
Distributions of the changes in scores on the health status instrument for persons who report important improvement and those who report no important change on the anchor. ROC = Receiver operating characteristic.
The ROC cut-off point is based on an ROC analysis, as applied in diagnostic studies. In this context, the health status instrument at issue is considered the diagnostic test, and the anchor functions as the gold standard [11
]. The anchor distinguishes persons who are importantly improved or deteriorated from persons who are not importantly changed. The instrument’s sensitivity is the proportion of importantly improved/deteriorated persons according to the anchor, who are correctly identified by the health status instrument as importantly improved/deteriorated. Its specificity is the proportion of ‘not importantly changed’ persons according to the anchor, who are correctly identified as ‘not importantly changed’ by the health status instrument. The ROC cut-off point is the value for which the sum of percentages of false positive and false negative classifications ([1-sensitivity] + [1-specificity]) is smallest. Note that the assumption in this is that false positive and false negative results are equally unwanted.
The 95% limit cut-off point is based on the distribution of persons who are, according to the anchor, not importantly changed. The underlying concept is that the MIC should be detectable beyond measurement error. In other words, one might be reluctant to label persons who show no important change between the two occasions of measurement according to the anchor as importantly improved/deteriorated on the health status instrument. Using the 95% limit cut-off point, MIC for improvement is defined as the 95% upper limit of the distribution of the persons who are not importantly changed according to the anchor [mean change + 1.645 SDchange1
]. Note that the 95% limit cut-off point corresponds with 95% specificity on the ROC curve.
Graphing the distribution allows one to judge how well an instrument distinguishes persons who, according to the anchor, are importantly improved or deteriorated from those not importantly changed. Moreover, the distance between the ROC cutoff point and the 95% limit cut-off point are clearly illustrated. Thus, the graph is important for seeing how the choice of a specific cut-off point influences the amount of misclassification. A flatter curve suggests a weaker correlation between anchor and health status instrument under study. Furthermore, differences in location and form of the curves of the ‘improved’ and ‘deteriorated’ persons indicate that the MICs for deterioration and improvement differ. In our theoretical example, considering the ROC cut-off points, the MIC for deterioration is larger than that for improvement, meaning that negative changes in scores must be larger than positive changes before persons think of themselves as importantly changed. Using the 95% limit cut-off point, the MIC values for improvement and deterioration are the same as long as the persons showing no important change on the anchor have a mean value of 0 on the health status instrument, and their values show a normal distribution: then both points are found at 1.96 * SD of the change scores of the not importantly changed
group. Note that the distribution of the importantly improved/deteriorated groups have no influence of the 95% limit cut-off point. A larger MIC for deterioration than for improvement was, for example, observed for all subscales of the Functional Assessment of Cancer Therapy instrument in cancer patients [14
]. However, using an 11 point numerical rating scale to measure pain intensity, Farrar et al. [15
] showed a smaller MIC for deterioration than for improvement.
Before presenting our example, we should emphasize that this anchor-based MIC distribution method provides a general framework, which can be applied to all kinds of anchors and definitions of minimal importance.