|Home | About | Journals | Submit | Contact Us | Français|
To provide a clinically useful means of interpreting change for individual patients on the Neurological Quality of Life (Neuro-QoL) adult short forms (SFs) by applying a Classical Test Theory concept for interpreting individual change.
Secondary analysis of existing data.
Persons with neurological conditions including stroke, epilepsy, amyotrophic lateral sclerosis, multiple sclerosis, and Parkinson’s disease residing in community settings.
Neuro-QOL SFs for Applied Cognition-General Concerns, Applied Cognition-Executive Function, Applied Cognition-Combined, Ability to Participate in Social Roles and Activities, Satisfaction with Social Roles and Activities, Positive Affect and Well-Being, Depression, Stigma, Upper Extremity Function (Fine Motor, Activities of Daily Living), Lower Extremity Function (Mobility), Anxiety, Sleep Disturbance, Fatigue, and Emotional and Behavioral Dyscontrol.
We estimated conditional minimal detectable change (cMDC) indices from the pooled standard errors (SEs) adjusted for a 95% confidence interval (CI) using the average of the SEs for any given pair of scores multiplied by the z-score, or [(SEScore1 +SEScore2)/2 · 1.96].
The cMDC indices are generally smallest in the mid-range of all scales, ranging from 3.6 to 11.2 T-score points, and higher on the outer quartiles ranging from 3.7 to 21.6 T-score points. The lowest mid-range cMDCs were for Satisfaction with Social Roles and Activities (3.6–4.7 T-score points) and the largest was for Sleep Disturbance (9.4–11.2 T-score points).
Change indices can help clinicians and investigators identify differences for individual patients or subjects that are large enough to motivate treatment change. cMDCs can reduce misclassification of magnitudes of change that are near the margins of error across the range of the Neuro-QoL SFs.
Evaluating change across time is critical to decision making in rehabilitation research and clinical practice. However, evaluating clinical change implies an ability to measure change at the individual patient level,1–3 rather than at the group-level, as is typical in most research. In the assessment of individuals, Liang differentiated between sensitivity to change, defined as measured change regardless of its clinical meaningfulness, and responsiveness, defined as measured change that is clinically meaningful.1 In this framework, sensitivity to change describes distribution-based indices such as minimal detectable change (MDC), smallest detectable change, and minimal real change,2 while responsiveness describes externally anchored indices like minimal clinically important difference (MCID) and clinically relevant change.2 In brief, distribution-based indices describe the smallest change that exceeds an instrument’s measurement error, while anchor-based indices represent the smallest change perceived as meaningful or relevant by patients or clinicians.5 However, this framework is not ubiquitous throughout healthcare, and some disciplines consider distribution-based and anchor-based indices to be different forms of responsiveness.2
Reporting of individual change indices for has improved the utility of many classic instruments for use with different patient populations, as demonstrated though instrument reviews available from the Rehabilitation Measures Database.4. Often a single change index value is reported, which assumes that measurement error is consistent across all possible scores for the instrument. Yet, statistically, instruments tend to be more reliable (discriminate better) near the midrange of the scale, and less so at either extreme. Accordingly, the margin of error may vary depending upon where scores fall along the range of the scale. To illustrate this, Stratford and colleagues described a conditional MDC (cMDC) index for the Roland Morris Questionnaire for back pain related disability.5 While the cMDC improves sensitivity to change b, it has only been reported instruments that are scored as the sum of their items, which are ordinal in nature. Newer instruments based on Item-Response Theory (IRT) provide measurement with interval properties and item- or score-level reliability (standard errors (SEs) are reported for each item and for each score across a scale range), presumably making them more sensitive to change.
The Neurological Quality of Life (Neuro-QoL) Measurement System6 was developed using IRT methods primarily for research applications, but the instruments may be ideal for clinical use.
Neuro-QoL instruments have large item banks with wide scale ranges which can be administered by computer adaptive test or short form (SF), and by use of internet-based survey platforms such as Assessment Center.7 However, interpreting change on a Neuro-QoL instrument is based on a reported SE for a pair of scores, which does not account for the overlap of the margins of error between the scores, or for a different confidence interval (CI). We sought to develop change indices for the Neuro-QoL instruments that are analogous to the cMDC, based on the assumption that the SE is analogous to the standard error of measurement (both provide a margin of error for a point score), which is the basis for calculating MDC () on classic instruments. This paper reports these indices for 14 Neuro-QoL SFs.
The Neuro-QoL Measurement System (http://www.neuroqol.org/Pages/default.aspx) comprise a set of self-report instruments of HRQOL for adults and children with neurological disorders, which are available as item banks for computer adaptive tests and as fixed-length SFs. We report on the 14 adult SFs, including Applied Cognition-General Concerns, Applied Cognition-Executive Function, Applied Cognition-Combined, Ability to Participate in Social Roles and Activities, Satisfaction with Social Roles and Activities, Positive Affect and Well-Being, Depression, Stigma, Upper Extremity Function (Fine Motor, Activities of Daily Living), Lower Extremity Function (Mobility), Anxiety, Sleep Disturbance, Fatigue, and Emotional and Behavioral Dyscontrol.
We estimated cMDC from the average of the SEs for any given pair of scores multiplied by the z-score for a given CI (in this case 1.96 for a 95% CI identified as a subscript), which gives [((SEScore1 +SEScore2)/2) · 1.96] using Microsoft Excel 2010.a Change scores were determined for each pair of T-scores on each of 14 Neuro-QoL SFs (13 Version-1 SFs plus the composite Version-2 Applied Cognition SF) and compared to the estimate for the cMDC(95) for that pair of scores. Change scores that do not meet the MDC(95) are those that do not exceed the pooled adjusted margin of error. All data were obtained from the publicly available Neuro-QoL User’s Manual (http://www.neuroqol.org/Resources/Neuro-QoLReports-Manuals/Pages/default.aspx),8 which details the instrument design and sampling procedures.
The cMDC values for the middle half and end quartiles the scale ranges are summarized for the 14 Adult Neuro-QoL short forms (table 1), and the index table for the Anxiety SF is provided as an example (figure 1). Index tables which include interactive cMDC calculators are provided for all 14 adult Neuro-QoL SFs in an Excel workbook (online supplement). The raw scores, T-scores, and the associated SEs increase across the top and down the left side of the table. To interpret change between two scores for an individual using the table, find the corresponding raw scores across the top (Time 1) and down the left side (Time 2) and locate the intersecting cell. If the cell is highlighted (grey in figure 1 or red in the online supplement) the scores probably do not represent a detectable change. Scores above the highlighted zone represent an increase on the construct (i.e., worsening Anxiety in figure 1), and scores below the zone represent a decrease. To use the calculator, enter the Time 1 and Time 2 scores and the resulting change score and an interpretation will be returned.
With a few exceptions, the cMDCs are smallest in the mid-range of the scales (table 1). The lowest mid-range cMDC was for Satisfaction with Social Roles and Activities and the largest was for Sleep Disturbance. The Upper Extremity Function and Applied Cognition – Executive Function SFs showed low cMDCs (6.1–6.4 and 6.7–7.1 T-score points respectively) at the low end of the scale range due to small SE values.
We aimed to facilitate clinical application of the Neuro-QoL SFs by developing distribution-based change indices at the individual level. Our method was similar to that reported for classic instruments.5. Instruments based on IRT have an advantage over classic instruments in that SE is estimated for each item or for each score across their scale ranges, which allows more granular reporting cMDC indices. We have not seen this concept applied to an IRT instrument, but have found single MCD indices which assume error is constant across a scale range reported IRT-based instruments.9, 10
The cMDC values aid one to reliably consider whether an observed change for an individual exceeds measurement error. These values should not be applied to group-level changes which may actually be smaller in magnitude than the cMDC. These change indices will benefit clinicians by facilitating care decisions based on whether or not patients have experienced true change. Clinicians can be confident that the change is not attributable to measurement error when an individual’s change score exceeds an MDC value. However, applying a single MDC across an entire scale range may result in misclassification Variability in SEs, and erroneous inferences of change at scale ends (or of no change in the mid-range) could affect decisions to continue or discontinue care.
A limitation of this study is that the cMDC values do not indicate whether a measured change is clinically important. The Neuro-QoL T-scores are interpretable using a population-based norm for each domain represented by a score of 50 with a SD of 10 points on an interval scale.6, 8 Thus, importance may be inferred by the magnitude of change relative to the norm, once one is confident that the scores represent change that exceeds measurement error. Future efforts could develop anchor-based MCID indices for Neuro-QoL instruments and develop cMDC tables and calculators for other IRT-based instruments.
Change indices can help clinicians and investigators identify differences that are large enough to motivate treatment change. cMDCs may reduce misclassification of magnitudes of change that are near the margins of error across the range of the Neuro-QoL SFs.
The study was supported from grants from the National Institute of Neurological Disorders and Stroke (NINDS) contract number HHSN265200423601C (PI: David Cella) and the National Institute on Disability and Rehabilitation Research (NIDRR), grant number H133B090024 (PI: Allen Heinemann). The contents represent original work and have not been published elsewhere. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated.
aMicrosoft Excel 2010, version 14.0.7145.5000 (32-bit), Microsoft Corporation, One Microsoft Way, Redmond WA, 98052
Reprints are not available.
We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated AND, if applicable, we certify that all financial and material support for this research (e.g., NIH or NHS grants) and work are clearly identified in the title page of the manuscript. (Allan J. Kozlowski, David Cella, Kristian P. Nitsch, Allen W. Heinemann)
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.