PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Arch Phys Med Rehabil. Author manuscript; available in PMC 2014 February 3.
Published in final edited form as:
PMCID: PMC3910090
NIHMSID: NIHMS410151

The Spinal Cord Injury- Functional Index: Item Banks to Measure Physical Functioning of Individuals with Spinal Cord Injury

David S. Tulsky, Ph.D.,a Alan Jette, PT, Ph.D.,b Pamela A. Kisala, M.A.,a Claire Kalpakjian, Ph.D.,a Marcel P. Dijkers, Ph.D.,c Gale Whiteneck, Ph.D.,d Pengsheng Ni, Ph.D.,b Steven Kirshblum, M.D.,e,f,h Susan Charlifue, Ph.D.,d Allen W. Heinemann, Ph.D.,g Martin Forchheimer, M.P.P.,a Mary Slavin, PT, Ph.D.,b Bethlyn Houlihan, M.S.W., M.P.H.,b Denise Tate, Ph.D.,a Trevor Dyson-Hudson, M.D.,f,h Denise Fyffe, Ph.D.,f,h Steve Williams, M.D.,h and Jeanne Zanca, Ph.D., MPTc

Abstract

Objective

To develop a comprehensive set of patient reported items to assess multiple aspects of physical functioning relevant to the lives of people with spinal cord injury (SCI) and to evaluate the underlying structure of physical functioning.

Design

Cross-sectional

Setting

Inpatient and community

Participants

Item pools of physical functioning were developed, refined and field tested in a large sample of 855 individuals with traumatic spinal cord injury stratified by diagnosis, severity, and time since injury

Interventions

None

Main Outcome Measure

SCI-FI measurement system

Results

Confirmatory factor analysis (CFA) indicated that a 5-factor model, including basic mobility, ambulation, wheelchair mobility, self care, and fine motor, had the best model fit and was most closely aligned conceptually with feedback received from individuals with SCI and SCI clinicians. When just the items making up basic mobility were tested in CFA, the fit statistics indicate strong support for a unidimensional model. Similar results were demonstrated for each of the other four factors indicating unidimensional models.

Conclusions

Though unidimensional or 2-factor (mobility and upper extremity) models of physical functioning make up outcomes measures in the general population, the underlying structure of physical function in SCI is more complex. A 5-factor solution allows for comprehensive assessment of key domain areas of physical functioning. These results informed the structure and development of the SCI-FI measurement system of physical functioning.

Keywords: Spinal Cord Injuries, Mobility Limitations, Outcome Assessment (Health Care), Activities of Daily Living, Psychometrics, Walking, Self Care, Quality of Life

A major treatment goal in the rehabilitation of persons with spinal cord injury (SCI) is to maximize the restoration of physical functioning. Documenting the extent of recovery is imperative for: 1) assessing treatment efficacy; 2) evaluating the cost-effectiveness of treatment interventions; 3) examining the impact of policy changes on patient outcomes; 4) evaluating the quality of care being provided; and 5) providing appropriate, long-term prognostic information to patients and their families, as well as to insurance carriers.13 In order to document recovery of rehabilitation interventions, reliable and valid tools are necessary to assess physical functioning outcomes in the SCI population.

Several outcomes measures are currently used to assess physical functioning in SCI. 413 The most commonly used scales (e.g., Functional Independence Measure) have two important shortcomings with respect to their use in this population: comprehensiveness of the measure’s content to assess the full range of SCI severity and the breadth of content to ensure all important aspects of physical functioning are covered, including the perspective of individuals with SCI in assessing outcomes. It is difficult for any single instrument to include the large number of items necessary to cover the range of severity levels seen among persons with SCI. For example, an instrument designed for use in individuals with high level tetraplegia is not likely to have sufficient range to be meaningful if used with individuals with paraplegia. Yet, comprehensive range is an essential measurement property for an outcome instrument designed to assess change in physical functioning following any level and completeness of SCI. The current measures of physical functioning vary in terms of both their range and the breadth of physical functioning items covered; for instance, the Walking Index for Spinal Cord Injury (WISCI)9 focuses only on ambulation and the Quadriplegia Index of Function (QIF)13 on physical functioning among individuals with tetraplegia. The Spinal Cord Independence Measure (SCIM-III)14 and Functional Independence Measure (FIM)4 focus on a greater breadth of physical functioning, but this breadth does not necessarily cover all aspects of physical functioning. For example, none of the current measures assesses fine motor functioning apart from that which may be implied by self-care. Measures developed for the general population often assess ambulation wholly in terms of mobility, while for persons with SCI, the use of a wheelchair (which some consider an extension of their bodies15) constitutes mobility.

An important shortcoming of existing measures is the failure to incorporate the perspective of individuals with SCI in the item development process and the content of the current measures (e.g. FIM, QIF, etc) was selected by rehabilitation clinicians and researchers. These measures rely exclusively on performance, observation and the use of clinician raters. While professional observer rated methods provide an important source of information, there are many activities that cannot be assessed directly and observational methods are subject to bias by raters. Patient reported outcomes (PROs) provide an alternative method of assessment and are increasingly recognized as an essential component of outcome measurement in clinical research and clinical practice. A PRO is defined as any report on a patient’s health condition or health status that comes directly from the patient, without the interpretation of the response by a clinician or anyone else.16 Many have argued that PROs are essential to the measurement of outcomes and should be collected alongside performance-based measures.1719

In response to growing recognition of the value of PROs, the National Institutes of Health (NIH) initiated the Patient Reported Outcomes Measurement Information System (PROMIS) in 2004, to develop new measures for use in clinical research and health care delivery.20 In a parallel project, to address the specific health related QOL issues relevant to populations with neurological disorders, the Neuro-QOL system was developed.21 Despite the broad scope of their work, PROMIS and Neuro-QOL did not address issues specific to the SCI population. Like other existing instruments, the PROMIS and Neuro-QOL instruments conceptualize physical functioning as either one general domain (PROMIS, overall physical functioning)22 or two (Neuro-QOL, overall physical functioning and upper extremity functioning)23. With SCI, however, given the nature of the injury, impairments impact many areas of functioning in different ways, leading one to conclude that physical functioning is a multi-factorial construct24 and it is questionable if a one- or two-factor measurement scale is adequate, especially for those clinical and research situations where a detailed assessment of the person’s abilities and disabilities in multiple aspects of physical functioning is needed.

The aims of this study were to develop a comprehensive set of items that assess multiple aspects of physical functioning relevant to the lives of people with SCI and to evaluate the underlying conceptual structure of physical functioning in these individuals. Specifically, we sought to determine if a one- or two-factor model of physical functioning, or a more complex model, best summarizes the empirical SCI-FI data and is therefore most appropriate for use in an SCI population. If distinct dimensions of functioning are shown to exist, item banks to measure each subdomain separately are necessary. The current work sets the stage for item response theory analyses and the development of a computerized adaptive test (CAT) for each item bank, modern methods to create and administer measures that are reported in a related paper.25 The instrument we developed is called the Spinal Cord Injury – Functional Index (SCI-FI).

METHODS

Development of the SCI-FI item pools

SCI-FI development (Figure 1) began with a thorough literature review, input from focus groups conducted with individuals with SCI and SCI clinicians, and feedback from experts in SCI rehabilitation. As reported by Slavin et al.,15 12 focus groups with individuals who had SCI and 6 with SCI clinicians were held at 6 SCI Model System (SCIMS) centers, and a rigorous qualitative analysis was conducted to extract potential activities, skills and tasks to be included in SCI-FI.26 Focus group feedback was used to identify important domains and subdomains of physical function, including several categories of mobility (manual wheelchair, power wheelchair, ambulation, and transfers/changing body position), self-care and upper extremity functioning (fine hand use/manipulating objects, lifting objects, reaching/hand and arm use, toileting), sexual functioning, and use of communication devices.24

Figure 1
SCI-FI Item Bank Development Process

A central goal in developing the item pool was to construct direct linkages with some existing PRO scales, which would be accomplished by placing items from these different existing PRO measurement tools on a common metric and creating a statistical link between the scores of the two instruments.27 By embedding items from Neuro-QOL and PROMIS verbatim into the SCI-FI, a statistical link can be created to convert SCI-FI scores to scores yielded by those measures, through score conversion tables. Such linking is important when scores are being compared with those obtained in other population and groups. For the SCI-FI study, the PROMIS and Neuro-QOL item banks were reviewed for content that overlapped with the suggestions made in the focus groups with SCI individuals and clinicians and where appropriate, PROMIS and Neuro-QOL items were embedded verbatim into the pool of candidate items of the SCI-FI measure. PROMIS contains a large item bank of 124 calibrated items measuring a single, general factor of physical functioning22 while Neuro-QOL contains two item banks measuring upper extremity (20 items) and mobility (19 items), respectively.23 Following a detailed review of the PROMIS and Neuro-QOL items, a total of 33 PROMIS items and 37 Neuro-QOL items (19 of which were common to both measures) were selected based on content relevance. The Neuro-QOL item banks incorporate a series of items from the Activity Measure for Post Acute Care (AM-PAC),27,28 therefore, these AM-PAC items were also included in the SCI-FI. By including the AM-PAC and the Neuro-QOL items within the SCI-FI and co-calibrating all items using item response theory, it makes it possible to create linking lookup tables so that investigators can obtain a total score equivalent for either the AM-PAC or Neuro-QOL even though only a subsample of items for either measure are contained within the SCI-FI CAT. Furthermore, items were incorporated from a pediatric SCI-targeted measure of physical function29,30 which will allow linking across the lifespan. This linking is important when children with SCI are enrolled in studies and will be retested longitudinally into adulthood. The final linkage step was to ensure that there were common items with a PROMIS physical functioning supplemental scale for individuals who utilize adaptive technology (AT), which was being developed concurrently with the SCI-FI .17

The current investigators added to these “legacy items” a total of 619 new items based on the new physical functioning content suggested by the focus groups. After investigators made an initial pass at deleting items for redundancy, several steps were taken (see Figure 1) to ensure the content validity of the remaining new items (the legacy items had undergone similar scrutiny previously). First, cognitive debriefing interviews were conducted with individuals with SCI who in a structured interview read each item, responded based upon their level of functional ability, and then reviewed the meaning of the item and their thought process behind their response. A minimum of five individuals with SCI reviewed each newly written item and discussed the language, comprehensibility, ambiguity, and, most importantly, the relevance of the item to individuals with SCI. The cognitive interview process we used has been described by DeWalt et al.31

As a second step to ensuring that the phrasing of the new items was robust and would be amenable to future Spanish translation, a team of native Spanish speaking translation science experts reviewed each item for “translatability” to Spanish and provided feedback on specific words and phrases that would “lose meaning” upon translation. This team identified multiple instances of problematic wording and grammar, and in each case suggested an alternative way to phrase the item so that the final scale would be translation-ready.

A third step was to perform a reading level analysis, reviewing all of the new items for readability, ensuring that no item was written above a third grade reading level. We used the Lexile Framework™32 to evaluate each item. Items were re-written if the review identified that they were at too high a reading level.

At this stage, the 124 legacy items were merged back in with the 204 final “new” items, for a complete item pool consisting of 328 items. Finally, the co-investigators reviewed the item banks and suggested wording changes to new items and “binned” the items into subdomains of physical functioning (e.g. changing and maintaining body position, transfers, walking and running, wheelchair mobility, bathing, eating, grooming, toileting, sexual functioning, fine hand use / manipulating objects, and use of communication devices; see Figure 1).24 The qualitative review process, from focus groups and item development through item binning, suggested that the structure of physical functioning in individuals with SCI is more complex than a simple one or two factor model and the next phase of the study (described below) evaluated alternative, more complicated, structures of physical functioning.

Calibration Study

Participants

The item calibration sample used for field testing included 855 participants with traumatic SCI recruited from six SCIMS centers: New England Regional SCI Center, University of Michigan Model SCI System, Northern New Jersey SCI System, Rocky Mountain Regional SCI System, Mount Sinai SCI Model System, Midwest Regional SCI Care System. The institutional review board at each site reviewed and approved this study. Inclusion criteria specified that participants were 18 years or older and could speak and understand English fluently. The sample was stratified by level (paraplegia vs. tetraplegia) and completeness of injury (complete vs. incomplete) as well as time since injury (TSI; < 1 year, 1–3 years and > 3 years) to ensure that the sites recruited a heterogeneous sample of individuals with SCI.33

Data Collection Procedures

All items were presented to participants by a trained data collector in an interview format, either in person or over the phone. Since some items (e.g. ambulation, wheelchair use, bowel and bladder management) only apply to certain subgroups of individuals, participants first answered screening questions to determine the need to administer these “supplemental items” beyond the basic set administered to every respondent. Participants were asked to respond to each item based on their capacity to perform the activity without special equipment or help from another person, except when such help was explicitly stated in the question. Participants were shown a response card to help guide them through the interview, as the response options differed somewhat from one block of items to the next. All responses were entered into a web-based, data collection system which allowed data to be automatically uploaded, and stored on a secure server immediately. The study team reviewed the data for quality and for adherence to the sampling stratification quotas.

Data Analysis

Based upon the focus group feedback regarding the important subdomains of physical functioning in SCI,24 several competing models of physical functioning were developed and tested. Confirmatory factor analysis (CFA) was used for this analysis as it allows for direct comparison of alternative models. Because of the ordinal nature of the items, the CFAs were conducted based on polychoric correlation matrices with weighted least squares with mean and variance adjustment (WLSMV) parameter estimation.

To explore the factor structure, different models of increasing complexity were created, fit to the data and compared to each other using a competing models approach. Several indices of model fit for each of the more complicated models were compared against the fit indices of a general one-factor model of physical functioning and a two-factor model (specifying a general mobility factor and general self-care factor). The one-factor model parallels the PROMIS structure while the two-factor model parallels the Neuro-QOL structure. Several additional, more complex models were also outlined that reflect constructs outlined in the ICF model34 of impairments, activity limitations and participation restrictions and distinctions of function that were suggested by our qualitative review of focus group feedback.

Each successive measurement model was evaluated in terms of indices of goodness-of-fit. These indices were selected because they are moderately sensitive to effects of sample size and degrees of freedom.3538 The Chi-square index divided by degrees of freedom (χ2/df) was used to calculate the Tucker–Lewis Index (TLI), which is a non-normed comparative fit index that makes adjustment for the number of degrees of freedom in the model.39 Bentler’s Comparative Fit Index (CFI),40 which also compares model specifications to baseline models, was calculated. For both the TLI and CFI, values greater than 0.90 indicate good model fit, with those above 0.95 indicating extremely good model fit.37 In addition, the root mean square error of approximation (RMSEA),41 which compensates for model complexity, was calculated. Whereas exact fit to a model would be indicated by a RMSEA value of 0.00, values of less than 0.08 indicate reasonable model fit while values of 0.06 or lower indicate a very close fit.

To prepare for the confirmatory factor analyses, a final content analysis of the items was performed to evaluate the authors’ “binning” of items and to select the most representative items for the initial analyses which is described below. Model specification proceeded parallel to this effort and items from the binning were included within the specified models. Within the self-care and fine motor item banks, there were three content-specific subdomains, related to communication, toileting, and sexual functioning, respectively. Rather than develop these areas of functioning as separate item banks, the research team binned these items along with other self care and fine motor items, since the item content focused on the use of upper extremity or, more specifically, fine motor movement that is involved in each of these activities (e.g., using the keypad on a touch-tone phone). Because of the inclusion of gender-specific items (e.g. shaving, tampon use) the Self Care items completed by males and females varied slightly and therefore this item bank was assessed separately by gender. Model specifications can be found in Table 1.

Table 1
Model Specifications for Confirmatory Factor Analysis

A limitation of the CFA procedure is that given the large pool of 328 items, a sample size of 855 participants is not sufficient for model identification; there are too many items to be analyzed simultaneously. However, the items were believed to demonstrate sufficient redundancy and it was assumed that model evaluation could proceed with a subset (approximately 1/3) of the items. Therefore, items were reviewed and a subset of 119 items selected for the initial analyses, with representative items (i.e., those items that were expected to have the highest loading on each respective factor) being selected from among all of the bins (or domains) that are described above. The initial evaluation included eight alternative factor models. To determine the robustness of the factor structure and ensure that the initial results were not an artifact of the item selection process, the models specified in the CFAs were then retested using a second subset of 115 items, selected from across all of the bins or domains that were not included in the initial analyses, and the model fit statistics for the eight alternative factor models were compared between the two item subsets. This replication analysis allowed us to determine the robustness of the factor structure and ensured that the initial results were not an artifact of the item selection process.

In preparing the data for analysis, we imputed all responses to the walking items as “unable to do/cannot do” for subjects with complete injuries who responded that they were “never able to walk” in the relevant screening question. This allowed us to include them in the overall analyses. Moreover, a large percentage of the sample used a wheelchair or ambulated, but do not do both. As the alternative factor models were developed, there was concern that the strong inverse relationship between Wheelchair and Ambulation items would impact the factor structure. Specifically, the lack of covariance between the ambulation and wheelchair mobility factors in SCI populations could cause model identification problems when the factor structure was tested through structural equation modeling or confirmatory factor analytic methods.

Consequently, we decided that some models were to be tested including the ambulation items but without the wheelchair items or, alternatively, tested with the wheelchair items while excluding the ambulation items.

Following these initial CFAs, each bank was analyzed separately to assess its unidimensionality. Rather than pit successive models against one another, for this analysis single unidimensional models were run for each of the factors identified as significant in the previous step (e.g., basic mobility). These subdomain analyses were conducted and evaluated using the fit statistics described above. All CFAs were conducted using MPlus version 6.0.a To ensure that the item banks met the assumptions necessary for subsequent Item Response Theory (IRT) analysis25, it was necessary to assess the local independence of items in each bank, which means that for individuals with the same level of the construct being measured, responses to each item should be independent of one another.42 If this is not the case, something other than the construct being measured is influencing the variation in a cluster of two or more items, which violates the assumption of unidimensionality. To test for local item dependence (LID), residual correlations between items were examined, and items exhibiting absolute residual correlations greater than 0.2 were removed.

RESULTS

As shown in Table 2, the SCI-FI item pool for field testing comprised 328 items, including 204 new, SCI-specific physical functioning items, 37 Neuro-QOL Physical Health items, 25 AM-PAC items, 33 PROMIS Physical Functioning items, 57 Shriners Pediatric SCI-CAT items, and six PROMIS Assistive Technology items.

Table 2
SCI-FI Item Distribution by Source

Demographic data for the calibration study participants are shown in table 3. Approximately 46% of the individuals were diagnosed with paraplegia and 54% with tetraplegia; 46% of the sample had complete injuries and 54% had incomplete injuries. Finally, 31% of the sample was injured less than a year, 28% were injured 1–3 years, and 41% had long-term injuries (> 3 years). The sample closely matches the proposed stratification by injury type, injury severity, and time since injury.

Table 3
Background Characteristics of the Sample

The model specifications were outlined in Table 1, while the results of the model assessments for the initial subset of items (i.e., k=119) are shown in Table 4. All of the assessed models had very good fit to the model as assessed by CFI, TLI and RMSEA. Model 1, the one-factor model of physical functioning, had fit statistics demonstrating acceptable model fit. Model 2, the two-factor upper extremity and mobility structure, showed a slight improvement over the one-factor model, indicating that it also had exceptional fit to the data. Models 3 and 4, alternate three-factor models, also had good model fit, but they were not superior to the two-factor solution.

Table 4
Initial Confirmatory Factor Analysis for SCI-FI – Sample of 119 Items

Models 5 and 8 have a more complicated structure where mobility is divided into basic mobility, ambulation, and wheelchair mobility. These models hadinadmissible statistics above 1, indicative of model specification errors. This result is not surprising given the nature of SCI, where a substantial proportion of the participants either do not ambulate or do not use a wheelchair. As a result, the variance in these item banks is zero, making it difficult to analyze wheelchair mobility and ambulation within the same model. Thus, we developed alternative models that removed either wheelchair or ambulation items or the corresponding subdomain factors from the model (Models 5a and 8a (excluding ambulation items) and Models 5b and 8b (excluding wheelchair mobility items) demonstrate improved model fit. Models 8a and 8b have among the best fit statistics of all of the models. However, it is unclear if this improvement in fit alone would justify adoption of Models 8a/8b specificallyas all models are supported by the fit statistics. However, models 8a/8b are more representative of the focus group data and expert input. All of the multi-factor models demonstrated slight improvement in model fit over the one-factor model.

We replicated the model tests with the second subset of items (k=115; Table 5). Like before, all models demonstrated very good fit. Models with both Ambulation and Wheelchair items on separate factors did not converge and the co-variance matrices were not positive definitive. Models 8a and 8b demonstrate slightly improved model fit. These models do, however, conform to the structure suggested in the focus groups with distinct factors of self-care, fine motor, basic mobility, ambulation, and wheelchair mobility.

Table 5
Replication of Confirmatory Factor Analysis for SCI-FI – Sample of 115 Items

The last set of CFA analyses determined whether the basic mobility, wheelchair mobility, ambulation, self care, and fine motor domains were sufficiently unidimensional to be developed as calibrated item banks. For example, the 54 items that had been “binned” into the basic mobility subdomain were analyzed in a unidimensional analysis. Table 6 summarizes the results of each model. Unlike the previous analyses where fit statistics were compared with one another, the fit statistics in Table 6 are not meant for comparative review.

Table 6
Goodness-of-Fit Indices for Unidimensional CFAs for the SCI-FI Final Item Banks*

Following the unidimensional confirmatory factor analyses, we removed several items due to LID (10 items), missing data (7 items), and content concerns (5 items). Each of these item pools demonstrates exceptionally good fit, supporting their unidimensionality. The CFI and TLI for all banks exceeded 0.929 and for the Ambulation and Fine Motor banks, these CFI and TLI were 0.999 and 0.998, respectively. Similarly, RMSEA was less than 0.08 for all banks except Basic Mobility, for which it was 0.081.

DISCUSSION

The SCI-FI marks a significant advance in measuring Physical Functioning in individuals with SCI. Its content is based upon direct feedback from individuals with SCI as well as SCI clinicians and psychometric analyses25 provide evidence of the content validity of the SCI-FI. We followed the PROMIS standards for scale development throughout the qualitative and quantitative phases of this study. No other measure of physical functioning is as comprehensive and uniquely tailored to the needs of individuals with SCI as the computer adaptive SCI-FI measure, with its 5 distinct, unidimensional item banks: Basic Mobility, Wheelchair Mobility, Ambulation, Self-care, and Fine Motor, respectively.

We sought to develop patient reported physical functioning item banks that are both targeted to individuals with SCI and are linked with new “generic” physical functioning measures, allowing cross-condition comparisons. By using items from the legacy item pools verbatim, scores derived from the SCI-FI can be linked to scores generated from the PROMIS, Neuro-QOL, AM-PAC, and Shriner’s Pediatric SCI-CAT measures. An example of this methodology for and potential of this type of linking can be found in Haley et al’s (2011)27 linkage of the AM-PAC to the Neuro-QOL. However, such an extensive series of linkages has never been accomplished before, and allowing researchers and clinicians the ability to translate scores from one instrument to another will greatly facilitate comparisons between different medical populations as well as between pediatric and adult SCI samples.

The five-factor model (model 8) that was tested in this study included both ambulation and wheelchair mobility and yielded model specification errors (i.e., the model was “not positive definite”), signaling the presence of inadmissible parameter estimates.43 Inadmissable estimates can result from issues such as sampling variation, outliers, small samples with only two indicator variables per factor, linear dependency of one variable on another, or model specification error.35 In this case, model specification error was likely to be the problem as the correlation between Ambulation and Wheelchair Mobility took on an impossible value (e.g. .012). This finding is likely due to the mutual exclusivity of responses to walking and wheelchair items, respectively - an artifact of the variety of consequences of SCI, where many people who use wheelchairs for mobility are unable to walk at all while others, who are able to walk, do not use a wheelchair. The results from models 8a and 8b support this interpretation. Specifically, when the items related to wheelchair mobility only are tested (model 8a) or items related to ambulation only are tested (model 8b), the fit statistics support the inclusion of ambulation and wheelchair mobility, respectively, as distinct from basic mobility. Conceptually, ambulation and wheelchair mobility are critical functional abilities for individuals with SCI and both clinicians and researchers will have need for separate assessments. As a result, the 5 factor model is preferable for use in an SCI population.

The CFA examination of the unidimensionality of the five final item banks (Table 6) demonstrated that all of the items within each bank form a unidimensional hierarchy. This further supports the five-factor model and provides support for the validity of calibrating the items using IRT and developing as a CAT (see ------- et al. this issue25) where any given respondent will only be asked a fraction of the items included in a given item bank.

The CFA fit statistics support all of the models tested within this study, including the one-factor (model 1) and the two-factor (model 2) models that more closely resemble the PROMIS and Neuro-QOL structures of physical functioning, respectively. CFA fit statistics also support the other models and model fit improves slightly with successive increases in the complexity of the model structure. When mobility is divided into multiple subdomains (i.e., basic mobility, wheelchair mobility, and ambulation) and self-care is separated into a general ADL factor that is distinct from the fine motor ability factor (models 8a and 8b), the fit statistics improve over other models. This more complicated structure (in models 8a and 8b) more closely represents the feedback which was received from both the individuals with SCI and the SCI clinicians who participated in the focus group phase of the study. Together, these results suggest that while there may be a place for one or two factor models when studying physical function across multiple impairment groups, when the clinical or research focus is exclusively on people with SCI, more specific, refined aspects of physical functioning are important in assessing functional ability and ability limitations.

Study Limitations

The five-factor structure was adopted by the research team in the belief that five individual item banks would have more clinical utility in the SCI population. Such clinical utility data do not yet exist and future research will need to test the responsiveness of these 5 item banks to meaningful changes in physical functioning, and assess the utility of separate measures in clinical management. Future research should also compare the results of clinician ratings of physical functioning performance with patient reported outcomes of physical functioning to determine the incremental validity of the SCI-FI.

As discussed earlier, the large number of SCI-FI items (k = 328) makes it impossible to evaluate the factor structure with structural equation modeling of all items in a single analysis. The procedures that were followed represent an approach to evaluate the factor structure in light of this limitation. Selection of a subset of items with varied content across all of the domains and subdomains facilitated examination of the factor structure. These results were replicated with a different subset of items, helping to both ensure the robustness of the CFA results as well as providing assurance that the initial selection of items did not bias the results. The final set of analyses, which examined all of the items within their subdomain to ensure that each item bank was unidimensional, demonstrate that the 5 item banks meet the assumptions for the IRT analyses and can be calibrated and developed as 5 CAT banks.25.

CONCLUSIONS

In summary, the SCI-FI represents an ambitious attempt to develop a patient reported measure of physical functioning in the SCI population. A rigorous item development process occurred that involved literature reviews, integration with previously existing items and scales, expert opinion, and feedback from consumers. Following these procedures, a large multisite field test was conducted in which 855 individuals with traumatic SCI participated. The results of this study support a five-factor solution breaking down physical functioning into key subdomains that are relevant to individuals with SCI and the clinicians who treat them. These analyses and results have informed the structure of the final SCI-FI scale which has been calibrated using IRT and developed as 5 distinct item banks.25

Acknowledgments

This work was supported by US Department of Education, National Institute of Disability and Rehabilitation Research Grant Nos. H133N060022, H133N060024, H133N060014, H133N060005, H133N060027, and H133N060032.

We certify that no party having a direct interest in the results of the research supporting this article has or will confer a benefit on us or on any organization with which we are associated AND, if applicable, we certify that all financial and material support for this research (eg, NIH or NHS grants) and work are clearly identified in the title page of the manuscript.

List of Abbreviations

ADL
Activities of Daily Living
AM-PAC
Activity Measure for Post-Acute Care
AT
Adaptive Technology
CAT
Computer Adaptive Test
CFA
Confirmatory Factor Analysis
CFI
Comparative Fit Index
FIM
Functional Independence Measure
ICF
International Classification of Functioning, Disability, and Health
IRT
Item Response Theory
LID
Local Item Dependence
NIH
National Institutes of Health
PROMIS
Patient Reported Outcomes Measurement Information System
PROs
Patient Reported Outcomes
QIF
Quadriplegia Index of Function
QOL
Quality of Life
RMSEA
Root Mean Square Error of Approximation
SCI
Spinal Cord Injury
SCI-FI
Spinal Cord Injury- Functional Index
SCIM-III
Spinal Cord Independence Measure
SCIMS
Spinal Cord Injury Model Systems
TLI
Tucker-Lewis Index
WISCI
Walking Index for Spinal Cord Injury
WLSMV
Weight Least Squares with Mean and Variance Adjustment

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Suppliers:

Mplus Statistical Analysis with Latent Variables User's Guide [computer program]. Version 6. Los Angeles: Muthen & Muthen; 2007.

REFERENCES

1. Lammertse DP, Jackson AB, Sipski ML. Research from the model spinal cord injury systems: Findings from the current 5-year grant cycle. Arch Phys Med and Rehab. 2004;85(11):1737–1739. [PubMed]
2. Ditunno JF. Outcome measures: evolution in clinical trials of neurological/functional recovery in spinal cord injury. Spinal Cord. 2010 Sep;48(9):674–684. [PubMed]
3. Tator CH. Review of treatment trials in human spinal cord injury: Issues, difficulties, and recommendations. Neurosurgery. 2006 Nov;59(5):957–982. [PubMed]
4. Guide for the Uniform data Set for Medical Rehabilitation (including the FIM instrument) Buffalo, NY: State University of New York at Buffalo; 1997.
5. Keith RA, Granger CV, Hamilton BB, Sherwin FS. The Functional Independence Measure: a new tool for rehabilitation. Advances in clinical rehabilitation. 1987;1:6–18. 1987. [PubMed]
6. Catz A, Itzkovich M, Steinberg F, et al. The Catz-Itzkovich SCIM: a revised version of the Spinal Cord Independence Measure. Disability and Rehabilitation. 2001 Apr;23(6):263–268. [PubMed]
7. Alexander MS, Anderson KD, Biering-Sorensen F, et al. Outcome measures in spinal cord injury: recent assessments and recommendations for future directions. Spinal Cord. 2009;47(8):582–591. [PMC free article] [PubMed]
8. Catz A, Itzkovich M, Tesio L, et al. A multicenter international study on the Spinal Cord Independence Measure, version III: Rasch psychometric validation. Spinal Cord. 2007 Apr;45(4):275–291. [PubMed]
9. Ditunno PL, Dittuno JF. Walking Index for Spinal Cord injury (WISCI II): scale revision. Spinal Cord. 2001 Dec;39(12):654–656. [PubMed]
10. Burns AS, Delparte JJ, Patrick M, Marino RJ, Ditunno JF. The reproducibility and convergent validity of the Walking Index for Spinal Cord Injury (WISCI) in chronic spinal cord injury. Neurorehabil. Neural Repair. 2011 Feb;25(2):149–157. [PubMed]
11. Mahoney F, Barthel D. Functional evaluation: The Barthel Index. Maryland Medical Journal. 1965;14:61–65. [PubMed]
12. Granger CV, Albrecht GL, Hamilton BB. Outcome of comprehensive medical rehabilitation: Measurement by PULSES profile and the Barthel Index. Archives of Physical Medicine and Rehabilitation. 1979;60(4):145–154. [PubMed]
13. Gresham GE, Labi MLC, Dittmar SS, Hicks JT, Joyce SZ, Stehlik MAP. The Quadriplegia Index of Function (QIF) - Sensivitiy and reliability demonstrated in a study of 30 quadriplegic patients. Paraplegia. 1986;24(1):38–44. [PubMed]
14. Itzkovich M, Gelernter I, Biering-Sorensen F, et al. The Spinal Cord Independence Measure (SCIM) version III: reliability and validity in a multi-center international study. Disabil Rehabil. 2007 Dec 30;29(24):1926–1933. [PubMed]
15. Slavin MD, Kisala PA, Jette AM, Tulsky DS. Developing a contemporary functional outcome measure for spinal cord injury research. Spinal Cord. 2010 Mar;48(3):262–267. [PubMed]
16. U.S. Department of Health and Human Services FDA. Guidance For Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Silver Spring, MD: U. S. Department of Health and Human Services, Food and Drug Administration; 2009. [PMC free article] [PubMed]
17. Amtmann D, Cook KF, Johnson KL, Cella D. The PROMIS Initiative: Involvement of Rehabilitation Stakeholders in Development and Examples of Applications in Rehabilitation Research. Archives of Physical Medicine and Rehabilitation. 2011;92(10):S12–S19. [PMC free article] [PubMed]
18. Chang C-H. Patient-Reported Outcomes Measurement and Management with Innovative Methodologies and Technologies. Quality of Life Research. 2007;16:157–166. [PubMed]
19. Acquadro C, Berzon R, Dubois D, et al. Incorporating the patient's perspective into drug development and communication: An ad hoc task force report of the patientreported outcomes (PRO) harmonization group meeting at the Food and Drug Administration, February 16, 2001. Value in Health. 2003;6(5):522–531. [PubMed]
20. Cella D, Riley W, Stone A, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult selfreported health outcome item banks: 2005–2008. J Clin Epidemiol. 2010 Nov;63(11):1179–1194. [PMC free article] [PubMed]
21. Cella D, Nowinski C, Peterman A, et al. The Neurology Quality-of-Life Measurement Initiative. Archives of Physical Medicine and Rehabilitation. 2011;92(10 Supplement):S28–S36. [PMC free article] [PubMed]
22. Rose M, Bjorner JB, Becker J, Fries JF, Ware JE. Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient- Reported Outcomes Measurement Information System (PROMIS) J Clin Epidemiol. 2008 Jan;61(1):17–33. [PubMed]
23. Gershon RC, Lai JS, Bode R, et al. Neuro-QOL: quality of life item banks for adults with neurological disorders: item development and calibrations based upon clinical and general population testing. Qual Life Res. 2011 Aug 27; [PMC free article] [PubMed]
24. Slavin MD, Kisala PA, Jette AM, Tulsky DS. Developing a contemporary functional outcome measure for spinal cord injury research. Spinal Cord. 2009;48(3):262–267. [PubMed]
25. Jette A, Tulsky D, Ni P, et al. (This issue).
26. Kisala PA, Tulsky DS. Opportunities for CAT applications in medical rehabilitation: development of targeted item banks. J Appl Meas. 2010;11(3):315–330. [PubMed]
27. Haley SM, Ni P, Lai JS, et al. Linking the activity measure for post acute care and the quality of life outcomes in neurological disorders. Arch Phys Med Rehabil. 2011 Oct;92(10 Suppl):S37–S43. [PMC free article] [PubMed]
28. Haley SM, Coster WJ, Andres PL, et al. Activity outcome measurement for postacute care. Med Care. 2004 Jan;42(1 Suppl):I49–I61. [PubMed]
29. Calhoun CL, Haley SM, Riley A, Vogel LC, McDonald CM, Mulcahey MJ. Development of items designed to evaluate activity performance and participation in children and adolescents with spinal cord injury. Int J Pediatr. 2009;2009 854904. [PMC free article] [PubMed]
30. Mulcahey MJ, DiGiovanni N, Calhoun C, Homko E, Riley A, Haley SM. Children's and parents' perspectives about activity performance and participation after spinal cord injury: initial development of a patient-reported outcome measure. Am J Occup Ther. 2010 Jul-Aug;64(4):605–613. [PubMed]
31. DeWalt DA, Rothrock N, Yount S, Stone AA. Evaluation of item candidates: the PROMIS qualitative item review. Med Care. 2007 May;45(5 Suppl 1):S12–S21. [PMC free article] [PubMed]
32. The LEXILE framework for reading [computer program] Durham, NC: MetaMetrics, Inc.; 1995.
33. National Spinal Cord Injury Statistical Center. Spinal Cord Injury: Facts and Figures at a Glance. Birmingham, AL: University of Alabama; 2011.
34. Organization WH. International Classification of Functioning, Disability, and Health. Geneva: WHO; 2008.
35. Bollen KA. Structural equations with latent variables. New York: John Wiley & Sons; 1989.
36. Bollen KA, Long JS. Introduction. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. pp. 1–9.
37. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equation Model. 1999;6(1):1–55. 1999/01/01.
38. Marsh HW, Balla JR, McDonald RP. Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychol. Bull. 1988 May;103(3):391–401.
39. Tucker L, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1–10.
40. Bentler PM. Comparative fit indexes in structural models. Psychol. Bull. 1990 Mar;107(2):238–246. [PubMed]
41. Browne MW, Cudeck R. Alternative ways of assessing model fit. In: Bollen KA, Long JS, editors. Testing structural equation models. Newbury Park, CA: Sage; 1993. pp. 136–162.
42. Steinberg L, Thissen D. Use of Item Response Theory and the Testlet Concept in the Measurement of Psychopathology. Psychological Methods. 1996;1(1):81–97.
43. Millis S, Malina AC, Bowers DA, Ricker J. Confirmatory Factor Analysis of the Wechsler Memory Scale-III. J. Clin. Exp. Neuropsychol. 1999;21(1):87–93. [PubMed]