PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Rehabil Res Dev. Author manuscript; available in PMC Dec 7, 2012.
Published in final edited form as:
PMCID: PMC3517188
NIHMSID: NIHMS413705
Development of CRIS: Measure of community reintegration of injured service members
Linda Resnik, PhD, PT, OCS,1,2* Matthew Plow, PhD,2 and Alan Jette, PhD, PT3
1Providence Department of Veterans Affairs Medical Center, Providence, RI
2Department of Community Health, Brown University, Providence, RI
3Department of Health Policy and Management, and Health and Disability Research Institute, Boston University School of Public Health, Boston, MA
*Address all correspondence to Linda Resnik, PhD, PT, OCS; Providence VA Medical Center, 830 Chalkstone Avenue, Providence, RI 02908; 401-273-7100, ext 2368; or Center for Gerontology and Healthcare Research, Brown University, 121 S. Main Street, Providence, RI 02912; 401-863-9214; fax: 401-863-3489. Linda_Resnik/at/brown.edu
Identification and prevention of community reintegration problems of veterans is an important public health mandate. However, no veteran-specific measure exists. Study purposes were to (1) develop the Community Reintegration for Service Members (CRIS) measure and (2) test the validity and reliability of the measure. Formative research identified challenges in community reintegration postdeployment. The World Health Organization’s International Classification of Functioning, Disability and Health participation domain guided item-bank development. Items were refined through cognitive interviews and clinician consultation. Pilot studies with 126 veterans examined unidimensionality, internal consistency, reliability, and construct validity. Three unidimensional CRIS scales were developed. Working subjects had better CRIS scores then unemployed subjects. Subjects with posttraumatic stress disorder, substance abuse, or mental health problems had worse scores than subjects without these conditions. The correlations between the CRIS and the 36-Item Short Form Health Survey scales of role physical, role emotional, and social functioning were 0.44–0.80. CRIS has strong reliability, conceptual integrity, and construct validity.
Keywords: community reintegration, disability measurement, International Classification of Functioning, Disability and Health, OIF/OEF, participation, Rasch analysis, rehabilitation, reliability, validity, veterans
More than 1.5 million U.S. soldiers, sailors, and marines have been deployed in Operation Iraqi Freedom and Operation Enduring Freedom (OIF/OEF). Demobilization and the return home can be challenging, especially for injured veterans [12]. The ultimate goal of rehabilitative efforts is to help those injured adjust to life at home and in the community [3], which is also called community reintegration. Community reintegration is especially challenging for injured veterans because it may be complicated by the co-occurrence of physical injuries with postwar adjustment difficulties, such as posttraumatic stress disorder (PTSD), depression, substance abuse, and severe mental illness [12]. Community reintegration may be particularly problematic for OIF/OEF veterans, who have an unusually high prevalence of traumatic brain injury (TBI) [4], and PTSD [56].
Thus, it is important to assess community reintegration and, whenever possible, to intervene early to prevent long-term consequences for returning service members’ families and society. However, no community reintegration measure exists specifically for the veteran population. In fact, no single generic measurement tool has been developed that can be used in all populations [7]. Some existing measures are population-specific and have been developed, for example, for those with strokes, spinal cord injuries, or head injuries [3,89]. Furthermore, existing measures cover vastly different integration domains.
Recent revision of the World Health Organization’s International Classification of Functioning, Disability and Health (ICF) [10] offers an alternative method of defining and measuring the domains of community integration. According to the ICF, the domain of participation focuses on the person’s involvement in society (i.e., community integration). People are considered to have healthy participation if they take part in all life areas or life situations in which they wish to participate, in a manner or to the extent that is expected of an individual without restrictions in that culture of society [11]. The ICF taxonomy includes nine overall domains: learning and applying knowledge; general tasks and demands; communication; mobility; self-care; domestic life; interpersonal relationships; major life areas; and community, social, and civic life.
Many questionnaires that are used to assess participation in life were developed using narrower models of disability [12] and thus fail to cover all nine domains of participation as identified by the ICF. Our review found that very few measures contained at least one item pertaining to each of the nine domains and in many instances, single questions addressed more than one domain [13]. To date, no community integration measure has been developed that incorporates issues specific to injured service members. Even the most comprehensive existing measures do not have adequate coverage of subcategories relevant to the OIF/OEF veteran population based on a recently completed study that identified numerous areas of restriction in participation in OIF/OEF veterans [14].
Thus, we aimed to develop a new measure of community reintegration for injured service members: the Community Reintegration for Service Members (CRIS). The purposes of this study were to (1) design and develop a measure of community reintegration of injured service members using the results of our qualitative research on injured service members in conjunction with the ICF framework and (2) conduct preliminary tests of the measure’s validity and reliability.
While the full description of the ICF taxonomy is beyond the scope of this article, a brief overview is provided here. The ICF model is divided into two components: the first covers functioning and disability, which includes four domains: (1) body function and (2) structure, (3) activities, and (4) participation. The second component of the model covers contextual factors including environment and personal factors. In the ICF graphic model, the body function and structure, activities, and participation domains are distinct from each other. However, in the ICF extensive system of taxonomy, only one coding structure exists for both activity and participation. Nevertheless, the two domains are conceptually distinct. According to the ICF, activities focus on the person’s individual functioning and are more likely to be performed alone [15]. In contrast, participation focuses on the person’s involvement in society (i.e., social functioning), and participation would more likely be performed with others.
Annex 3 of the ICF presents four options for differentiating between activities and participation. The first option is to exclusively designate some of the nine domains as activities and others as participation. The second option is to designate some domains as activities and others as participation—allowing partial overlap. The third option is to designate all broad categories as participation and all detailed categories as activities. In our approach to the measurement of community reintegration, we employed the fourth approach, which was to consider all codes as both activities and participation depending upon the content. Thus, as a general rule, we considered simple tasks and actions to be activities and complex functional tasks and actions to be participation (i.e., community reintegration). Questions pertaining to simple tasks were not included in the CRIS.
This approach is consistent with the finding of Jette et al., who demonstrated, using factor analysis, that activity and participation items from the Activity Measure for Post-Acute Care and the Participation Measure for Post-Acute Care (PM-PAC) loaded together within larger subdomains that roughly corresponded to ICF chapters [16].
Overview
The study was conducted in three stages outlined as follows. In stage 1, challenges in community reintegration were identified through formative research [14]. Items related to each identified challenge were adapted from existing measures or written specifically for the CRIS. In stage 2, the CRIS item pool was tested in two pilot studies to evaluate structural validity and test-retest reliability. Finally, in stage 3, a subset of items with strong reliability and good fit were selected for inclusion in the fixed-form CRIS and internal consistency, test-retest reliability, and construct validity were examined.
Stage 1: Formative Research and Development of CRIS
A qualitative study was conducted with 14 injured veterans, 12 caregivers, and 14 clinicians. The study design and findings are reported elsewhere [14]. In summary, veterans and caregivers discussed challenges in daily life and role function. Clinicians discussed common challenges they had observed in OIF/OEF patients. Two investigators categorized the data using the ICF domains of activities and participation and a directed approach to content analysis [17]. Numerous community reintegration problem areas were identified and coded using the ICF taxonomy. These included challenges in acquiring complex skills; focusing attention; solving problems; reading; undertaking multiple tasks; carrying out daily routine; handling stress; producing communication; sustaining a conversation; moving around; using transportation; driving; self-care; maintaining one’s health; doing housework; caring for household objects; perceiving social cues; physical contact in relationships; forming relationships; regulating behavior; family and intimate relationships; acquiring, keeping, and terminating jobs; complex economic transactions; economic self-sufficiency; community life; recreation and leisure; socializing; political life; and citizenship [14].
Development of CRIS Item Pool
Item development was an iterative process that included item writing followed by cognitive testing, refinement, conceptual and psychometric evaluation, and further refinement [18]. Items forming the initial item pool were generated by adapting existing items and constructing new items. We began by reviewing existing community integration measures and behavioral outcome measures for TBI. We performed a content analysis of these measures, classifying every item into one or more of the nine ICF participation domains [1920]. The content of each domain was further analyzed and coded into more precise subdomains by using the ICF taxonomy. Constructs and themes identified from the formative research phase were used to produce statements representing each of the dimensions and themes identified, which were then compared with the coded content of existing items. We developed new items when necessary.
In adapting existing items and writing new items, we created separate items for each of three dimensions of participation: frequency of participation, perceived limitation in participation, and satisfaction with participation [11]. Frequency of participation is considered a more objective measure [21] that does not consider individual perceptions, preferences, personal choices, and values. Because we believed that participation must also be understood in terms of consumer perceptions and experience, we developed separate scales to assess perceived limitation in participation and satisfaction with participation. Prior research shows that these dimensions (frequency, perceived limitations, and satisfaction) are only weakly and inconsistently correlated [22].
We phrased questions to facilitate comprehension, minimize recall bias, and enhance completion, which we felt was particularly important for service members who might have mild to moderate cognitive difficulties due to TBI. Thus, we worded questions to assess current life situation, with no comparison to life before injury or to other persons who were not injured or who had not been deployed, and framed questions to address participation within the last 2 weeks to minimize recall bias. Finally, we framed questions to minimize attribution to a specific health condition or event [23].
Content Validation and Cognitive Testing
Clinicians evaluated the content validity of proposed CRIS topic areas. Ten clinicians from Department of Veterans Affairs (VA) polytrauma and medical centers and the broader clinical community who had experience with injured service members were selected to participate in content validation. These clinicians were given descriptions of the ICF classification system along with written instructions to read over the list of proposed CRIS content areas and to gauge the importance of including each on a 7-point scale (1 = must include to 7 = should omit). We calculated average clinician ratings for each area and planned to omit content areas with average scores >4. No content areas were rated >4, and thus none was dropped.
Next, we asked seven additional clinicians to review the initial item bank and provide comments on language, content, and importance of each item on the same 7-point scale. Content raters included a polytrauma physician, two primary care physicians, a physical therapist, a recreational therapist, a psychologist, and an occupational therapist. Several questions with scores >4 were deleted and several questions were reworded or new items written based upon these clinicians’ suggestions. Next, we conducted cognitive-based interviews with seven OIF/OEF veterans to provide feedback about the questions and response formats and further refinements were made based upon cognitive testing [2425]. The initial CRIS item bank consisted of items divided into 30 scales: a 97-item Extent of Participation scale, a 116-item Perceived Limitation in Participation scale, and an 83-item Participation Satisfaction scale. Each item used 7-point Likert-type scales.
Stage 2: Pilot Studies
Two pilot studies of the CRIS item bank were conducted. Both studies examined structural validity of the CRIS scales to examine unidimensionality of the hypothesized domains, evaluate internal consistency of the scale items, and examine construct validity using known group analysis. The second pilot study examined test-retest reliability of the scales.
Pilot Study 1
Sample and Design
Pilot 1 employed a convenience sample of 50 veterans recruited from the primary care service of the Providence VA Medical Center (PVAMC) (Table 1), including both primary care homeless veteran clinics and usual primary care clinics.
Table 1
Table 1
Subject characteristics: Pilot studies.
Approach
We conducted preliminary analyses of unidimensionality of CRIS scales using WinSteps (Beaverton, Oregon) [26] and selecting items with the smallest percentage of missing data. Questions related to parenting and employment were omitted because of sample size limitations. We ran separate Rasch models within each hypothesized dimension (Extent of Participation, Perceived Limitation in Participation, Participation Satisfaction), examined the item-person map, and then checked the weighted fit statistics or mean square infit statistic and Rasch residual factor analysis to test the model fit and unidimensionality.
The Rasch model is a probability model that describes the person’s response as a function of person and item parameters. The person parameter is called the latent variable and represents a person’s underlying ability. In a Rasch model, the item parameter only has the difficulty parameters or threshold parameters and the discrimination parameter is set as 1 [27]. WinSteps is a statistical program that can be used to estimate the item and person parameters of a Rasch model. It is based on joint maximum likelihood estimation [26]. An item-person map provides a general picture of how good the match is between the scale and the sample. An infit statistic >1.4 represents noise in the data and indicates that the item does not belong to the unidimensional construct [26]. We examined the scale plot of the item measures and their corresponding factor loadings to determine whether a systemic pattern between positive and negative loading existed in terms of the item content. The Rasch residual factor analysis extracts the common factor underlying the items that explains the most residual variance. If this factor is only random noise, then the residuals have no meaningful structure [26]. Next, we examined the alphas for each of the CRIS dimensions identified in the analysis.
To assess CRIS construct validity, we examined differences in CRIS scores by known groups’ validation. “Known group validity” refers to the comparison of mean difference across known categories. The mean of different categories was calculated based on the person scores estimated from the Rasch model. The sample was classified into three known groups: (1) engagement in productive activity, (2) presence of PTSD, and (3) presence of depression. We hypothesized that veterans who were (1) working or volunteering and did not have (2) PTSD or (3) depression would be better integrated into the community and thus would have higher (i.e., better) CRIS scores.
Productive activity was a 3-level variable categorized as engaged in productive activity (which included those working or volunteering), retired, and not engaged in productive activity. PTSD and depression were dichotomous variables based on documented diagnoses in the medical record. Four separate analyses of variance (ANOVAs) were performed using the independent variable of productive activity and dependent variables of CRIS scales (description follows).
Results
Descriptive analyses of the preliminary Rasch modeling identified a distinct Extent of Participation dimension; the item-person map was acceptable, the scale covered the entire sample, and no gap in the scale was noted. Of the 60 items, 8 were misfit. A systemic pattern emerged in the residual matrix. Within this analysis, we identified two subscales in the Extent of Participation domain that we interpreted as an Emotional/Cognitive subdomain and a Social/Interpersonal subdomain. Rasch analyses were also used to examine the Perceived Limitation in Participation and the Participation Satisfaction domains; however, the underlying structure was less distinct and the scales needed additional items at the highest and lowest levels of participation within this sample. In the Participation Satisfaction domain, 0 of 71 items were misfit, while in the Perceived Limitation in Participation domain, 16 out of 103 items were misfit.
The percentage of variance in the data explained by each of the three Rasch models (i.e., Extent of Participation, Perceived Limitation in Participation, Participation Satisfaction domains) was 64.3, 59.7, and 71.3 percent, respectively. Alphas of scales identified in the item response theory (IRT) analysis ranged from 0.82 to 0.98. We calculated scores for each of the four scales identified in the preliminary IRT analysis as the mean score of all completed questions within the scale multiplied by the number 10. Thus the range of scores for each of the scales was 10 to 70, with higher scores indicating better integration.
As hypothesized, descriptive statistics of scale scores showed differences in CRIS scale scores between groups (results not shown). Subjects who were working had higher scores on all four scales than those who were retired or not working. Results of the ANOVA showed lower (i.e., worse) scores on three of the four scales for veterans with PTSD compared with veterans without PTSD. Results of the ANOVA on depression indicated comparable scores for all CRIS scales, except for Participation Satisfaction, which was significantly lower for veterans with depression (p = 0.03).
We used the results of pilot 1 to refine the CRIS measure, revising all misfit items and adding additional items to address the higher and lower ends of participation of the Perceived Limitation in Participation and the Participation Satisfaction domains. All new items were cognitively tested in a sample of six veterans recruited from PVAMC’s primary care clinics. Following refinement of the new items, we conducted a second pilot study of psychometrics of the CRIS.
Pilot Study 2
Sample and Design
A convenience sample of 76 subjects was recruited from the PVAMC’s primary care clinics. Efforts were made to recruit into the sample veterans who were parents of young children and veterans who were working. Subject characteristics are shown in Table 1. During the first visit, each subject was administered the CRIS items and demographic questions. During the second visit, CRIS items as well as the Craig Handicap Assessment and Reporting Technique (CHART) and the 36-Item Short Form Health Survey (SF-36) were readministered to the subjects. Sixty-six subjects returned for a follow-up visit within 1 week.
Approach
Structural validity analyses were conducted using data from the first visit. We performed preliminary Rasch analyses following the methods described in pilot 1. To assess the preliminary construct validity of the revised measure, we examined differences in scores by known groups’ validation following the methods described in pilot 1. Mean ± standard deviation scores for the overall sample were as follows: Extent of Participation 50.2 ± 8.7 (range 28.0–64.0), Perceived Limitation in Participation 50.1 ± 10.2 (range 28.1–69.4), and Participation Satisfaction 50.0 ± 10.3 (range 23.1–68.9). The sample was classified into four known groups: (1) employment status, (2) diagnosis of PTSD, (3) diagnosis of substance abuse, and (4) diagnosis of any mental health problem (inclusive of PTSD, depression, substance abuse, and other diagnoses). Because the sample for pilot 2 was younger with fewer participants who were retired than the sample in pilot 1, we classified employment status as either working (which included those working full-time or part-time or in work training) or not working (because of disability, unemployment, or retirement). We classified participants based upon the presence or absence of any mental health problem based on documented diagnoses in the medical record. We hypothesized that (1) veterans who were working and (2) those without PTSD, (3) substance abuse, or (4) any mental health problem would be better integrated into the community and thus would have higher (i.e., better) CRIS scores.
Three separate general linear models (GLMs), controlling for potential confounding by age, were performed using the independent variable of employment status and dependent variables of CRIS (1) Extent of Participation scale, (2) Perceived Limitation in Participation scale, and (3) Participation Satisfaction scale. Similar sets of GLMs were performed using the independent variables of PTSD, substance abuse, and any mental health problem.
Reliability was examined by calculating the intraclass correlation coefficient (ICC) of each scale. ICCs were calculated as the Shrout & Fleiss (type 2,1) ICC, which is generally denoted by ICC (2,1) [28]. ICC (2,1) is a two-way mixed-effects single-measure reliability measure in which the target is random effect, the number of measurements on each target is fixed effect, and the unit of analysis is the individual measurement instead of the mean of measurements.
Results
These structural analyses revealed three distinct, unidimensional subdomains of community participation. The results showed that unidimensionality of the Extent of Participation scale improved in the second version of the CRIS item bank, with only one dimension identified, in contrast to the findings from pilot 1, which identified two subdomains in the Extent of Participation scale. The percentage of variance in the data explained by each of the three Rasch models (i.e., Extent of Participation, Perceived Limitation in Participation, Participation Satisfaction subdomains) was 53.2, 85.2, and 73.3 percent, respectively. In the Extent of Participation scale, the distribution of items across the range of community participation (referred to as the item-person map) was acceptable, the scale covered the entire sample, and no gap in the scale was noted. In Rasch analyses of the Perceived Limitation in Participation scale, a gap at the highest levels of participation within this sample was revealed, indicating a need for additional items in the item bank. These results show that the gap at the lower level of Perceived Limitation in Participation (identified in the pilot 1 analysis) was closed in the second version of CRIS. In the Participation Satisfaction scale, gaps were identified at the highest and lowest levels of participation within this sample.
In the Extent of Participation scale, 3 items were misfit; in the Perceived Limitation in Participation scale, 20 items were misfit; and in the Participation Satisfaction scale, 14 items were misfit. Misfit items were removed and the alpha of each scale was calculated. The final scales included 95 items in the Extent of Participation scale, 107 items in the Perceived Limitation in Participation scale, and 85 items in the Participation Satisfaction scale. Scale alphas showed excellent internal consistency. Internal consistency reliability coefficients were 0.92 for Extent of Participation, 0.94 for Perceived Limitation in Participation, and 0.93 for Participation Satisfaction.
GLM results showed statistically significant differences in scores between groups in the hypothesized direction. Subjects who were working had higher scores on all scales than those who were not working. Results of the GLMs showed lower scores on all scales for veterans with PTSD, substance abuse history, or mental health problems than veterans without those diagnoses.
The ICC for the Extent of Participation reliability was 0.92 (95% confidence interval [CI] 0.87 to 0.95). The ICC for the Perceived Limitation in Participation scale was 0.94 (95% CI 0.91 to 0.96), while the ICC for the Participation Satisfaction scale was 0.93 (95% CI 0.89 to 0.96).
Stage 3: Fixed-Form CRIS Scales
Design
Our objective was to develop three separate comprehensive scales, one for frequency of participation, one for perceived limitation in participation, and one for satisfaction with participation, that could be administered in approximately 15 minutes each.
Approach
We chose items for inclusion based on comprehensiveness of content and item reliability, keeping in mind the preliminary assessment of item difficulty as calculated by the IRT analysis. Thus, we included one or more items that addressed each content area identified in our formative research and attempted to obtain items covering the entire range of item difficulty. We included only items with ICCs of 0.6 or higher. After making initial selections, we examined item-to-total correlations within each scale and replaced any items with correlations lower than 0.2 with alternative items that covered the same content area. Linear regression models (with fixed-form scores as independent variables) were used to estimate the proportion of variance (R2) explained in full scales. To examine the construct validity of the fixed-form scales, we repeated the GLMs performed in pilot 2, this time using scores of fixed-form scales as dependent variables. Thus, we performed separate GLMs, controlling for potential confounding by age, using the independent variables of employment status, PTSD, substance abuse, and any mental health problems for each of the CRIS scales.
We examined convergent and discriminant validity of the CRIS fixed-form scales by performing pairwise correlations of the CRIS scales/measures with other items, which in theory should be highly related, as well as with measures, which in theory should not be strongly related. We examined correlations between CRIS sub-scales and two subscales of the CHART [8]: the occupation scale and the social integration scale; and four scales from the SF-36: physical functioning, role physical, role emotional, and social functioning. The 7-item CHART occupation subscale assesses number of hours of participation in occupational activities. The 6-item CHART social integration subscale assesses type of living arrangements; presence or absence of romantic involvement; and amount of contact with friends, associates, and strangers. The physical function scale is a 10-item sub-scale of the SF-36 that measures difficulty with performance of physical activities [29]. The 3-item SF-36 role emotional subscale measures difficulty with role function in work or daily activities attributable to mental health problems. Finally, the 2-item SF-36 social functioning subscale measures interference with social activities related to physical and emotional problems [29].
Alpha coefficients of the three fixed-form scales were Extent of Participation 0.91, Perceived Limitation in Participation 0.93, and Participation Satisfaction 0.97. Fixed forms, which employed approximately half the number of items of the full scale, were able to predict 0.97–0.98 of the variance within the full scales. A summary of CRIS items classified by domain and subcategory of activities and participation taxonomy is shown in Table 2. The CRIS scales are included in the Appendix (available online only).
Table 2
Table 2
Summary of number of items in each content area by International Classification of Functioning, Disability and Health (ICF) coding classification.
As hypothesized, results of the GLMs show differences in CRIS scale scores between groups (Table 3). After controlling for age, subjects who were working had higher scores on all scales than those who were not working. Subjects with PTSD, substance abuse history, or mental health problems had lower scores than veterans without those diagnoses.
Table 3
Table 3
Descriptive statistics and differences by known groups of Community Reintegration for Service Members (CRIS) scale scores.*
Results of the pairwise correlation analysis (Table 4) show that CRIS scales were strongly correlated with the SF-36 scales of role physical, role emotional, and social functioning. In contrast, CRIS scales were weakly correlated with the CHART occupation scale and not significantly correlated with the CHART social integration scale. CRIS scales were weakly to moderately correlated with the SF-36 physical function scale.
Table 4
Table 4
Correlations between Community Reintegration for Service Members scales and other measures.
The results of our analyses demonstrate that the CRIS instrument is a comprehensive measure with conceptual integrity; excellent reliability; strong content; and construct, convergent, and discriminant validity. The validity of the dimensions is based on face validity, item fit, and the Rasch residual factor analysis. These results showed that most of the items fit the data across two pilot studies: more than 50 percent of the variance could be explained by the Rasch model in two pilot studies and the residual pattern generally supported the unidimensionality within each proposed domain.
The construct validity of the CRIS measure is supported by the analysis of known group differences and by the concurrent and discriminant validity analyses. As expected, we observed moderate to strong correlations between CRIS scales and three SF-36 scales. Each of these scales aims to measure the constructs of role and social functioning. We observed weaker to no correlations been CHART subscales and CRIS scores, because CHART scales measure only quantity of occupational engagement, i.e., hours of work or productive activity and number of friends or business associates. In contrast, CRIS and SF-36 scales ask about the frequency of restrictions in participation, perceived limitations in participation, and participation satisfaction. The stronger correlation between the CRIS Extent of Participation scale and the SF-36 versus the CHART suggests that this scale is a more subjective measure of frequency of participation and difficulties with participation (similar to SF-36 scales) than the CHART. The weak to moderate correlations between the SF-36 physical function scale and CRIS scales make sense given that a person’s participation in role function is influenced by activity limitations as well as the environment and other personal factors.
We believe that further development, validation, and use of the CRIS is important. Numerous reintegration problems in Gulf War and OIF/OEF veterans have been reported, including marital difficulties, financial difficulties, problems with alcohol or substance abuse, medical problems, behavioral problems such as depression or anxiety [2], homelessness [30], and motor vehicle accidents [31]. To date, no systematic efforts have estimated the scope of these problems. At present, neither the VA nor Department of Defense electronic medical records contain standardized data elements related to functional health and community reintegration. Use of the CRIS would provide a method for comprehensive, standardized assessment and monitoring of community reintegration outcomes of these vulnerable veterans. Unlike the PM-PAC or other generic measures of participation, the CRIS is specially designed to address community integration issues from a service person’s perspective. In future studies, researchers could compare the CRIS to generic measures of participation, such as the PM-PAC, to determine whether value is added with a service person-specific measure.
Limitations
One limitation of our study is that it was not possible to do exploratory and confirmatory factor analysis because of the small sample size. However, we believe that our relatively small sample size was adequate to yield stable parameter estimates within each major domain because of the few parameters in the Rasch model. Linacre has suggested that a sample of 50 well-targeted examinees is a conservative estimate of the sample size required to obtaining useful estimates; thus our pilot sizes, while not ideal, were adequate for generating estimates in a well-designed, focused pilot study [32]. From the item parameter recovery point of view, evidence suggests that increasing the number of items to be analyzed has little effect on the item parameter recovery but increasing the number of categories will increase the error variance of the parameter estimates [33]. The sample size, therefore, was reasonable for a Rasch model pilot study. We considered using a two-parameter logistic model to assess discrimination and fit of items: the use of this type of model would have required a larger sample size than was available to us in this pilot work.
Enhanced clinical information systems are a key component of improving care delivery for patients with chronic and complex conditions. Routine assessment of community reintegration would enhance patient assessment and targeting of referrals to services such as mental health, social services, and benefits as well as drive interventions that address underlying factors related to poor community reintegration. However, the use of fixed-form assessment methodology for a comprehensive reintegration instrument like the CRIS presents common problems that may limit its ultimate utility.
Future Research Directions
The fundamental problem encountered with short-form instruments is the occurrence of frequent floor and ceiling effects, in which large numbers of individuals score at either the top or the bottom of the range of possible scores. These effects severely reduce measurement precision [34]. The CRIS was developed as a more comprehensive and lengthy instrument (with 147 items across three basic subscales) to minimize concerns about inadequate measurement precision and inadequate coverage of important participation domains. While achieving excellent psychometric properties, as demonstrated in these analyses, the length of the CRIS may limit its utility in the clinical environment.
One promising solution to measurement problems of traditional fixed-form instruments is offered through the combined application of computer-adapted testing (CAT) [35] and IRT [3637]. CAT methodology uses a computer interface for the patient that is tailored to the unique ability level of the patient on a specific outcome dimension. Administering outcome items that are either too easy or too hard provides little information. An adaptive test first asks questions in the middle of the ability range and then directs questions to the level based on the patient’s responses, without asking unnecessary questions. This allows for administration of fewer items while gaining precise information regarding an individual’s placement along a continuum of community integration. The strategy of matching items to respondents has been used to achieve short and precise educational and psychological tests for decades. CAT results from the use of a simple form of artificial intelligence that selects questions tailored to the test-taker, shortens or lengthens the test to achieve the desired precision, scores everyone on the standard metric so that results can be compared, and displays results instantly. In practice, this approach minimizes the number of items that are administered to an individual and obtains an estimate of participation in any particular content area.
CAT applications require a large set of items in any one outcome domain (called item banks); items that consistently scale along a dimension of low to high proficiency; and rules guiding starting, stopping, and scoring procedures. The current fixed-form CRIS can provide the foundation for an item pool that could be used to develop a CAT version of this instrument. A significant challenge in developing CAT instruments is the need for large, representative data sets that are used to establish item and response characteristic curves in item pools that are needed for the complex modeling in the CAT programs [35].
The careful use of CAT-based outcome assessment applied to community reintegration holds considerable promise to make outcomes assessments briefer and less burdensome to patients and thus more acceptable for use in busy clinical and research settings. CAT methodology will provide data that is applicable across different clinical settings and applications, more efficient and less costly to administer, and where needed, more precise than conventional assessment approaches. Thus, they may have considerable promise as a vehicle for advancing community reintegration assessment while avoiding the pitfalls of traditional methodology. The CRIS is a comprehensive instrument with sound psychometric properties that can be used to measure community reintegration of veterans. To reduce respondent burden, we recommend further development of the CRIS through the combined application of CAT and IRT.
Acknowledgments
Funding/Support: This material is based on work supported by VA Health Services Research and Development Service grants SDR 07-327 and TRP-04-179, RI Foundation grant 20052665, and National Research Service award HS00011-22.
Abbreviations
ANOVAanalysis of variance
CATcomputer-adapted testing
CHARTCraig Handicap Assessment and Reporting Technique
CIconfidence interval
CRISCommunity Reintegration for Service Members
GLMgeneral linear model
ICCintraclass correlation coefficient
ICFInternational Classification of Functioning, Disability and Health
IRTitem response theory
OIF/OEFOperation Iraqi Freedom and Operation Enduring Freedom
PM-PACParticipation Measure for Post-Acute Care
PTSDposttraumatic stress disorder
PVAMCProvidence VA Medical Center
SF-3636-Item Short Form Health Survey
TBItraumatic brain injury
VADepartment of Veterans Affairs

Footnotes
Financial Disclosures: The authors have declared that no competing interests exist.
Author Contributions:
Study concept and design: L. Resnik, A. Jette.
Acquisition of data: L. Resnik, M. Plow.
Analysis and interpretation of data: L. Resnik, M. Plow, A. Jette.
Drafting of manuscript: L. Resnik, M. Plow, A. Jette.
Critical revision of manuscript for important intellectual content: L. Resnik, M. Plow, A. Jette.
Statistical analysis: L. Resnik, M. Plow, A. Jette.
Obtained funding: L. Resnik.
Study supervision: L. Resnik.
Additional Contributions: We wish to acknowledge the analytical assistance of Pengsheng Ni, PhD, MPH, and Lan Jiang, MS, and the work of research assistants Regina Lynch and Melanie Parent. Matthew Plow is currently a postdoctoral fellow at the University of Illinois at Chicago College of Allied Health Care, Chicago, Illinois.
1. Litz B, Orsillo SM. Iraq war clinician guide. 2. White River Junction (VT): U.S. National Center for Post-Traumatic Stress Disorder; 2004. The returning veteran of the Iraq war: Background issues and assessment guidelines; pp. 21–32.
2. Doyle ME, Peterson KA. Re-entry and reintegration: Returning home after combat. Psychiatr Q. 2005;76(4):361–70. doi: 10.1007/s11126-005-4972-z. [PubMed] [Cross Ref]
3. McColl MA, Davies D, Carlson P, Johnston J, Minnes P. The community integration measure: Development and preliminary validation. Arch Phys Med Rehabil. 2001;82(4):429–34. doi: 10.1053/apmr.2001.22195. [PubMed] [Cross Ref]
4. Lew HL, Poole JH, Alvarez S, Moore W. Soldiers with occult traumatic brain injury. Am J Phys Med Rehabil. 2005;84(6):393–98. doi: 10.1097/01.phm.0000163703.91647.a7. [PubMed] [Cross Ref]
5. Hoge CW, Castro CA, Messer SC, McGurk D, Cotting DI, Koffman RL. Combat duty in Iraq and Afghanistan, mental health problems, and barriers to care. N Engl J Med. 2004;351(1):13–22. doi: 10.1056/NEJMoa040603. [PubMed] [Cross Ref]
6. Hoge CW, Auchterlonie JL, Milliken CS. Mental health problems, use of mental health services, and attrition from military service after returning from deployment to Iraq or Afghanistan. JAMA. 2006;295(9):1023–32. doi: 10.1001/jama.295.9.1023. [PubMed] [Cross Ref]
7. Minnes P, Buell K, Nolte ML, McColl MA, Carlson P, Johnston J. Defining community integration of persons with brain injuries as acculturation: A Canadian perspective. NeuroRehabilitation. 2001;16(1):3–10. [PubMed]
8. Whiteneck GG, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: A new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil. 1992;73(6):519–26. [PubMed]
9. Trigg R, Wood VA. The Subjective Index of Physical and Social Outcome (SIPSO): A new measure for use with stroke patients. Clin Rehabil. 2000;14(3):288–99. doi: 10.1191/026921500678119607. [PubMed] [Cross Ref]
10. Ustün TB, Chatterji S, Bickenbach J, Kostanjsek N, Schneider M. The International Classification of Functioning, Disability and Health: A new tool for understanding disability and health. Disabil Rehabil. 2003;25(11–12):565–71. [PubMed]
11. Schuntermann MF. The implementation of the International Classification of Functioning, Disability and Health in Germany: Experiences and problems. Int J Rehabil Res. 2005;28(2):93–102. doi: 10.1097/00004356-200506000-00001. [PubMed] [Cross Ref]
12. Perenboom RJ, Chorus AM. Measuring participation according to the International Classification of Functioning, Disability and Health (ICF) Disabil Rehabil. 2003;25(11–12):577–87. doi: 10.1080/0963828031000137081. [PubMed] [Cross Ref]
13. Resnik L, Plow MA. Measuring participation as defined by the International Classification of Functioning, Disability and Health: An evaluation of existing measures. Arch Phys Med Rehabil. 2009;90(5):856–66. doi: 10.1016/j.apmr.2008.11.010. [PubMed] [Cross Ref]
14. Resnik LJ, Allen SM. Using International Classification of Functioning, Disability and Health to understand challenges in community reintegration of injured veterans. J Rehabil Res Dev. 2007;44(7):991–1006. doi: 10.1682/JRRD.2007.05.0071. [PubMed] [Cross Ref]
15. Whiteneck G. Conceptual models of disability: Past, present, and future. In: Field MJ, Jette AM, Martin LG, editors. Workshop on disability in America: A new look. Summary and background papers. Washington (DC): National Academies Press; 2006. pp. 50–66.
16. Jette AM, Tao W, Haley SM. Blending activity and participation sub-domains of the ICF. Disabil Rehabil. 2007;29(22):1742–50. [PubMed]
17. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88. doi: 10.1177/1049732305276687. [PubMed] [Cross Ref]
18. Clark LA, Watson D. Constructing validity: Basic issues in objective scale development. Psychol Assess. 1995;7(3):309–19. doi: 10.1037/1040-3590.7.3.309. [Cross Ref]
19. Cieza A, Geyh S, Chatterji S, Kostanjsek N, Ustün B, Stucki G. ICF linking rules: An update based on lessons learned. J Rehabil Med. 2005;37(4):212–18. doi: 10.1080/16501970510040263. [PubMed] [Cross Ref]
20. Cieza A, Brockow T, Ewert T, Amman E, Kollerits B, Chatterji S, Ustün TB, Stucki G. Linking health-status measurements to the International Classification of Functioning, Disability and Health. J Rehabil Med. 2002;34(5):205–10. doi: 10.1080/165019702760279189. [PubMed] [Cross Ref]
21. Brown M, Dijkers MP, Gordon WA, Ashman T, Charatz H, Cheng Z. Participation objective, participation subjective: A measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil. 2004;19(6):459–81. doi: 10.1097/00001199-200411000-00004. [PubMed] [Cross Ref]
22. Johnston M, Nissim EN, Wood K, Hwang K, Tulsky D. Objective and subjective handicap following spinal cord injury: Interrelationships and predictors. J Spinal Cord Med. 2002;25(1):11–22. [PubMed]
23. Dubuc N, Haley SM, Kooyoomjian JT, Jette AM. Assessing disability in older adults: The effects of asking questions with and without health attribution. J Rehabil Med. 2004;36(5):226–31. doi: 10.1080/16501970410029780. [PubMed] [Cross Ref]
24. Jobe JB, Mingay DJ. Cognitive laboratory approach to designing questionnaires for surveys of the elderly. Public Health Rep. 1990;105(5):518–24. [PMC free article] [PubMed]
25. Jobe JB, Mingay DJ. Cognitive research improves questionnaires. Am J Public Health. 1989;79(8):1053–55. doi: 10.2105/AJPH.79.8.1053. [PubMed] [Cross Ref]
26. Linacre JM, Wright BD. A user’s guide to WinSteps. Chicago (IL): MESA Press; 1999.
27. Wright BD, Stone MH. Best test design. Chicago (IL): MESA Press; 1979.
28. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979;86(2):420–28. doi: 10.1037/0033-2909.86.2.420. [PubMed] [Cross Ref]
29. Ware JE, Snow KK, Kozinski M, Gandek B. SF-36 Health Survey: Manual and interpretation guide. Boston (MA): The Health Institute, New England Medical Center; 1993.
30. Brustein J. Back from Iraq. Gotham Gazette. 2006 Mar 19;Sect 4
31. Hooper TI, Debakey SF, Bellis KS, Kang HK, Cowan DN, Lincoln AE, Gackstetter GD. Understanding the effect of deployment on the risk of fatal motor vehicle crashes: A nested case-control study of fatalities in Gulf War era veterans, 1991–1995. ccid Anal Prev. 2006;38(3):518–25. doi: 10.1016/j.aap.2005.11.009. [PubMed] [Cross Ref]
32. Linacre JM. Sample size and item calibration stability. Rasch Meas Trans. 1994;7:28.
33. DeMars CE. Sample size and the recovery of nominal response model item parameters. Appl Psychol Meas. 2003;27(4):275–88. doi: 10.1177/0146621603027004003. [Cross Ref]
34. Rubenstein LM, Voelker MD, Chrischilles EA, Glenn DC, Wallace RB, Rodnitzky RL. The usefulness of the Functional Status Questionnaire and Medical Outcomes Study Short Form in Parkinson’s disease research. Qual Life Res. 1998;7(4):279–90. doi: 10.1023/A:1008838317063. [PubMed] [Cross Ref]
35. Ware JE, Jr, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: A brief summary of ongoing studies of widely used headache impact scales. Med Care. 2000;38(9 Suppl):II73–82. [PubMed]
36. McHorney CA, Cohen AS. Equating health status measures with item response theory: Illustrations with functional status items. Med Care. 2000;38(9 Suppl):II43–59. [PubMed]
37. Hambleton RK. Emergence of item response modeling in instrument development and data analysis. Med Care. 2000;38(9 Suppl):II60–65. Submitted for publication July 3, 2008. Accepted in revised form December 2, 2008. [PubMed]