|Home | About | Journals | Submit | Contact Us | Français|
The examination to determine if a veteran has service-connected posttraumatic stress disorder (PTSD) affects veterans’ lives for years afterwards. This study examined factors potentially associated with veterans’ perception of their examination’s quality.
Three hundred eighty-four veterans participated in a clinical trial in which they received either a semi-structured interview or the examiner’s usual interview. Immediately after the interview, veterans completed confidential ratings of the examination’s quality and of their examiners’ interpersonal qualities and competence. Extensive data characterizing the veterans, the 33 participating examiners, and the examinations themselves were collected.
Forty-seven percent of Caucasian veterans versus 34% of African American veterans rated their examination quality as “excellent.” In multivariate analysis, African Americans were less likely than Caucasians to assign a higher quality rating (odds ratio .61, 95% confidence interval .38 – .99). African Americans also rated their examiners as having significantly worse interpersonal qualities but not lower competence. Ratings were not significantly related to the veterans' age, gender, marital status, eventual diagnosis with PTSD, Global Assessment of Functioning score, the examiners’ perception of the prevalence of malingering, or the presence of a third party in the examination.
Ratings of disability examinations were generally high, although African American veterans' ratings were less favorable than Caucasian veterans' ratings.
Veterans who claim a mental health condition was caused or exacerbated by their military service can apply for disability payments from the Department of Veterans Affairs. Once an application is filed, the veteran has an examination with a mental health professional that is central to assessing the claim. The approval or denial of a PTSD claim has far-reaching implications (1) as it can result in lifelong priority access to VA care, financial remuneration, and an official acknowledgement that the veteran was harmed by military service. As of 2009, there were 345,520 veterans receiving service-connected payments for PTSD (2), a number reflecting that disability awards are often continued for decades after the initial award (3).
Many veterans find the PTSD Compensation interview to be stressful and indicate that the Compensation examinations are conducted by examiners who do not understand them, question them skeptically, and display unfamiliarity with the military (4). These views are shared by many representatives of organizations that support veterans with their applications (5).
Veterans’ perceptions that their examinations were of lower quality are damaging. Compensation examinations are a potential portal of entry to engagement in VA treatment, and an off-putting interview may make veterans less pre-disposed to engage in VA treatment (6). Veterans’ perceptions that examinations are unfair can also become self-fulfilling prophecies, in that distrustful veterans may be more difficult to interview. Perceptions that disability determinations are capricious might also undermine public support for this indemnification program (7). For all these reasons, it is important to understand veteran satisfaction with the Compensation examination and whether there are factors associated with veterans’ perceptions that their Compensation examination was of lower quality.
In this study, we examined veteran, examiner, and examination characteristics potentially associated with satisfaction among veterans evaluated for service connection for PTSD. A wide range of potential predictors were considered because there are few data concerning claimants’ satisfaction with evaluative, forensic examinations of any type (8). Of particular interest was veterans’ race. Abundant literature has shown that African American patients, compared with Caucasians, have less trust in medical professionals than do Caucasians (9). Prior analyses that accounted for potentially confounding differences in PTSD disability awards, such as PTSD symptom severity and degree of disability, showed that African-American veterans were 13% less likely to receive such awards than Caucasians (10). Furthermore, this racial difference in PTSD disability award was found to directly mediate a higher subsequent burden of poverty among African American veterans relative to other veterans (32).
This study was embedded within a multi-site cluster randomized clinical trial of veterans being evaluated for an initial PTSD service-connection claim. Veterans were randomly assigned either to examiners who conducted their usual examination or to examiners who incorporated semi-structured assessments of PTSD and associated functional impairment into the interview. The semi-structured interviews incorporated the Clinician Administered PTSD Scale (CAPS) (11) to assess PTSD and the World Health Organization Disability Assessment Scale (WHODAS-II) (12) to assess functional impairment. The study design was hierarchical with veterans clustered within clinicians who were nested within medical centers. Examiners remained in their study arm throughout the study and did not cross over. Both veterans and examiners provided written, voluntary informed consent for participation and the study was approved by all participating study sites’ institutional review boards. As part of study participation, veterans and examiners agreed to audio recording of the Compensation examination.
Veterans’ subjective experiences of their Compensation examinations were assessed by research staff immediately after each veteran had undergone the PTSD examination using a brief paper-and-pencil questionnaire. The questionnaire items were adapted from measures used for similar purposes by the Veterans Benefits Administration to assess satisfaction with Compensation examinations and from other consumer satisfaction surveys (13, 14).
On the questionnaire, veterans were asked the summary question, “Overall, how would you rate the quality of today's Compensation & Pension examination?” with response options of excellent, very good, good, fair, and poor. Given zero “poor” ratings and very few ratings in the “fair” category (n=17, 5%), combining the “fair” and “good” response categories yielded a three-level ordinal scale for analysis (fair/good, very good, excellent), and ratings were coded so that higher scores corresponded to higher quality. This summary measure was the pre-defined primary outcome because it allowed veterans to consider the quality of all facets of the examination (13).
Veterans also rated their agreement with statements about the examiner’s interpersonal qualities and competence. The interpersonal qualities statements veterans rated began with “My examiner was…” and continued with “courteous,” “paid attention to what I had to say,” “took a personal interest in me,” and “was reassuring.” Agreement was rated on a Likert scale anchored by 1 = strongly disagree, 2 = somewhat disagree, 3 = neither agree nor disagree, 4 = somewhat agree, and 5 = strongly agree. These items rating interpersonal qualities had acceptable internal consistency (Cronbach’s alpha=.71)
Veterans rated the examiner's competence by rating separate items on the extent the examiner “was very thorough,” “seemed to know what s/he was doing,” “seemed very experienced,” “had a lot of skill when working with me” and was “fair.” These items rating competence had high internal consistency (Cronbach’s alpha =.85). The distinction between professional competence and personal qualities has been a key feature of surveys of satisfaction with healthcare providers (15).
We hypothesized that veterans’ satisfaction with their examinations would be impacted by characteristics of the veteran, examiner, and examination. Demographic data collected on veterans and included as predictors included age, gender, marital status, race, and education. Data extracted from the disability examination report included PTSD diagnosis (present, absent), substance use disorder, and the Global Assessment of Functioning (GAF) score. The GAF is a global rating scale that rates combined psychiatric and social functioning on a 0–100 scale (16). Demographics were also collected on examiners.
In addition, examiners completed a paper questionnaire (17), consisting of questions about examiners' training to conduct PTSD Compensation examinations, years conducting PTSD Compensation examinations, and attitudes towards claimants. Examiners were asked separate questions about whether they had received formal training in each of seven PTSD examination-related topics.
To elucidate examiners’ general attitudes concerning whether veterans are prone to either exaggerate or avoid discussing and thus minimize symptoms (18), each examiner was asked: "What percentage of the veterans you interview exaggerate symptoms?" The same question was asked regarding the percentage of examined veterans who minimize symptoms.
After each examination, the examiner recorded how much time had been spent conducting the interview. Examiners also indicated whether someone other than the veteran--such as a spouse or a Veterans’ Service Organization representative --- had been present during the examination.
The veteran rating of the overall quality of the PTSD examination was the primary dependent variable. A proportional odds logistic regression was used to assess the relationship between rating of overall quality and the covariates listed above. To account for the data structure of veteran ratings clustered within PTSD examiner, standard errors were obtained using bootstrap covariance matrix estimates. For continuous covariates, odds ratios are presented comparing the 75th percentile to the 25th percentile of the respective distribution. An alternative analysis was conducted treating quality as a dichotomous (excellent vs. not excellent) variable. The Interpersonal Quality and Competence scales were analyzed using linear regression on the same set of pre-specified covariates as above. A variance stabilizing log transformation was also employed to assess robustness to non-normality.
All statistical analysis was performed using R version 2.13.1 and R packages rms and Hmisc (19). Statistical significance was assessed at the alpha=.05 level
As described in detail elsewhere (20), 999 eligible veterans were assigned to clinicians who were participating in the study between March 17, 2009 and September 29, 2010. Altogether, 406 of the 999 potentially eligible veterans consented (41%). The retention rate for data collection was 95%, yielding 384 veterans, of whom 384 completed the evaluation of the examiners. The 384 veterans were examined by 33 examiners at six geographically scattered sites. The average number of veterans examined per examiner was 12 (range 1–42).
Most veterans in the study were male (n=366, 95%) and were married (n=239, 62%). Forty percent (n=154) had had some education after high school. With regard to race, 60% (n=228) were Caucasian, 26% percent (n=100) were African-American, and 14% (n=54) indicated "other" race. Of those indicating "other" race, 60% (31/52) were Hispanic. Study veterans had served mainly in the Army (67%), in combat (91%), and in Vietnam (56%) or in the Iraq and/or Afghanistan (36%) conflicts. Veteran age reflected the war era of the veterans, with twelve percent (n= 45) aged 27 or younger and sixty-four percent (n = 246) aged 51 or older.
In the Compensation examination reports, 65% (250/384) were diagnosed with PTSD, and 49% (188/382) were found to have a substance use disorder. Average GAF score was 55.0 +/− 10.5, reflecting moderate disability.
One examiner was Hispanic and the rest were non-Hispanic Caucasians. Clinical examiners were 61% female, 97% psychologists, with an average of 5.5 years of PTSD diagnostic experience. They had received an average of 5.2 of the seven PTSD/Compensation-related trainings inquired about. On average, these examiners estimated that about 10.6% (+/− 9.3% of veterans exaggerate symptoms and 13.6% +/− 15.9% of veterans minimize them. Examiners reported having spent an average of 184.6 minutes +/− 81.7 on the examination itself.
Forty-one percent (156/377) of the veterans rated their overall examination quality as “excellent,” 38% (n=142) rated it as “very good” and 21% (n=77) as “fair” or “good.” On a five-point scale, the mean rating of examiner competence was 4.72 +/−= .46 and that of examiner interpersonal qualities was 4.71 +/− .43. Ratings of examiners’ competence and interpersonal qualities were correlated with Pearson’s r = .72 (p = <.001). The correlations between the rating of overall quality and the competence and interpersonal qualities scales were .62 (p < .001) and .54 (p < .001), respectively.
Overall examination quality ratings were high with 79% (294/372) of veterans rating their examinations as "excellent" or "very good". As indicated in Table 1, while many predictor variables yielded relatively large coefficient estimates, the only factor significantly associated with overall rating of quality was race. Specifically, the estimated odds ratio for African American veterans versus Caucasian veterans for higher quality rating was .61 (p = .047, 95% CI = .40 – .99). Thus, the odds for a rating in a higher quality category among African American veterans was estimated to be 39% less than that for Caucasian veterans. Although the confidence interval is relatively wide, this result is consistent with the unadjusted raw data shown in Table 2: a lower proportion of African Americans (34%) indicated that their examination had been excellent than Caucasians did (47%).
Because this difference appeared concentrated at the highest end of the rating scale, an alternative analysis was conducted in which the overall rating of quality was treated as dichotomous (excellent vs. not excellent). In this multivariable logistic regression, the effect size for African-American veterans’ versus Caucasian veterans’ ratings were heightened even more (odds ratio= .54 (p = .016, 95% CI =.33 – .89) . Thus, as indicated by both the unadjusted and adjusted analysis, African-Americans were, compared with Caucasians, less satisfied with the quality of their exam.
Table 3 presents the results of the linear regression model on examiner competence on the pre-specified list of covariates. None of the coefficient estimates approached statistical significance, nor were the effects very large in magnitude.
Table 4 presents the results of the linear regression model on interpersonal quality. Compared with Caucasian veterans, African American veterans rated their examiners’ interpersonal qualities lower (p= .01, 95% CI = −.18 – −.02). The coefficient for the race category “Other” was similar to that of African-American (indicating lower interpersonal quality ratings compared with Caucasians), albeit not estimated with the same amount of certainty and not reaching statistical significance.
Several follow-up analyses were conducted to further elucidate the relationship between race and Compensation examinations. Because a structured examination might be more consistently delivered and less prone to differentially upset African American veterans, we reviewed the rates of low examination quality ratings within the structured and unstructured examination groups. African Americans rates of excellent quality ratings were 32% in the structured examination group and 36% in the unstructured group (p = .91), and the interaction of group (semi-structured exam or not) and race was not significant when added to the model. Thus, there was no evidence that the race effect was ameliorated by use of the semi-structured examination.
Overall ratings of C&P examination quality were predominantly "excellent" or "very good" by both African American and Caucasian veterans. However, African American veterans rated their examinations as having been of lower quality and rated their examiners lower on interpersonal qualities than Caucasian veterans did. This finding persisted even after controlling for other potential predictors of dissatisfaction. It is remarkable that despite the limited range of satisfaction in this study, only veterans’ race and race alone predicted lower ratings by veterans. There was no support in the data that other measured covariates accounted for veteran satisfaction.
African American veterans’ lower quality ratings may relate to characteristics of the veterans or of their examiners. When compared with patients of other races, African Americans have been shown to provide less information and to be less assertive with medical providers (21, 22); lack of assertiveness may be related to less trust in medical providers (21–24). It is also possible that African American veterans are treated differently than other veterans in the disability examination, because they tend to have received less previous PTSD treatment (25). Consequently, they may be less experienced and less comfortable discussing their PTSD symptoms with a medical professional than Caucasian veterans. Alternatively, it is possible that examiners are less empathic with African American veterans than with Caucasian veterans, a phenomenon described in medical settings (26). Reliance on pre-existing stereotypes by examiners may also be exacerbated in stressful situations (27, 28), such as a Compensation examination. Only Caucasian examiners evaluated African American veterans in this study, and such racial discordance has been associated with worse outcomes in clinical settings (30, 31).
A better qualitative understanding of the mechanisms behind the disproportionately lower ratings by African American veterans would have clinical implications. If the differences are attributable to African American veterans’ reticence, for example, teaching veterans the advantages of volunteering information before their examination might reduce future racial differences (29). On the other hand, differences arising from examiner-based characteristics would suggest the need for examiner-based solutions such as further examiner training and monitoring. While standardizing the Compensation examination with use of semi-structured interviews did not reduce the racial difference in satisfaction in the present study, perhaps more targeted examiner training (e.g., on cultural awareness) would.
It is important to note that the veteran’s perspective is only one component of high quality examinations. Other important stakeholders whose judgment of quality may differ from veterans’ are the Veterans Health Administration that conducts the examinations, the Veterans Benefits Administration which must decide service-connection based on the examination reports, and government and taxpayers who fund the awarded benefits (14).
The strength and generalizability of the study findings are open to more than one interpretation. The findings of racial differences did not reflect substantial dissatisfaction with the examinations---they reflected African Americans’ assigning relatively more ratings that were not excellent but were still mostly quite satisfactory (i.e. very good ratings) compared with Caucasian veterans. Two main interpretations of this are possible: one is that the range and extent of racial differences would be even greater in a non-research setting, but it is also possible that the racial difference in Compensation ratings is in fact a modest effect. A concern for generalizability is that the study data do not include descriptions of the standard examinations conducted. The range of examiner styles and methods employed do not allow for conclusions about what components of examinations might account for racial differences.
Overall, the findings suggest veterans’ race is important to how they perceive their examinations’ quality. In future studies of digitally-recorded examinations, we hope to elucidate the provider-veteran interactions that might be changed to improve veterans' satisfaction with their examinations.
This project was supported by the VA Health Service Research and Development QUERI Program SDR 06-331 and the VISN 1 MIRECC.
We would like to thank the E3-PTSD collaborative investigators for their work in this study: Heather Davis-Underwood MPH; Elliot Fielstein PhD; Elise Ratchford; Katherine Strickland, Dorothy Scanlan; Karin E. Thompson PhD; Sheila Corrigan PhD; Janet C’deBaca PhD; Mitzi Dearborn PhD; Michelle Sharp PhD; Christina M. Klauber, Elizabeth Jones, Erin Olgren, Eli Reich, Rachel Coleman, Yordanka Koleva, Lisa Fenton, the Veterans Benefits Administration Systematic Technical Accuracy Review (STAR) program, and the VA medical centers that participated in this project.