One hospital returned data for only the physician sample and no other employees and was dropped from analyses. All results reported reflect data for 29 hospitals. A total of 4,547 surveys was returned, for an overall response rate of 49 percent. Job category was related to response rate, with senior managers having the highest (68 percent) and physicians the lowest (37 percent); excluding physicians, the response rate was 57 percent. The response rate also varied by hospital, ranging from 26 to 73 percent.
Psychometric Properties of PSCHO
Eighty-eight percent of the items had correlations of 0.40 or higher with their respective scale scores (adjusted for overlap) from the 11 scales resulting from the MTA, suggesting adequate item internal consistency (Kerlinger 1973
; Ware et al. 1997
). Further, in 329 out of 400 comparisons (82 percent), the correlations between items and their hypothesized scales were significantly higher than their correlations with any other scale; they were higher, though not significantly so, in an additional 37 comparisons. Altogether, appropriate discriminant validity was observed for 92 percent of item-to-scale correlations. Internal consistency reliabilities ranged from 0.61 to 0.89.
We did not find strong support for the hypothesized “fear of shame” scale, which consists of three items related to individual feelings of shame associated with mistakes and help seeking that could affect willingness to come forward with safety concerns in a timely manner. Internal consistency reliability for this scale was low (0.46), and none of the items achieved item-to-respective scale correlations above the 0.40 criterion. We did not treat these items as a scale in the analyses.
Results of the CFA in the validation sample supported the scale structure suggested by the MTA in the derivation sample. The root mean square error of approximation was 0.065, and Bentler's normed comparative fit index was 0.98. Both values were in the range indicative of a good fit of the model to the data (Bentler and Bonett 1980
; Hu and Bentler 1999
Respondents were evenly divided between males and females (). Most, 68 percent, were between the ages of 41 and 60. Thirteen percent of respondents were senior managers. Over one-third (36 percent) were physicians. Finally, 17 percent were HHU employees.
Demographic Information of Respondents*
Problematic Response by Item and Scale
Results for individual items grouped by scale are displayed in . The mean percent problematic response across all hospitals was 17.5, ranging from 12.0 to 23.7. Across all hospitals, individual item percent problematic response ranged from 4.4 to 49.6, with a mean of 18.1. The mean problematic response across all scales was 18.7 percent. Thirty-four items had a percent problematic response of 10 percent or greater, and 10 had a problematic response of 25 percent or greater. The scale with the highest α-coefficient was “senior leadership” (.89).
Overall, 50 percent of respondents indicated that their workgroups did not recognize individual safety achievement through rewards, nor were they rewarded for timely action to identify a serious mistake (41 percent). Forty-three percent reported never to have witnessed a coworker perform an unsafe act during patient care. Seven percent believed that asking for help was a sign of incompetence, and 4 percent indicated that they would not report a mistake with significant consequences that no one had noticed.
Across all items, an average of 19.9 percent of respondents selected the neutral mid-point of the scale. The percent of neutral responses ranged from 5.2 (“If I make a mistake that has significant consequences and nobody notices, I do not tell anyone about it”) to 39.3 percent (“Clinicians who make serious mistakes are usually punished”). This latter item is part of the “fear of blame” scale, designed to capture a punitive climate. The average percent problematic response plus neutral response over all items was 37.9 percent. Items rated by participants as “not applicable” were not included in our analyses.
Scale Problematic Response by Position and Job Type
We examined the mean percent problematic response by scale for senior managers compared with nonsenior managers (i.e., all others) as well as clinicians versus nonclinicians (). There was a statistically significant difference between the mean problematic response for senior managers (9.8 percent) and that for all others (18.3 percent). Senior managers had significantly lower overall means than nonsenior managers ( p<.05) on the “senior leadership,”“workgroup leadership,”“workgroup norms,”“workgroup recognition,” and “problem responsiveness” scales. In contrast, clinicians and nonclinicians had relatively similar responses, with no statistically significant differences between their overall means. When we looked at the data by workgroup, we found statistically significant differences among workgroups for the “learning” and “problem responsiveness” scales.
Scale Responses by Management Level, Clinician Status, and Workgroup
We found a significant correlation between senior managers and clinicians. We constructed a multiple regression model using percent problematic response as the dependent variable and management level as the independent variable, controlling for facility- and individual-level characteristics (geographic region, teaching status, metropolitan location, bed size, nurse-to-patient ratio, case mix, gender, age, HHU, and nurse). We found that senior managers have a significantly lower percent problematic response than other job types (data not shown).
To assess the relationship between percent problematic and percent neutral responses, we investigated the percent neutral response over all items by management level by creating a dummy variable for senior managers. For all but three items, senior managers had a lower percent neutral response than other staff (data not shown). The three items for which senior managers had higher frequencies of neutral response were “My unit recognizes individual safety achievement through rewards and incentives” (28.2 percent versus 20.6 percent), “Telling others about my mistakes is embarrassing” (21.5 percent versus 15.9 percent), and “I am rewarded for taking quick action to identify a serious mistake” (34.6 percent versus 28.3 percent).
Scale Problematic Response by Workgroup
To investigate the effect of hospital workgroup on problematic response, we created seven workgroup categories based on similarity of clinical functions: ambulatory (home care, off-campus ambulatory care, and main-campus ambulatory care); ED and urgent care; ICU; nonclinical; OR and PACU; pharmacy and lab; and ward (). The three workgroups with the highest overall mean percent problematic responses (ED/urgent, mean=23.4 percent; ICU, mean=22.7 percent; and OR/PACU, mean=20.7 percent) substantially overlapped with the areas categorized as HHUs in our sampling strategy. For seven of the 12 scales, the nonclinical workgroup had the lowest mean percent problematic response. Either ED/urgent or ICU had the highest percent problematic response for all scales, with the exception of the “fear of blame,”“psychological safety,” and “workgroup norms” scales.
Effects of Management Level within Job Type
We explored the variation in percent problematic response by management level and job type by creating four job categories: physician (including resident), nurse, other clinician, and nonclinician. Management level was broken down into three groups: senior manager, supervisor, and front-line worker. Nonclinicians had the largest percent of senior managers; 16.4 percent of responding nonclinicians were senior managers, compared with 14.7 percent of physicians, 8.9 percent of other clinicians, and 5.2 percent of nurses. Of the 240 nonclinician senior managers, 236 identified themselves as “other” (i.e., having administrative duties) on the demographic questions.
We calculated a mean percent problematic response for each group (). Across all job categories, front-line workers had the highest mean and senior managers the lowest. Nurses had the largest variation in mean percent problematic response by management level (senior manager nurses, 9.5 percent; front-line nurses, 21.8 percent; standard deviation, 12.2 percent), closely followed by nonclinicians (senior manager nonclinicians, 7.7 percent; front-line nonclinicians, 18.1 percent; standard deviation, 8.1 percent). Physicians as a group and front-line workers overall had almost identical variations.
Variation in Problematic Response by Management Level and Job Type.