Variation in Emergency Medical Services Workplace Safety Culture

doi:10.3109/10903127.2010.497900

Prehosp Emerg Care. Author manuscript; available in PMC 2011 October 1.

Published in final edited form as:

Prehosp Emerg Care. 2010 Oct–Dec; 14(4): 448–460.

doi: 10.3109/10903127.2010.497900

PMCID: PMC2935310

NIHMSID: NIHMS215386

Variation in Emergency Medical Services Workplace Safety Culture

P. Daniel Patterson, PhD, MPH, EMT-B, David T. Huang, MD, MPH, Rollin J. Fairbanks, MD, MS, EMT-P, Scott Simeone, BS, MS, NREMT-P, Matthew Weaver, BS, NREMT-P, and Henry E. Wang, MD, MS

P. Daniel Patterson, University of Pittsburgh School of Medicine, Department of Emergency Medicine and Center for Emergency Medicine of Western Pennsylvania, Inc., Iroquois Building, 3600 Forbes Avenue, Suite 400A, Pittsburgh, PA 15261, Email: pattersond/at/upmc.edu;

Contributor Information.

Address correspondence and requests for reprints to: P. Daniel Patterson, PhD, Department of Emergency Medicine, University of Pittsburgh School of Medicine, 3600 Forbes Avenue, Iroquois Bldg., Suite 400A, Pittsburgh, PA 15261, (T) 412-647-3183, (F) 412-647-6999, Email: pattersond/at/upmc.edu.

The publisher's final edited version of this article is available at Prehosp Emerg Care

See other articles in PMC that cite the published article.

Abstract

Introduction

Workplace attitude, beliefs and culture may impact the safety of patient care. This study characterized perceptions of safety culture in a nationwide sample of Emergency Medical Services (EMS) agencies.

Methods

We conducted a cross-sectional survey involving 61 Advanced Life Support EMS agencies in North America. We administered a modified version of the Safety Attitudes Questionnaire (SAQ), a survey instrument measuring dimensions of workplace safety culture (Safety Climate, Teamwork Climate, Perceptions of Management, Job Satisfaction, Working Conditions, and Stress Recognition). We included full-time and part-time paramedics and Emergency Medical Technicians. We determined the variation in safety culture scores across EMS agencies. Using Hierarchical Linear Models (HLM), we determined associations between safety culture scores and individual and EMS agency characteristics.

Results

We received 1,715 completed surveys from 61 EMS agencies (mean agency response rate 47%; 95% CI 10%, 83%). There was wide variation in safety culture scores across EMS agencies [mean (min, max)]: Safety Climate 74.5 (Min 49.9, Max 89.7), Teamwork Climate 71.2 (Min 45.1, Max 90.1), Perceptions of Management 67.2 (Min 31.1, Max 92.2), Job Satisfaction 75.4 (Min 47.5, Max 93.8), Working Conditions 66.9 (Min 36.6, Max 91.4), Stress Recognition 55.1 (Min 31.3, Max 70.6). Air medical EMS agencies tended to score higher across all safety culture domains. Lower safety culture scores were associated with increased annual patient contacts. Safety climate domain scores were not associated with other individual or EMS agency characteristics.

Conclusion

In this sample, workplace safety culture varies between EMS agencies.

Keywords: Safety culture, teamwork, EMS, prehospital

INTRODUCTION

Organizational safety culture refers to the collective beliefs and perceptions of workers regarding the organization and safety of their workplace operations.¹ The Joint Commission, Agency for Healthcare Research and Quality, and Institute for Healthcare Improvement have all recommend frequent evaluations of organizational safety culture.²^-⁴ The National EMS Advisory Council called upon the EMS industry to develop a “culture of safety.”⁵ Research in high reliability industries such as nuclear power and aviation have linked organizational safety culture to accidents, safety audit scores, and safety behavior.¹^,⁶^,⁷ Previous research of in-hospital environments link safety culture scores to patient outcomes.⁸

Preventable adverse events occur in 1 of every 1,000 air medical EMS transports.⁹ A recent study has identified adverse events in ground EMS as well.¹⁰ Prehospital airway management errors are common and potentially harmful.¹¹^-¹³ Other studies and reviews highlight ambulance crashes, patient mishandling, malfunctioning equipment, medical mismanagement and protocol deviations.¹⁴^-¹⁷ EMS personnel often report feeling stress and burnout.¹⁸^-²¹ Other studies suggest that many Emergency Medical Technicians (EMTs) are concerned about the accuracy of care decisions, they suffer from poor sleep quality, high fatigue, and have a questionable commitment to their job.²²^-²⁴ Both sleep and fatigue have been linked to medical error and performance in physicians and nurses.²⁵^-²⁹ These observations suggest that EMS workplace culture may influence patient safety.

Prior studies have characterized organizational safety culture in the hospital inpatient setting, Intensive Care Units (ICUs), nursing wards, ambulatory care, and in skilled nursing facilities.³⁰^-³³ However, there have been no descriptions of the safety culture in EMS. In this study we sought to characterize variation in workplace safety culture in EMS and test the psychometric properties of our survey tool.

METHODS

Study Design

This study was approved by the University of Pittsburgh Institutional Review Board. We performed a cross-sectional survey of North American EMS agencies.

Study Setting

We enrolled a convenience sample of 62 EMS agencies from the United States and Canada. We selected only agencies that provided Advanced Life Support Care. While we did not utilize a formal sampling scheme, we tried to include agencies from a range of practice and geographic settings. Currently, there are no reliable and valid lists of all EMS agencies in the U.S. Our strategy for recruitment included advertising on a single web site and circulating a study flyer on popular EMS leadership email Listservs.

Methods of Measurement

We measured organizational safety culture using a structured 60-question survey instrument, the EMS Safety Attitudes Questionnaire (EMS-SAQ), which has been previously described.³⁴ We developed the EMS-SAQ by modifying the Intensive Care Unit Safety Attitudes Questionnaire (ICU-SAQ); a widely used and validated survey instrument characterizing workplace safety culture in hospital critical care units.³¹^,³⁵ Historically, the ICU-SAQ and related versions of the SAQ were based upon the Flight Management Attitudes Questionnaire, which assessed airline cockpit safety culture.³⁰ The SAQ has been adapted for and validated in a range of medical settings such as ambulatory care, the operating room, ICU, and skilled nursing facilities.³¹^-³³^,³⁶ We chose to modify the ICU-SAQ over other safety culture tools because it is widely used across different health care settings.³⁷

The ICU-SAQ used 30 core questions to characterize 6 safety domains; 1) safety climate [7 items], 2) job satisfaction [5 items], 3) perceptions of management [4 items], 4) teamwork climate [6 items], 5) working conditions [4 items], and 6) stress recognition [4 items]. We retained the same domains in the EMS-SAQ. We modified the wording to ensure consistency with EMS practice and convention. For example, we changed “In the ICU, it is difficult to discuss mistakes.” to “At this EMS agency, it is difficult to discuss mistakes.” Respondents provided answers to each question using a 5-point Likert scale (Strongly Agree to Strongly Disagree) [See Appendix].

We determined that our initial version of the EMS-SAQ had positive psychometric properties.³⁴ Specifically, tests revealed positive reliability and instrument validity scores. While we updated the instrument for this study, we did not alter the Core questions of the survey.

Data Collection and Processing

Eligible respondents were identified by each EMS agency. An individual was eligible if he/she worked full-time, part-time, or was a volunteer paramedic, EMT, first responder, prehospital nurse, or EMS physician who worked an average of at least 1 EMS shift per week. We excluded all managerial, administrative, or clerical personnel.

We administered the survey using two modalities: 1) paper forms, and 2) internet-based survey. Each EMS agency selected the modality deemed most convenient for the EMS agency. We permitted EMS agencies to use both modalities. The paper version of the survey consisted of questions printed in “bubble-sheet” format. Coordinators at each EMS agency supervised survey distribution. Respondents returned completed surveys in a self-addressed postage-paid envelope. We used a commercial survey vendor (keysurvey.com) to provide the electronic version of the survey. Each EMS agency provided the email for eligible employees. The vendor emailed a secure survey link to each potential participant. Up to three reminders were sent to non-respondents.

Completion of the survey was voluntary. The paper survey did not contain individual identifiers, and the electronic version was de-identified by the vendor prior to analysis. We used a separate paper-based instrument completed by the agency contact to determine characteristics of each EMS agency.

Outcome Measures

The primary outcomes were the survey scores for each safety domain: 1) safety climate [7 items], 2) job satisfaction [5 items], 3) perceptions of management [4 items], 4) teamwork climate [6 items], 5) working conditions [4 items], and 6) stress recognition [4 items]. We calculated the score for each domain using the method prescribed by Sexton, et al.³⁰ We first converted each Likert ranking to a point scale ranging from 0 to 100; Disagree Strongly=0, Disagree Slightly=25, Neutral=50, Agree Slightly=75, and Agree Strongly=100. We calculated the domain score by adding the individual response scores and dividing by the total number of items. For example, if a respondent answered Disagree Strongly, Neutral, Neutral, and Agree Slightly on the 4 items of Stress Recognition, the domain score would be 43.75.

Prior efforts also dichotomized the safety domain scores to “positive” (domain score ≥75) and “non-positive” (domain score <75) responses.³⁰ For example, if a respondent’s domain score for teamwork climate was 43.75, the respondent’s responses were classified as “non-positive.” To count as positive, a respondent would need to respond with an average response of Agree Slightly or higher. Following prior approaches by Sexton, et al., we examined these classifications as a proportion at the agency level and labeled it as the “percentage of positive responses” (PPR) [See Appendix].

Independent measures

EMS agency demographics were collected from agency contacts and included: agency type and geography, number of employees, number of annual patient contacts, agency affiliation, and percentage of patient contacts that were cardiac or trauma. EMS agencies designated their practice setting (i.e., rural ground, urban ground, air medical, or both ground and air medical). EMS agencies also reported their number of employees (i.e., 1-20 employees, 21-50, 51-100, or 101-400), number of annual patient contacts (i.e., ≤2,500, 2,501-5,000, 5,001-10,000, or >10,000), affiliation (i.e., hospital-based model, fire-based model, third service/government model, or private/free standing model), and the percentage of annual patient contacts that were cardiac arrests or trauma (i.e., ≤2% or >2%).

Individual characteristics included age category (i.e., 18-30, 31-40, 41-50, or >50), by total years of EMS experience (i.e., ≤5, 6-10, 11-15, 16-20, or >20), by total years at their current EMS agency (i.e., ≤5, 6-10, or >10), by position type (i.e., EMT-Basic, EMT-Intermediate, EMT-Paramedic, Prehospital Nurse, or Other), by employment status (i.e., career full-time, career part-time, or volunteer), by education (i.e., some high school, high school graduate or General Educational Development [GED], some college, college [e.g., Associate’s Degree or Bachelor’s], or college graduate level).

Primary Data Analysis

Like others before us,⁸^,³⁰^,³²^,³⁸ we used Confirmatory Factor Analysis (CFA) to determine if the items administered to our targeted sample actually measured the hypothesized 6 domain survey model. We used Cronbach’s Alpha to examine the inter-correlations amongst the 6 EMS-SAQ domains. Scores range from 0-100 for each domain with higher values indicating that a set of items measure a single domain/construct.³⁹^,⁴⁰ Values lower than 0.70 raise questions about item wording and interpretation and whether or not the construct includes the appropriate number of items.³⁹ We evaluated 3 standard measures of model fit to determine if our survey responses loaded onto a hypothesized 6 domain survey structure: the Root Mean Squared Error of Approximation (RMSEA), Bentler’s comparative fit index (CFI), and the Bentler and Bonett (1980) non-normed fit index (NNFI).⁴⁰^-⁴² An RMSEA less than 0.06, and CFI and NNFI greater than 0.9 are considered acceptable indices of instrument validity and model fit.⁴⁰^-⁴² The CFI and NNFI are less susceptible to sample size and considered compliments of the RMSEA.⁴⁰^-⁴²

We calculated mean domain scores and PPR for each individual EMS agency, depicting the variation graphically and with descriptive statistics. To identify potential associations between domain scores and individual and EMS agency characteristics, we used hierarchical linear models, modeling EMS agency as a fixed effect. For each characteristic, we fit two models; a linear model for the raw domain score and a logit model for the percentage of positive responses. We examined associations between the 6 EMS-SAQ domains and agency and individual demographic variables. We performed all analyses using SAS v.9.1 (SAS Inc., Cary, North Carolina)

RESULTS

Of the 62 EMS agencies participating in the study, we excluded one agency due to low response rate (9.6%). The remaining 61 agencies were distributed across all 4 U.S. Census regions and included one EMS agency from Canada (Figure 1). Most were rural-ground EMS agencies (Table 1). Most agencies employed between 21 and 50 employees and were affiliated with a private/freestanding model. Approximately 43% of all EMS agencies had ≤2,500 patient contacts in 2007.

Figure 1

Map of participating EMS agencies (n=61)

Table 1

Demographics of EMS agencies

We received 1,715 completed surveys from 61 agencies. The mean survey response rate per EMS agency was 47% (95% CI 10%, 83%). Response rates varied slightly by method for survey completion (paper only [n=16]=52%, combination of electronic and paper [n=3]=49%, and electronic only [n=42]=45%). We excluded 120 surveys that were missing 2 or more demographic variables. The final analysis included 1,595 surveys.

In this sample of EMS personnel, the survey responses demonstrated good internal consistency (Cronbach’s alpha for safety climate, alpha=0.82; teamwork climate, alpha=0.83; perceptions of management, alpha=0.68; job satisfaction, alpha=0.8; working conditions, alpha=0.75, stress recognition, alpha=0.78). Instrument validity testing confirmed the presence of a 6 domain structure and good model fit properties: RMSEA= 0.04, CFI=0.97, NNFI=0.95.

Most respondents were male and EMT-Paramedic certified (Table 2). The most common stratums were 18-30 (27.4%) and 31-40 (37.3%). The most common stratum for total years of EMS experience was less than 5 years (28.5%). The most common stratum for total years of experience at the current EMS agency was less than 5 years (44.9%). Three quarters of respondents (77.6%) were career full-time employees and half (50.3%) had an associate’s or bachelor’s degree.

Table 2

Characteristics of survey respondents

Agency mean domain scores varied across EMS agencies (Figures 2a-2f); Safety Climate 74.5 (95%CI 72.4, 76.6; Min 49.9, Max 89.7), Teamwork Climate 71.2 (95%CI 68.6, 73.7; Min 45.1, Max 90.1), Perceptions of Management 67.2 (95%CI 63.9, 70.5; Min 31.1, Max 92.2), Job Satisfaction 75.4 (95%CI 72.8, 78.0; Min 47.5, Max 93.8), Working Conditions 66.9 (95%CI 64.0, 69.7; Min 36.6, Max 91.4), Stress Recognition 55.1 (95%CI 52.9, 57.2; Min 31.3, Max 70.6).

Figures 2a-2f

Mean domain scores across agencies for the 6 EMS-SAQ domains

The mean safety climate score for air-medical EMS agencies was greater than mean scores in private/free standing and fire based model agencies (Table 3). The mean safety climate score was also highest in agencies with fewer employees, agencies with lower annual patient contacts, and agencies with a higher proportion of acute patients.

Table 3

Variations in EMS-SAQ domain mean scores across agency characteristics

The percentage of respondents with a positive perception (PPR) varied across EMS agencies for each domain: Safety Climate 58.6% (95%CI 52.7, 64.5; Min 0%, Max 100%), Teamwork Climate 52.8% (95%CI 46.5, 59.1; Min 0%, Max 100%), Perceptions of Management 43.8% (95%CI 36.7, 50.9; Min 0%, Max 90%), Job Satisfaction 63.4% (95%CI 57.5, 69.4; Min 0%, Max 100%), Working Conditions 46.3% (95%CI 41.4, 51.3; Min 0%, Max 100%), Stress Recognition 28.6% (95%CI 25.2, 32.0; Min 0%, Max 57.1%) (Figures 3a-3f).

Figures 3a-3f

Percentage of positive responses (PPR) across agencies for the 6 EMS-SAQ domains

The PPR for safety climate was highest among air-medical only agencies, but did not differ significantly across categories of model affiliation (Table 4). A lower proportion of EMS agency patient contacts was associated with increased PPR for safety climate. EMS agencies with greater than or equal to 2% of trauma and cardiac related patient contacts had a higher PPR than agencies with less than 2%. Notably, the PPR for stress recognition did not vary across any of the selected agency characteristics.

Table 4

Variations in the percentage of positive responses (PPR) across agency characteristics

The mean safety climate score was lower for EMT-Paramedics than the mean score for all other position types (Table 5). The mean safety climate scores were highest among respondents between the ages of 41 and 50, highest among respondents with less experience in EMS, and highest among respondents with less experience at the current EMS agency of employment when compared to their respective referent groups.

Table 5

Variations in EMS-SAQ domain mean scores across individual characteristics

The PPR for safety climate did not differ across most respondent demographic factors (Table 6). The PPR for safety climate was highest among Prehospital Nurses and other positions and was lowest among paramedics (p<0.0001; Table 6). The PPR for other domains of safety culture varied across some, but not all respondent demographic characteristics. Notably, the PPR for stress recognition varied across 1 of the 7 measured respondent demographics; Education. The PPR for stress recognition increased with categories of higher education.

Table 6

Variations in the percentage of positive responses (PPR) across individual characteristics

DISCUSSION

This study proposes and tests a survey adapted for EMS from a previously validated safety culture survey. Safety culture assessments are now common practice across most healthcare organizations. These assessments serve multiple purposes, including setting safety benchmarks, targeting problem areas, program evaluation, and meeting regulatory requirements. In this study sample, we observed wide interagency variation in workplace safety culture. Scores at the lower of this variation raise the question: Is the patient’s safety more susceptible in these EMS agencies much more so than in agencies with higher EMS-SAQ scores? Conversely, do higher scores suggest that select EMS agencies hold a greater awareness of safety and practice accordingly?

Wide variation in workplace safety culture is not surprising given that the EMS work environment contains many threats to patient and provider safety. Suyama, et al. showed that in one urban environment injury rates associated with lost time at work were higher among paramedics and EMTs than fire and police.⁴³ In a study of two urban EMS agencies, Maguire, et al. determined that the risk of injury among EMS personnel was 1.5 times higher than firefighters, 5.8 times higher than health services personnel, and 7 times higher than the national average reported by the U.S. Department of Labor.⁴⁴ Other studies show that many EMS personnel often deviate from written protocols, fail to properly secure patient airways, experience high levels of stress and burnout, suffer from poor sleep quality and high fatigue, and have a questionable commitment to the profession.¹⁵^,¹⁹^,²⁴^,⁴⁵^-⁴⁸ When combined, these factors may surface as non-positive perceptions of worker safety culture.⁴⁹

Prior safety culture studies have identified a pattern of variation in safety culture scores across settings. In an international study of safety culture, Sexton and colleagues’ identified wide variation in safety culture scores across 203 clinical units (i.e., operating rooms, ambulatory care settings, and ICUs).³⁰ A statewide study of ICU safety culture in Michigan revealed wide variation in scores across ICUs in a single state.⁸ Positive perceptions of teamwork climate ranged from a low of 16% to 92%. In a study of 4 ICUs, Huang et al discovered significant variation in scores within a single institution, with positive perceptions of safety climate ranging from approximately 30% to 50% and positive perceptions of job satisfaction ranging from 20% to 70% across ICUs.³¹ In this context our observation of wide safety culture variation across EMS agencies is not surprising. Potential factors underlying culture variation include regional practice differences, varying economic resources, and different leadership structure and styles.

A common mechanism for error or adverse event classification and reporting does not exist. Measurement of adverse events and medical errors in EMS is difficult. Research by Hobgood, et al. suggests that there is limited reliability and accuracy in paramedics and EMTs self-report of errors.⁵⁰ Threats to safety may be present absent actual errors or adverse events. While we believe that culture instruments could complement – but should not replace – direct AE measurement, culture is a potential contributor to poor safety.⁵¹ The overall utility of workplace safety culture instruments may lie in their ability to highlight safety conditions at individual EMS agencies. The EMS-SAQ offers a novel approach to patient safety, providing a barometer of safety attitudes rather than direct measures like errors or adverse events.

LIMITATIONS

While our convenience sample may not represent all ALS level EMS agencies in Northern America, few sampling alternatives exist. A valid and reliable list of all EMS agencies in the U.S. does not exist. Sampling individual EMS personnel akin to the National Registry’s Longitudinal Emergency Medical Technician Attributes and Demographics Study (LEADS) is not appropriate for studies of workplace safety culture.

Approximately 32% of our study sample includes EMS agencies from Minnesota. Among these agencies, mean domain scores were slightly lower for 5 of the 6 domains compared to all other agencies. Safety climate (70.7 vs. 76.6; p=0.006), teamwork climate (66.6 vs. 73.7; p=0.005), stress recognition (51.0 vs. 57.4; p=0.003), perceptions of working conditions (61.7 vs. 69.7; p=0.005), perceptions of management (63.9 vs. 68.9; p=0.141), job satisfaction (71.8 vs. 77.4; p=0.041). Findings should not be interpreted as representative of EMS nationwide.

Response rates across EMS agencies were similar to other studies of EMS agencies and individual EMS workers.⁴⁶^,⁵² Notably, we observed a slightly better average response rate in this study compared to other multi-site studies of safety culture.⁵³^-⁵⁶ Individual agency response rates below 60% may reduce the accuracy of perceptions of safety.⁵⁷ In our study, agencies with ≥60% (n=12) agency level response rates were not significantly different (p>0.05) than agencies with a response rate ≤59% (n=49) when comparing agency type and geography, number of employees, total patient contacts, or by percentage of patient contacts that were cardiac arrest or trauma. However, mean domain scores for perceptions of management and satisfaction were lower among agencies with a lower response rate than among agencies with a higher response rate (p<0.05).

Both respondent and agency factors may help explain differences in EMS-SAQ scores and thus represent residual confounding. Our study was not designed to identify a likely list of agency and respondent characteristics predictive of variations in safety culture scores. Rather, the primary purpose of our study was to characterize safety culture in the EMS setting using a reliable and valid measure of safety culture. Nonetheless, we conducted additional analyses to test for such differences which may be used to develop testable hypotheses in future research. We employed 12 hierarchical linear models with the 6 EMS-SAQ domain mean scores and 6 PPR proportions as outcomes and agency and individual variables as independent variables. We found few and potentially clinically insignificant differences in EMS-SAQ scores across agency and individual characteristics.

We did not examine the linkage between EMS-SAQ scores and adverse events or medical errors as in a recent studies of safety culture.⁸ Exploring this linkage in EMS would be methodologically difficult. Identifying medical errors and adverse events is a time-intensive exercise for which standards for identification and classification are limited.¹⁰

CONCLUSION

Workplace safety culture varies widely in this sample of EMS agencies. The EMS-SAQ can provide insights into prehospital safety.

Acknowledgments

Disclosure of funding

This study was supported by grants from the MedEvac Foundation International (www.fareonline.org) and Pittsburgh Emergency Medicine Foundation (PEMF www.pemf.net). During the execution of this study, Dr. Patterson was supported by the Emergency Medicine Patient Safety Foundation (EMPSF) and Society for Academic Emergency Medicine (SAEM) Patient Safety Research Fellowship. Dr. Wang was supported by Clinical Scientist Development Award K08-HS013628 from the Agency for Healthcare Research and Quality, Rockville, MD. Dr. Patterson is currently supported by a KL2 grant (KL2 RR024154) from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and NIH Roadmap for Medical Research. Dr. Fairbanks is supported by a Career Development Award from the NIBIB, K08-EB009090. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official view of NCRR or NIH. Information on NCRR is available at http://www.ncrr.nih.gov/. Information on Re-engineering the Clinical Research Enterprise can be obtained from http://nihroadmap.nih.gov/clinicalresearch/overviewtranslational.asp.”

Appendix A

Questions of the Emergency Medical Service Safety Attitudes Questionnaire (EMS-SAQ)

Respondents provided 5-point Likert responses to each question

(1=Disagree Strongly, 2=Disagree Slightly, 3=Neutral, 4=Agree Slightly, 5=Agree Strongly)

I Like my Job.
EMS Personnel input is well-received in this EMS agency.
I would feel safe being treated by this EMS agency as a patient.
Medical errors are handled appropriately at this EMS agency.
This EMS agency does a good job of training new personnel.
Working at this EMS agency is like being part of a large family.
The management of this EMS agency supports my daily efforts.
I receive appropriate feedback about my performance.
In this EMS agency, it is difficult to discuss errors.
Staff turnover at this agency is high.
This EMS agency is a good place to work.
Management does not knowingly compromise the safety of patients.
The levels of staffing at this EMS agency are sufficient to handle the number of calls.
I am encouraged by my colleagues to report any patient safety concerns I may have.
The culture at this EMS agency makes it easy to learn from the errors of others.
This EMS agency deals constructively with problem personnel.
At this EMS agency, it is difficult to speak up if I perceive a problem with patient care.
When my workload becomes excessive, my performance is impaired.
I am provided with adequate, timely information about events that might affect my work.
Many EMS personnel at this agency have other full-time or part-time jobs.
I have seen others make errors that had the potential to harm patients.
I know the proper channels to direct questions regarding patient safety.
I am proud to work at this EMS agency.
Disagreements at this EMS agency are resolved appropriately (i.e., not who is right, but what is best for the patient).
I am less effective at work when fatigued.
I am more likely to make errors in tense or hostile situations.
I have the support I need from other personnel to care for patients.
It is easy for personnel at this EMS agency to ask questions when there is something they do not understand.
Personnel here work together as a well-coordinated team.
I have co-workers who are actively looking for additional full-time or part-time work.
Morale at this EMS agency is high.
Trainees in my discipline are adequately supervised.
I have made errors that had the potential to harm patients.
Fatigue impairs my performance during emergency situations.
During emergency situations (i.e., cardiac arrests, traumas, etc.), my performance is not affected by working with inexperienced or less capable personnel.
Personnel frequently disregard rules or guidelines (i.e., treatment protocols, standard operating procedures, etc.) that are established for this EMS agency.
A confidential reporting system is helpful for improving patient safety.
I may hesitate to use a reporting system because I am concerned about being identified.
This agency provides me with the training to prevent ambulance driving accidents.
I have co-workers who are actively looking to leave this agency for other employment.
This agency could do more to improve emergency vehicle driver safety.
When moving a patient, I have the training to prevent injury to the patient.
When moving a patient, I have the right equipment to prevent injury to the patient.
All the necessary information for treating patients is routinely available to me.
Patient safety is constantly reinforced as the priority in this EMS agency.
Emergency vehicle or aircraft accidents occur at this EMS agency.
Emergency vehicle or aircraft accident close-calls (near-misses) occur at this EMS agency.
Patient handling mishaps (i.e., stretcher collapse, patient drop or fall, etc.) occur at this EMS agency.
Medical adverse events (i.e., incidents where a patient was harmed from medical care or medical equipment malfunction) occur at this EMS agency.
Medical adverse event close-calls (near-misses) occur at this EMS agency.

Appendix B

Mean domain score calculations

Questions (or items) are grouped by domain

a. Safety Climate Domain	(Questions: 3, 4, 8, 9, 14, 15, 22)
b. Teamwork Climate Domain	(Questions: 2, 17, 24, 27, 28, 29)
c. Stress Recognition Domain	(Questions: 18, 25, 26, 34)
d. Perceptions of Management Domain	(Questions: 7, 12, 13, 19)
e. Working Conditions Domain	(Questions: 5, 16, 32, 44)
f. Job Satisfaction Domain	(Questions: 1, 6, 11, 23, 31)

Questions 9 and 17 are reverse coded to match the positive valence of the other questions.
The Likert scale responses are coded to a 100 point scale.
- Disagree Strongly = 0
- Disagree Slightly = 25
- Neutral = 50
- Agree Slightly = 75
- Agree Strongly = 100
Scores for each question/item are totaled and divided by the total number of questions/items within each domain.

Percentage of positive responses (PPR) scores calculation

The percentage of positive responses (PPR) is the proportion of respondents that have a positive perception of a domain.³⁰

For each individual respondent, we identified positive (score ≥75) and non-positive (score <75) responses for each domain. For example, if a respondent’s average score for the four items that measure the Stress Recognition domain was 72, that respondent would be classified as a “non-positive response.” If a respondent’s average Stress Recognition score was 82, that respondent would be classified as a “positive response.” To be considered positive, a respondent would need to record an Agree Slightly or higher for each of the items within a given domain.
We identified the proportion of respondents with a positive perception for each domain

Contributor Information

David T. Huang, University of Pittsburgh School of Medicine, Department of Emergency Medicine and Center for Emergency Medicine of Western Pennsylvania, Inc., Iroquois Building, 3600 Forbes Avenue, Suite 400A, Pittsburgh, PA 15261; University of Pittsburgh School of Medicine, Department of Critical Care Medicine and Clinical Research and Systems Modeling of Acute Illness Laboratory [CRISMA], Email: huangdt/at/ccm.upmc.edu.

Rollin J. Fairbanks, University of Rochester School of Medicine, Department of Emergency Medicine, Email: terry.fairbanks/at/rochester.edu.

Scott Simeone, University of Pittsburgh School of Medicine, Department of Emergency Medicine and Center for Emergency Medicine of Western Pennsylvania, Inc., Iroquois Building, 3600 Forbes Avenue, Suite 400A, Pittsburgh, PA 15261, Email: simeones/at/upmc.edu.

Matthew Weaver, University of Pittsburgh School of Medicine, Department of Emergency Medicine and Center for Emergency Medicine of Western Pennsylvania, Inc., Iroquois Building, 3600 Forbes Avenue, Suite 400A, Pittsburgh, PA 15261, Email: weavermd/at/upmc.edu.

Henry E. Wang, University of Alabama at Birmingham School of Medicine, Department of Emergency Medicine, Email: hwang/at/uabmc.edu.

REFERENCES

1. Zohar D. Safety climate in industrial organizations: theoretical and applied implications. J Appl Psychol. 1980;65(1):96–102. [PubMed]

2. Joint Commission Sentinel Event Alert. Issue 40: Behaviors that undermine a culture of safety. [Accessed October 12, 2009]. 2008. http://www.jointcommission.org/SentinelEvents/Sentineleventalert/sea_40.htm.

3. Quality AfHRa Patient Safety Culture Surveys. [Accessed 10/12/2009, 2009]. 2009. http://www.ahrq.gov/QUAL/patientsafetyculture/

4. Improvement IfH Develop a Culture of Safety. [Accessed 10/12/2009, 2009]. 2009. http://www.ihi.org/IHI/Topics/PatientSafety/SafetyGeneral/Changes/Develop+a+Culture+of+Safety.htm.

5. Reinert A, Wingrove G. [Accessed 10/12/2009]. How the principles of Just Culture can lead to greater safety in EMS. EMSResponder.com. http://www.emsresponder.com/web/online/Rural-EMS-Resource-Guide/Rural-Responder--Learning-from-Our-Mistakes/49$9779. Published 6/26/2009.

6. Zohar D. A group-level model of safety climate: testing the effect of group climate on microaccidents in manufacturing jobs. J Appl Psychol. 2000;85(4):10. [PubMed]

7. Lee T. Assessment of safety culture at a nuclear reprocessing plant. Work Stress. 1998;12(3):217–237.

8. Pronovost PJ, Berenholtz SM, Goeschel CA, et al. Improving patient safety in intensive care units in Michigan. J Crit Care. 2008;23(2):207–221. [PubMed]

9. MacDonald RD, Banks BA, Morrison M. Epidemiology of adverse events in air medical transport. Acad Emerg Med. 2008;15(10):923–931. [PubMed]

10. Patterson PD, Martin-Gill C, Roth RN, et al. Identification of Adverse Events in ground EMS by medical directors. Prehosp Emerg Care. 2010;14(Suppl 1):15. Abstract.

11. Katz SH, Falk JL. Misplaced endotracheal tubes by paramedics in an urban emergency medical services system. Ann Emerg Med. 2001 Jan;37(1):32–37. [PubMed]

12. Dunford JV, Davis DP, Ochs M, Doney M, Hoyt DB. Incidence of transient hypoxia and pulse rate reactivity during paramedic rapid sequence intubation. Ann Emerg Med. 2003 Dec;42(6):721–728. [PubMed]

13. Wang HE, Lave JR, Sirio CA, Yealy DM. Paramedic intubation errors: isolated events or symptoms of larger problems? Health Aff (Millwood) 2006 Mar-Apr;25(2):501–509. [PubMed]

14. Wang HE, Weaver MD, Abo BN, Kaliappan R, Fairbanks RJ. Ambulance stretcher adverse events. Qual Saf Health Care. 2009;18(3):213–216. [PubMed]

15. Rittenberger JC, Beck PW, Paris PM. Errors of omission in the treatment of prehospital chest pain patients. Prehosp Emerg Care. 2005;9(1):2–7. [PubMed]

16. Ray AM, Kupas DF. Comparison of rural and urban ambulance crashes in pennsylvania. Prehosp Emerg Care. 2007;11(4):416–420. [PubMed]

17. Wang HE, Fairbanks RJ, Shah MN, Abo BN, Yealy DM. Tort claims and adverse events in emergency medical services. Ann Emerg Med. 2008;52(3):7. [PubMed]

18. Knox M. Independent Tribune. Concord, NC: Oct 8, 2006. Emergency shortage: Low pay, stressful work keep would-be paramedics away.

19. Patterson PD, Probst JC, Leith KH, Corwin SJ, Powell MP. Recruitment and Retention of EMTs: A qualitative study. J Allied Health. 2005;34(3):153–162. [PubMed]

20. Boudreaux E, Mandry C, Brantley PJ. Stress, job satisfaction, coping, and psychological distress among emergency medical technicians. Prehospital Disaster Med. 1997;12(4):9–16. [PubMed]

21. Slattery DE, Silver A. The hazards of providing care in emergency vehicles: an opportunity for reform. Prehosp Emerg Care. 2009;13(3):388–397. [PubMed]

22. Atack L, Maher J. Emergency medical and health providers’ perceptions of key issues in prehospital patient safety. Prehosp Emerg Care. 2010;14(1):95–102. [PubMed]

23. Patterson PD, Suffoletto BP, Kupas DF, Weaver MD, Hostler D. Sleep quality and fatigue among prehospital providers. Prehosp Emerg Care. 2010;14(2):187–193. [PMC free article] [PubMed]

24. Patterson PD, Moore CG, Sanddal ND, Wingrove G, LaCroix B. Characterizing job satisfaction and intent to leave among nationally registered emergency medical technicians: An analysis of the 2005 LEADS survey. J Allied Health. 2009;38(3):e84–91. [PubMed]

25. West CP, Tan AD, Habermann TM, Sloan JA, Shanafelt TD. Association of resident fatigue and distress with perceived medical errors. JAMA. 2009;302(12):1294–1300. [PubMed]

26. Chen I, Vorona R, Chiu R, Ware JC. A survey of subjective sleepiness and consequences in attending physicians. Behav Sleep Med. 2008;6(1):1–15. [PubMed]

27. Landrigan CP, Rothschild JM, Cronin JW, et al. Effect of reducing interns’ work hours on serious medical errors in intensive care units. N Engl J Med. 2004;351(18):10. [PubMed]

28. Lockley SW, Barger LK, Ayas NT, et al. Effects of health care provider work hours and sleep deprivation on safety and performance. Jt Comm J Qual Patient Saf. 2007;33(11 Suppl):7–18. [PubMed]

29. Dorrian J, Tolley C, Lamond N, et al. Sleep and errors in a group of Australian hospital nurses at work and during the commute. Appl Ergon. 2008;39(5):605–613. [PubMed]

30. Sexton JB, Helmreich RL, Neilands TB, et al. The Safety Attitudes Questionnaire: psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6(44):10. [PMC free article] [PubMed]

31. Huang DT, Clermont G, Sexton JB, et al. Perceptions of safety culture vary across the intensive care units of a single institution. Crit Care Med. 2007;35(1):12. [PubMed]

32. Modak I, Sexton JB, Lux TR, Helmreich RL, Thomas EJ. Measuring safety culture in the ambulatory setting: the safety attitudes questionnaire -- ambulatory version. J Gen Intern Med. 2007;22(1):5. [PMC free article] [PubMed]

33. Wisniewski AM, Erdley WS, Singh R, Servoss TJ, Naughton BJ, Singh G. Assessment of safety attitudes in a skilled nursing facility. Geriatr Nurs. 2007;28(2):11. [PubMed]

34. Patterson PD, Huang DT, Fairbanks RJ, Wang HE. The emergency medical services safety attitudes questionnaire. Am J Med Qual. 2010;25(2):109–115. [PubMed]

35. Thomas EJ, Sexton JB, Helmreich RL. Discrepant attitudes about teamwork among critical care nurses and physicians. Crit Care Med. 2003;31(3):4. [PubMed]

36. Makary MA, Sexton JB, Freischlag JA, et al. Operating room teamwork among physicians and nurses: teamwork in the eye of the beholder. J Am Coll Surg. 2006;202(5):746–752. [PubMed]

37. Colla JB, Bracken AC, Kinney LM, Weeks WB. Measuring patient safety climate: a review of surveys. Qual Saf Health Care. 2005;14(5):3. [PMC free article] [PubMed]

38. Waterson P, Griffiths P, Stride C, Murphy J, Hignet S. Psychometric properties of the hospital survey on patient safety culture: findings from the UK. Qual Saf Health Care. 2010 March 8; Epub ahead of print. [PubMed]

39. Nunnally J. Psychometric theory. McGraw-Hill; New York: 1978. New York.

40. Hatcher L. A Step-by-Step Approach to Using the SAS System for Factor Analysis and Structural Equation Modeling. SAS Institute Inc.; Cary, North Carolina: 1994.

41. Anderson JC, Gerbing DW. Structural equation modeling in practice: a review and recommended two-step approach. Psychol Bull. 1988;103(3):411–423.

42. Marsh HW, Balla JR, McDonald RP. Goodness-of-fit indexes in confirmatory factor analysis: the effect of sample size. Psychol Bull. 1988;103(3):411–423.

43. Suyama J, Rittenberger JC, Patterson PD, Hostler D. Comparison of public safety provider injury rates. Prehosp Emerg Care. 2009;13(4):451–455. [PubMed]

44. Maguire BJ, Hunting KL, Guidotti TL, Smith GS. Occupational injuries among emergnecy medical services personnel. Prehosp Emerg Care. 2005;9(4):405–411. [PubMed]

45. Wang HE, Lave JR, Sirio CA, Yealy DM. Paramedic intubation errors: isolated events or symptoms of a larger problem? Health Aff (Millwood) 2006;25(2):501–509. [PubMed]

46. Freeman VA, Slifkin R, Patterson PD. Recruitment and retention in rural and urban EMS: Results from a national survey of local EMS directors. J Public Health Manag Pract. 2009;15(3):246–252. [PubMed]

47. Chng CL, Eaddy S. Sensation seeking as it relates to burnout among emergency medical personnel: a Texas study. Prehospital Disaster Med. 1999;14(4):240–244. [PubMed]

48. Patterson PD, Suffoletto B, Hostler D, Kupas DF, Weaver M. Sleep quality and fatigue among prehospital providers. Prehosp Emerg Care. 2009;13(1):141–142. Abstract.

49. Yassi A, Hancock T. Patient safety -- worker safety: building a culture of safety to improve healthcare worker and patient well-being. Healthc Q. 2005;8:32–38. Spec No. [PubMed]

50. Hobgood C, Bowen JB, Brice JH, Overby B, Tamayo-Sarver JH. Do EMS personnel identify, report, and disclose medical errors? Prehosp Emerg Care. 2006;10(1):21–27. [PubMed]

51. Medicine Io. To Err is Human: Building a Safer Health System. National Academies of Science; Washington, DC: 2000.

52. Asch DA, Jedrziewski MK, Christakis NA. Response rates to mail surveys published in medical journals. J Clin Epidemiol. 1997;50(10):1129–1136. [PubMed]

53. Weingart SN, Farbstein K, Davis RB, Phillips RS. Using a multihospital survey to examine the safety culture. Jt Comm J Qual Saf. 2004;30(3):125–132. [PubMed]

54. Singer SJ, Gaba DM, Geppert JJ, Sinaiko AD, Howard SK, Park KC. The culture of safety: results of an organization-wide survey in 15 California hospitals. Qual Saf Health Care. 2003;12(2):112–118. [PMC free article] [PubMed]

55. Wagner LM, Capezuti E, Rice JC. Nurses’ perceptions of safety culture in long-term care settings. J Nurs Scholarsh. 2009;41(2):184–192. [PubMed]

56. Pringle J, Weber RJ, Rice K, Kirisci L, Sirio C. Examination of how a survey can spur culture changes using a quality improvement approach: a region-wide approach to determining a patient safety culture. Am J Med Qual. 2009;24(5):374–384. [PubMed]

57. Pronovost PJ, Sexton JB. Assessing safety culture: guidelines and recommendations. Qual Saf Health Care. 2005;14(4):231–233. [PMC free article] [PubMed]