PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmjoBMJ OpenVisit this articleSubmit a manuscriptReceive email alertsContact usBMJ
 
BMJ Open. 2017; 7(5): e013869.
Published online 2017 June 6. doi:  10.1136/bmjopen-2016-013869
PMCID: PMC5729970

Staffs’ and managers’ perceptions of how and when discrete event simulation modelling can be used as a decision support in quality improvement: a focus group discussion study at two hospital settings in Sweden

Abstract

Objective

To explore healthcare staffs’ and managers’ perceptions of how and when discrete event simulation modelling can be used as a decision support in improvement efforts.

Design

Two focus group discussions were performed.

Setting

Two settings were included: a rheumatology department and an orthopaedic section both situated in Sweden.

Participants

Healthcare staff and managers (n=13) from the two settings.

Interventions

Two workshops were performed, one at each setting. Workshops were initiated by a short introduction to simulation modelling. Results from the respective simulation model were then presented and discussed in the following focus group discussion.

Results

Categories from the content analysis are presented according to the following research questions: how and when simulation modelling can assist healthcare improvement? Regarding how, the participants mentioned that simulation modelling could act as a tool for support and a way to visualise problems, potential solutions and their effects. Regarding when, simulation modelling could be used both locally and by management, as well as a pedagogical tool to develop and test innovative ideas and to involve everyone in the improvement work.

Conclusions

Its potential as an information and communication tool and as an instrument for pedagogic work within healthcare improvement render a broader application and value of simulation modelling than previously reported.

Keywords: Simulation modeling, healthcare improvement, focus group, rheumatology, orthopedic care, emergency care

Strengths and limitations of this study

  • Customised simulation models were developed by the research team to address the specific needs of two hospital settings.
  • Focus group discussions were conducted in close connection to real healthcare improvement efforts.
  • The trustworthiness of the findings was strengthened by the fact that trained facilitators conducted the focus group and used a moderator guide, and that two researchers conducted the qualitative analysis.
  • The study includes two clinical settings, which limit the transferability of the findings.

Introduction

Healthcare improvement can be defined as joint efforts to improve patients’ health, healthcare operations and staff development.1 The complexity of healthcare improvement includes the size and scope of the initiative and the numerous contextual factors, which have caused the outcomes of healthcare quality improvement to be questioned.2–4 Improvement work often builds on testing changes, initially on a small scale, to foster knowledge of how changes are implemented and affecting practice.5 6 This strategy may make it difficult to predict outcomes not in close connection in time or space.7

Computer simulation modelling, such as discrete event simulation of healthcare systems and processes, has been proved as a valid tool for attending to problems in healthcare such as resource allocation, patient flows, epidemiological concerns and utilisation of resources.8–12

All types of simulation aim to imitate reality to test, educate and increase learning. Regardless of the many models developed to elucidate important healthcare related issues, little research has been done on the use, implementation and value of simulation in the everyday context of healthcare.13–15 However, research has shown that simulation modelling can enable informed decision making, develop system knowledge, determine critical factors for the development of an organisation, supply scenario analysis and options to choose from, help understand complex problems, facilitate communication and form plans and directions for future work.11 16

In the context of healthcare improvement, simulation modelling can help generate not only diagnostic data, but also knowledge and perspectives that can lead to predictive capacity.7 Even so, the uptake of simulation modelling and other operation research tools have been slow in healthcare rendering few actual model implementation cases to learn from.13 More research is needed to fully understand the impact and value of simulation modelling on the improvement of healthcare.8 12 14 15

The aim of this study was to explore healthcare staffs’ and managers’ perceptions of how and when discrete event simulation modelling can be used as a decision support in improvement efforts.

Method

Study design and setting

A qualitative study design was chosen to explore healthcare staffs’ and managers’ perceptions of discrete event simulation modelling, hereafter called simulation modelling, in two healthcare organisations: a rheumatology department and an orthopaedic section at a central surgical unit (which includes an orthopaedic department, an anaesthetic and intensive care unit and a surgical and urological department). The two units were part of two hospitals located in two different county councils in Sweden. An overview of the two organisational settings is presented in table 1.

Table 1
Overview of the included organisational settings

Study participants

Purposeful sampling was used to select the study participants. Thus, all healthcare staff and managers involved in the project groups of the respective ongoing improvement efforts were deemed suitable to participate in this study because of their knowledge of the routines and needs of the organisations. All potential study participants were contacted via telephone and email. Nine employees at the rheumatology department were invited. Of those, seven agreed to participate in the study. Two declined the invitation due to time constraints. At the orthopaedic section, eight employees were invited. Of those, seven accepted the invitation and one of those had a late cancellation due to acute illness and one declined due to a planned vacation. An overview of the study participants is presented in table 2.

Table 2
Study participants

Data collection

Data were collected through two focus group discussions (FGDs) at the two units. Focus groups can be defined as organised, interactive group discussions that aim to explore a certain topic.17 The method was chosen because focus groups are quite suitable when investigating experiences, attitudes and emerging ideas from a group.18 19 According to Morgan,20 p2 “the hallmark of focus groups is their explicit use of group interaction to produce data and insights that would be less accessible without the interaction found in the group.”

For both cases, customised simulation models were developed by the research team to address the units’ specific needs. At the rheumatology department, improvement efforts aimed to improve the referral process for newly diagnosed patients. At the orthopaedic section, improvement efforts aimed to improve access to care for patients with hip fracture. Both contexts had previous research and improvement collaborations with the authors.

The process of developing the two simulation models for the two contexts was somewhat distinct. In the rheumatology department, where authors had been involved in the improvement work, the participants were introduced to simulation modelling during that work as an optional tool for improvement. During a workshop, they had the opportunity to test a simulation model representing the processes in focus of the ongoing improvement and its included variables. Immediately after the workshop, the FGD was performed.

In the orthopaedic context, the participants were included in the simulation model building and validation of the simulation model during several meetings. The workshop and FGD were held at the end of the project time.

In both cases, the workshops began with a short introduction to simulation modelling and the researchers focus and questions. The researchers then presented the findings of the simulation model and led the FGDs on how and when simulation modelling could be used as a decision support in improvement efforts.

The FGD was conducted at the respective unit during 1 hour, and two trained researchers acted as moderators (CK and HHF in the rheumatology case and HHF and PM in the orthopaedic case). The more experienced researcher moderated the FGD and the other took notes and asked follow-up questions. The interviews were also recorded.

Before the initiation of the FGD, the participants were informed that the model developer would only be observing the FGD to let the participants freely discuss the model given their experience. The FGDs were initiated by asking all participants to comment briefly on their experience of the simulation model and their work with their specific questions. This ensured that all participants had a chance to share, and that the recordings included each participant. Following this introduction, the moderator’s questions guided the rest of the FGD. The following themes were covered in the moderator guide:

  • How was your experience of using the simulation model and the results?
  • What problems/issues/questions are suitable to address with simulation modelling in healthcare?
  • How can simulation modelling connect to improvement work?
  • What are the values and trustworthiness of simulation modelling?

Data analysis

The FGDs were transcribed verbatim by transcribers, which were then analysed using qualitative content analysis. Content analysis is commonly used in social sciences21 and is a systematic analysis of text.22 Two researchers (HHF and PM) conducted the coding and categorisation together, following the three general steps of performing content analysis outlined by Graneheim and Lundman.23 The transcripts were read through before meaning units were extracted and coded, and, finally, the codes were organised into categories. Each step was performed individually before collaboratively reviewing the analysed content and reaching a consensus.

Findings

Results presented below are organised according to the different categories derived from the qualitative content analysis.

When can simulation modelling be used?

Improvement support

The FGD participants described simulation as a tool that could be used to motivate change for staff and management. This included changes that present challenges when implementing because of budget constraints, big investments, staff resistance or lack of consensus. Simulation modelling can support change implementation by visualising both problems and potential solutions. Simulation completes the picture of the limited financial or process aspects of change by visualising effects that are different dimensions of the same change.

Test and evaluate ideas for improvement

Participants expressed repeatedly in both FGDs that simulation modelling could be used to evaluate the effect of changes that have either been implemented or that are under development or at an idea stage. Simulation modelling was considered a quicker and more efficient way to test new ideas, compared with testing in reality. Furthermore, changes can be tested without influencing patient care.

“It is exciting to be able to test a hypothesis in a computer environment to see what results you can anticipate,” said one FGD participant. “It is often difficult to test changes in real life; it takes time, costs money and results are sometimes uncertain.”

Participants also emphasised that simulation modelling could more systematically test change ideas and evaluate them, rather than just implementing changes without following up. In addition, they also shared how the model could be used as a pedagogical tool to develop and test ideas that could motivate staff to get engaged and involved in the improvement work.

“It's a pedagogical tool to use in our work team in improvement work. By thinking that nothing is forbidden to suggest, we can test many different things and start thinking outside the box. It is so easy to get stuck in patterns,” another participant commented.

Moreover, testing of ideas got some of the participants to think about existing work routines, and new improvement ideas were created, while previous ideas were rejected.

“We thought that if we just had more rooms, all our problems would go away,” a participant said. “But we saw no substantial change when we tested it in the model. It all depends on the number of doctors that are available after all; that is the answer we got. We even saw that, right now, we are not using all the rooms efficiently.”

How can simulation modelling be used?

Questions suitable for simulation modelling

Simulation modelling can be used to address concrete and measurable questions, including questions related to planning, resource allocation and staff scheduling. One area of application that was raised several times during the FGDs was related to cost efficiency.

“The future will be more about budgets and finances due to our expensive treatments. This [simulation model] can be a way to present financial figures to the management,” commented one participant.

“It is easier to connect numbers with a cause rather than to just talk about the cause itself,” said another participant.

This [simulation model] includes showing the cost efficiency of changes implemented and to make staff more aware of costs related to certain operations. Further on, it can be used to find ways to use resources more efficiently.

Validity

Participants stressed the importance of building the model from valid and reliable data. Using incorrect data or data in an incorrect way will undermine the simulation model and the staff´s trust in the model. The simulation model must reflect the organisation and real processes and data. Input from staff is important to ensure the validity of the model as they are the ones with knowledge of processes and operations.

As one participant put it, “The method is fine; the difficult part is what you put into the model.”

Participants expressed that to use any simulation model, it is important that all processes and logics incorporated in the model are well known to the user. This is to prevent misinterpretation of the model output and results. Relying on and trusting the model requires a deep understanding of the model and participation during the process of building the model.

“Users must be familiar with the model in order to critically evaluate results. [It’s] dangerous if results are interpreted in the wrong way,” expressed one participant.

Conducting simulation modelling projects

Results from the FGDs revealed two aspects of simulation modelling use: local use at the clinic in their improvement work and to guide management on planned changes or responses to changes suggested by management. Staff must be involved from the beginning to inform the building of the model with data and questions to test in the model. Involving staff from the beginning also helps develop trust in the simulation model.

“A simulation project must be approved by management and well supported by the staff,” said one participant.

“In improvement work, involvement from everyone is essential, perhaps simulation modeling can act as a tool to inform and inspire colleagues,” commented another.

Simulation modelling projects must be approved by management and staff and be well communicated within the organisation. The group working with the simulation model cannot be too big and there must be a person in charge of the work. The importance of using the simulation model together, in multidisciplinary teams, was emphasised. Specific persons holding positions, such as schedulers, nurse managers and heads of departments were mentioned as potential users.

“Simulation modeling should be used locally, at our clinic and by us,” a person said. “It should be used to show how we work and our results, at meetings and externally as marketing.”

Another participant described the simulation model as, “a good tool to show management what we do and our plans.”

Value and opportunities

Overall, participants expressed a positive attitude towards simulation modelling. They identified different opportunities for simulation modelling to add value to the development of their organisation. These include identifying trends and understanding complex relationships between processes and systems. Moreover, simulation can help the staff face future challenges collaboratively.

The following quotes help illustrate the overall outlook of simulation modelling among the participants:

“Simulation modeling at the right time can be very valuable. When initiating improvement, it is beneficial to see how small changes can have great effects.”

“You get an overview; you see the overall picture that can be used to stimulate improvement work.”

“Simulation modeling can help you find new models and aid the individual learning when seeing the relation to their respective work. To the organization, it might be more on how to accomplish goals and see how the overall picture is affected.”

Discussion

This study sets out to explore healthcare staffs’ and managers’ perceptions of how and when simulation modelling can be used as a decision support in improvement efforts. Simulation was described as a tool that could be used to evaluate and develop improvement ideas and help motivate the need to implement certain changes both for staff and for management. Also, simulation modelling can motivate difficult change by visualising effects and also financial aspects. Simulation modelling was best valued as a way to address concrete and measurable questions related to planning, resource allocation, staff scheduling and cost efficiency. Two areas of simulation modelling use in healthcare improvement were stressed in the two focus groups: locally in the clinical improvement work and to guide management on planned changes. Also, the early involvement of staff in the simulation project and use of correct data to validate the model is crucial for staff to trust and use the model.

When comparing the results to previous literature on simulation modelling, there are several evident similarities. Concerning the potential users of the simulation model, at the clinical level or in management, it is important to consider model knowledge and the ability to present it to others, data availability and support from leadership.14 Regarding testing and exploring what is outside of the box, Pidd describes simulation as a vehicle for experimentation where trial and error can be performed without concerns for reality but with a great opportunity to learn.23 Simulation modelling offers a holistic view on addressing change and improvement in complex systems and its inherent components by displaying the effects of change immediately.7 Visualising the problem and a potential solution was, in the FGDs, considered powerful in engaging healthcare staff in improvement work. This interactive opportunity has been proposed to help motivate staff to embrace change.7 However, the formation of a representable clinical team to facilitate the simulation modelling work is not always easy.15 Our first simulation focus group revealed a wish to include management, and the second focus group wanted to include the staff further. Coproducing the model with staff, managers and modellers, also known as facilitated modelling, are important to build a step-by-step understanding of the model logic and its validity.24–29 The early and continuous involvement of stakeholder in a structured model development process increases stakeholders trust in the model.30 Depending on the identified users, simulation models can be used to aid communication in creating a shared mental model.26 31–33

Drawing from our results, we can see values emerging from simulation modelling, such as providing a way to work with change and improvement (especially when initiating improvement work), and visually communicate planned changes, operations and the subsequent consequences. Simulation modelling can be a tool to be used at different levels of healthcare, but according to our findings, it might be best suited for local clinical improvement and management planning and allocating resources. Before initiating a simulation modelling project, the formation of the project team is essential for ensuring model validity and reliability. Bringing in the aspects of implementation, evaluation and research requires even greater consideration of the project team, but might enable the transition from in silico to in reality and create value for the healthcare organisation.

Limitations in our study design include the fact that we have drawn conclusions from limited empirical data gathered from two FGDs only. Nevertheless, an exploratory study like this could serve as a pilot for further research. All staff and managers involved in the improvement project teams were invited to attend the FGDs and thus the concept of saturation was not applicable. FGDs themselves can be subject to different impediments if not moderated well. The group dynamics are essential for allowing everyone to talk and present their opinion.18 34 35 Using trained facilitators, a moderator guide and introductory questions for everyone to answer may have helped participants feel free to share their perceptions openly.

Conclusions

The emerging categories from the content analysis are presented according to the research questions of how and when simulation modelling can assist healthcare improvement. Regarding how, the participants mentioned that simulation modelling can act as a tool for support and a way to visualise problems and potential solutions, as an information and communication tool to show management planned improvements and their effect. Simulation modelling could be used both locally and by management, but the user should be familiar with model logic and data to interpret results correctly. Relating to when, participants thought simulation modelling could be used as a pedagogical tool to develop and test ideas, to think outside the box and to get everyone involved in the improvement work.

This study showed that simulation modelling has more to offer than has been described in previous literature. Its potential as an information and communication tool, an instrument for pedagogic work within healthcare improvement and as a way to allow thinking outside the box render a broader application and value of simulation modelling.

10.1136/bmjopen-2016-013869.supp1Supplementary data:

bmjopen-2016-013869supp001.docx

Supplementary Material

Reviewer comments:
Author's manuscript:

Acknowledgments

The authors wish to thank the involved units for their participation in this study.

Footnotes

Contributors: HHF, PM, MU, DG and CK designed the study. HHF, PM, MU and CK collected the data. HHF, MU, PM and CK drafted the manuscript. Everyone have been involved in reading and critically revising the manuscript. All authors approved the final manuscript and are accountable for all parts of the work.

Funding: The study was supported by grants provided by the Stockholm County Council (ALF project).

Disclaimer: The funder had no involvement in study design, data analysis and manuscript preparation or publication decision.

Competing interests: None declared.

Ethics approval: Stockholm Ethical Review Board.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi:10.5061/dryad.5f5m7.

References

1. Batalden PB, Davidoff F What is 'quality improvement' and how can it transform healthcare? Qual Saf Health Care 2007;16:2–3. doi:10.1136/qshc.2006.022046 [PMC free article] [PubMed]
2. Blumenthal D, Kilo CM A report card on continuous quality improvement. Milbank Q 1998;76:625–48. doi:10.1111/1468-0009.00108 [PubMed]
3. Kaplan HC, Brady PW, Dritz MC, et al. The influence of context on quality improvement success in health care: a systematic review of the literature. Milbank Q 2010;88:500–59. doi:10.1111/j.1468-0009.2010.00611.x [PubMed]
4. Thor J, Herrlin B, Wittlöv K, et al. Evolution and outcomes of a quality improvement program. Int J Health Care Qual Assur 2010;23:312–27. doi:10.1108/09526861011029370 [PubMed]
5. Langley GJ. The improvement guide: a practical approach to enhancing organizational performance. San Francisco: Jossey-Bass, 2009.
6. Shewhart WA, Deming WE Statistical method from the viewpoint of quality control. New York: Dover, 1986.
7. Slovensky DJ, Morin B Learning through simulation: the next dimension in quality improvement. Qual Manag Health Care 1997;5:72–9. [PubMed]
8. Fone D, Hollinghurst S, Temple M, et al. Systematic review of the use and value of computer simulation modelling in population health and health care delivery. J Public Health Med 2003;25:325–35. doi:10.1093/pubmed/fdg075 [PubMed]
9. Brailsford SC, Harper PR, Patel B, et al. An analysis of the academic literature on simulation and modelling in health care. Journal of Simulation 2009;3:130–40. doi:10.1057/jos.2009.10
10. Günal MM, Pidd M Discrete event simulation for performance modelling in health care: a review of the literature. Journal of Simulation 2010;4:42–51. doi:10.1057/jos.2009.25
11. Forsberg HH, Aronsson H, Keller C, et al. Managing health care decisions and improvement through simulation modeling. Qual Manag Health Care 2011;20:15–29. doi:10.1097/QMH.0b013e3182033bdc [PubMed]
12. Jun JB, Jacobson SH, Swisher JR Application of discrete-event simulation in health care clinics: a survey. J Oper Res Soc 1999;50:109–23. doi:10.1057/palgrave.jors.2600669
13. Brailsford S. Overcoming the barriers to implementation of operations research simulation models in healthcare. Clin Invest Med 2005;28:312–5. [PubMed]
14. Brailsford SC, Bolt TB, Bucci G, et al. Overcoming the barriers: a qualitative study of simulation adoption in the NHS. J Oper Res Soc 2013;64:157–68. doi:10.1057/jors.2011.130
15. Monks T, Pearson M, Pitt M, et al. Evaluating the impact of a simulation study in emergency stroke care. Operations Research for Health Care 2015;6:40–9. doi:10.1016/j.orhc.2015.09.002
16. Tako AA, Kotiadis K, Vasilakis C, et al. Improving patient waiting times: a simulation study of an obesity care service. BMJ Qual Saf 2014;23:373–81. doi:10.1136/bmjqs-2013-002107 [PMC free article] [PubMed]
17. Becker S, Bryman A (Red) Understanding research for social policy and practice: themes, methods and approaches. Bristol: Policy, 2004.
18. Krueger RA, Casey MA Focus groups: a practical guide for applied research. Thousand Oaks, Calif: Sage Publications, 2000.
19. Pope C, van Royen P, Baker R Qualitative methods in research on healthcare quality. Qual Saf Health Care 2002;11:148–52. doi:10.1136/qhc.11.2.148 [PMC free article] [PubMed]
20. Morgan DL. Focus groups as qualitative research. Thousand Oaks, Calif: Sage Publications, 1997.
21. Dixon-Woods M, Agarwal S, Jones D, et al. Synthesising qualitative and quantitative evidence: a review of possible methods. J Health Serv Res Policy 2005;10:45–53. doi:10.1177/135581960501000110 [PubMed]
22. Krippendorf K. Content analysis: an introduction to its methodology. Thousand Oaks, Calif: Sage, 2004.
23. Graneheim UH, Lundman B Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today 2004;24:105–12. doi:10.1016/j.nedt.2003.10.001 [PubMed]
24. Pidd M. Computer simulation in management science. Hoboken, NJ: Wiley, 2004.
25. Aharonson-Daniel L, Paul RJ, Hedley AJ Management of queues in out-patient departments: the use of computer simulation. J Manag Med 1996;10:50–8. doi:10.1108/02689239610153212 [PubMed]
26. Alkaabi R, El Halim AOA, Mahmoud S Improving resource allocation efficiency in health care delivery systems 19th annual canadian conference on electrical and Computer Engineering, May 07-10. Ottawa, Canada, 2006:2360–5.
27. Cochran JK, Bharti A Stochastic bed balancing of an obstetrics hospital. Health Care Manag Sci 2006;9:31–45. doi:10.1007/s10729-006-6278-6 [PubMed]
28. Banks J. Handbook of simulation: principles, methodology, advances, applications, and practice. New York: Wiley, 1998.
29. Franco LA, Montibeller G Facilitated modelling in operational research. Eur J Oper Res 2010;205:489–500. doi:10.1016/j.ejor.2009.09.030
30. Tako AA, Kotiadis K PartiSim: A multi-methodology framework to support facilitated simulation modelling in healthcare. Eur J Oper Res 2015;244:555–64. doi:10.1016/j.ejor.2015.01.046
31. Elkhuizen SG, Das SF, Bakker PJ, et al. Using computer simulation to reduce access time for outpatient departments. Qual Saf Health Care 2007;16:382–6. doi:10.1136/qshc.2006.021568 [PMC free article] [PubMed]
32. Heinrichs M, Beekman R, Limburg M Simulation to estimate the capacity of a stroke unit. Stud Health Technol Inform 2000;77:47–50. [PubMed]
33. Rytilä JS, Spens KM Using simulation to increase efficiency in blood supply chains. Management Research News 2006;29:801–19. doi:10.1108/01409170610717826
34. Robson C. Real world research: a resource for social scientists and practitioner-researchers. Blackwell Publishers: Oxford, 2002.
35. Marshall C, Rossman GB Designing qualitative research. Thousand Oaks, Calif: Sage, 2006.

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group