Expert Panel Selection
We convened an expert panel for a one-day conference on November 4th, 2011 in Baltimore, Maryland. Panelists with direct experience and expertise in fields which contribute to the guidance, decision-making, and implementation of local and national interventions associated with mitigation of ED and/or hospital surge were selected by the advisory panel based upon their expertise in the field as evidenced by their record of publication, speaking engagements at national meetings, impact on national or regional policy, or representation of a directly related governmental organization. Of the 66 panelists invited, 34 (52%) attended the one day conference and were offered financial reimbursed for basic travel expenses only. To ensure appropriate representation, participants were selected from all key geographic areas of the U.S., and if a representative of a directly related governmental organization was unable to attend, they were asked to select a replacement to represent that organization. The panelists represented the fields of public health (8), disease surveillance (15), clinical medicine (13), emergency medicine operations (8), hospital operations (6), and systems experts (7), with some panelists representing more than one field. Representatives included those from the federal organizations (2), state and city health departments (6), corporate and non-profit organizations (2), and academic institutions (24). This study was reviewed and approved by the Institutional Review Board with a waiver of consent.
To accomplish our objectives, we used a mixed-methods design, involving an initial survey instrument followed by in-person discussion utilizing nominal group technique.15,16
Prior to the conference, the study team members (AD, MM, RB, KJ, RR) performed a review of the existing academic literature to identify potential ED and hospital-based interventions to manage crowding from a respiratory infectious disease outbreak.
This initial list of potential interventions was distributed to the 34 invited panelists who committed to attend the conference one month in advance. In addition, study investigators distributed definitions and brief descriptions of four types of infectious respiratory disease outbreaks to allow the panelists the opportunity to express their varied opinions on interventions relative to expected patient volume and severity of illness. These four scenarios were: 1) low volume/low severity (e.g. seasonal influenza), 2) high volume/low severity (e.g. 2009 H1N1), 3) low volume/high severity (e.g. Severe Acute Respiratory Syndrome [SARS]), and 4) high volume/high severity (e.g. 1918 H1N1).
Panelists were instructed to separately rate each intervention by both ease of implementation and importance for each of the four infectious respiratory disease outbreak scenarios using a 1-5 Likert scale. Ease of implementation included: consideration of factors such as cost, operational complexity, facility (time) of setup, and intensity of resource utilization. Importance was defined as the likely effectiveness of the intervention in reducing ED or hospital crowding or augmenting surge. Panelists were encouraged to suggest additional “write-in” interventions not already listed. Results were tallied in advance of the conference.
A full 1-day conference was held at the National Center for the Study of Preparedness and Catastrophic Response (PACER).17
Overview presentations were conducted by selected experts summarizing the current state of emergency department and hospital crowding, existing and novel surveillance mechanisms, and potential interventions to mitigate crowding as published in the peer-reviewed literature. The conference leaders then presented the results of the pre-conference survey and distributed these results to the panelists in tabular form. After these initial presentations, the panelists completed two exercises for which they were assigned to one of two equally sized groups, based upon panelists’ relevant expertise. One group evaluated ED-based interventions, and one evaluated hospital-based interventions. Each group was co-moderated by two subject matter experts (AD, MM, JP, JB, GK) with previous training through numerous instructional meetings and two mock moderation sessions to ensure uniform and unbiased facilitation of the proceedings.
During Exercise One, panelists were asked to prioritize potential interventions to mitigate surge in response to an infectious respiratory disease outbreak for each of four defined scenarios. Interventions were separated into two groups; those specifically for patients presenting with influenza-like illness (ILI) and those for patients without ILI. Infection control measures were included among the proposed interventions to focus efforts toward reducing disease spread and potentially reducing future patient volume. Using the nominal group technique, each of the two break-out groups (i.e. ED and hospital) was asked to discuss each associated intervention, (including the pre-panel “write-in” interventions), and suggest and similarly discuss any additional interventions not yet considered. Panel members were then asked to anonymously and independently categorize each intervention as low, medium, or high priority. Panelists were asked to consider both the ease of implementation and importance for a final overall priority consideration specific for each of the four respiratory infection outbreak scenarios.
In Exercise Two, panelists were asked to identify key data sources to trigger the implementation of a specific intervention in a given respiratory virus illness scenario. Considering the worst case scenario (high volume and high severity) and the recent 2009 H1N1 scenario (high volume, low severity), each group was given one intervention, which was highly rated in Exercise One, for each of the four scenarios. Additionally, each group was given a list of potential data sources based on those used in the literature, current medical practice, and current surveillance systems, but was also encouraged to consider and suggest alternate data sources that may serve as a “trigger” for implementation of the selected intervention. Examples of such data sources included ED wait times, hospital volume, laboratory data, syndromic and laboratory-based surveillance systems, and novel data sources that might be used to indicate increasing respiratory disease (e.g. Twitter). Panelists were asked to individually select the top three data sources that could be useful to ED or hospital leaders to “trigger” the decision to implement the top-rated interventions to mitigate ED or hospital crowding due to a respiratory infection outbreak. Similarly, using the nominal group technique, panelists proposed other potential data sources. All potential data sources were discussed and each panel member anonymously rated them on a 1-5 Likert scale with 1 being “not at all important” as a trigger and 5 as “very important” for a trigger for the selected interventions.
To evaluate data from Exercise One, final categorization data were assigned a numerical score where Low Priority = 1, Medium Priority =3, and High Priority =5. These numerical scores were averaged by scenario to obtain a numerical final score for each intervention in each scenario. These final means were then categorized as either, Low Priority (1.0-2.0), Medium Priority (2.1-4.0), or High Priority (4.1-5.0). Additionally, numerical scores were averaged across all scenarios to obtain a total prioritization score for each intervention across all scenarios. Finally, scores for each intervention within a scenario were averaged to obtain a scenario prioritization score. For Exercise Two, the individual resultant ratings for each potential data source were averaged to obtain a mean “importance” score for each potential data source.