|Home | About | Journals | Submit | Contact Us | Français|
The efficacy of new antibacterial agents for the treatment of hospital-acquired pneumonia (HAP) and ventilator-associated pneumonia (VAP) has typically been compared with that of established antibacterial agents in noninferiority clinical trials. However, the US Food and Drug Administration (FDA) has reevaluated the appropriateness of noninferiority trial designs for a variety of diseases, including HAP and VAP. The resulting regulatory uncertainty regarding appropriate trial design is an important barrier to the development of new antibacterial agents.
After a recent, successful workshop focusing on community-acquired pneumonia (CAP) that was cosponsored by the Infectious Diseases Society of America (IDSA) and the FDA, the FDA released a draft guidance on the design of trials for community-acquired bacterial pneumonia (CABP) that has greatly clarified regulatory expectations for such studies. In the guidance, the FDA specifically referred to the disease entity as CABP rather than CAP to emphasize the critical need to establish a bacterial etiology of infection for noninferiority clinical trials of the disease.
After the successful workshop on CABP, the FDA, the IDSA, the American Thoracic Society (ATS), the Society of Critical Care Medicine (SCCM), and the Ameri-can College of Chest Physicians (ACCP) jointly sponsored a follow-up workshop focusing on hospital-acquired bacterial pneumonia (HABP) and ventilator-associated bacterial pneumonia (VABP) from 31 March through 1 April 2009. In accordance with the precedents established by the FDA guidance on CABP, the follow-up workshop focused specifically on HABP and VABP (as opposed to HAP and VAP) to underscore the need to establish a microbiological diagnosis during clinical trials of antibacterial agents for treatment of these diseases.
The workshop provided a forum for scientific discussion to clarify appropriate design elements of clinical trials of HABP and/or VABP. This position paper reflects the conclusions and suggestions of the societies that resulted from the workshop. For topics on which clear consensus could not be achieved or on which strongly held dissenting opinion was evident, alternative design features are presented.
Data reviewed at the workshop and summarized in this supplement and position paper make clear that there is an unequivocal and substantial treatment effect of antibiotic therapy for HABP and VABP. Thus, noninferiority trials are appropriate for the study of experimental antibacterial agents for the treatment of HABP and/or VABP. On the basis of the reviewed data, the societies support the following design features for registration studies of HABP and/or VABP.
The current uncertainty in acceptable designs for clinical trials of HABP and/or VABP is contributing to disincentives in the discovery and development of new drugs for these diseases. After a related workshop on CAP, the FDA released a guidance document that provided clear directions for conduct of trials of CABP. The societies desire similar approval and dissemination of clear and scientifically and clinically defensible guidelines for future clinical trials of new antibacterial agents for the treatment of HABP and VABP.
Nosocomial pneumonia, including HAP and VAP, is the second leading type of nosocomial infection and the leading cause of death from nosocomial infection in the United States [1–3]. An estimated 300,000 HAP and VAP infections occur per year in the United States, and the mortality rate among patients with HAP and/or VAP is ≥20% despite treatment [3, 4]. Furthermore, increasing antibacterial resistance because of the increasing incidence of MDR, XDR, or truly pan–drug-resistant gram-negative bacilli continues to increase the mortality associated with these infections [5–16].
Unfortunately, at the same time that increasing drug resistance has created a crucial need to develop new treatments, the development of new antibacterial agents has been decreasing dramatically [17, 18]. Uncertainty about regulatory requirements for the appropriate design of clinical trials testing the efficacy of antibacterial agents is a major barrier to research and development and likely has contributed to the decrease in availability of new antibacterial agents . In January 2008, the IDSA and the FDA jointly convened a workshop to elucidate an appropriate clinical trial design for CAP . The workshop allowed experts from academia, industry, and the FDA to share pertinent knowledge about clinical trials for CAP. On the basis of the scientific and regulatory discussions at the workshop, the IDSA published a position paper synthesizing the crucial elements of appropriate trial design for CAP . Subsequently, the FDA released a draft guidance on the design of trials for CABP , which has greatly clarified regulatory expectations for such studies. In the guidance document, the FDA specifically referred to the disease entity as CABP rather than CAP, to emphasize the crucial need to establish a bacterial etiology of infection for noninferiority clinical trials of the disease.
After the successful workshop on CABP, the FDA, the IDSA, the ATS, SCCM, and ACCP jointly sponsored a follow-up workshop focusing on HABP and VABP from 31 March through 1 April 2009. In accordance with the precedents established by the FDA guidance on CABP, the follow-up workshop focused specifically on HABP and VABP (as opposed to HAP and VAP) to underscore the need to establish a microbiological diagnosis during clinical trials of antibacterial agents for treatment of these diseases.
This position paper is based on the data presented, discussions held, and opinions expressed at the HABP and/or VABP workshop and an ongoing dialogue subsequent to the workshop. Conclusions and suggestions presented in this document are those of the societies. There is no intent to represent the views of industry or the FDA. The societies’ goal is to consider the data and represent the best interests of patients by providing clarity to clinical investigators, clinicians, the pharmaceutical industry, and regulatory officials regarding appropriate clinical trial design for the study of investigational antibacterial agents in the treatment of HABP and VABP. For topics on which a clear consensus could not be achieved or on which strongly held dissenting opinion was evident, alternative design features are presented.
Consideration is given to 8 specific aspects of clinical trial design for HABP and/or VABP: (1) justification for a noninferiority versus a superiority hypothesis; (2) primary and secondary end point evaluations and the patient populations in which they should be assessed; (3) enrollment criteria including microbiological diagnostic methodologies; (4) advisability and difficulties with study of HABP and VABP in the same clinical trial rather than in separate trials; (5) appropriate standard comparator and adjunctive therapy; (6) factors by which enrollment should be stratified; (7) trial integrity issues, including blinding, use of international sites, and the desirability of a clinical trials network; and (8) core components of a HABP and/or VABP clinical trials program.
The inherent difficulty of conducting clinical trials to determine whether new antibacterial agents are superior in efficacy, compared with approved agents has been discussed elsewhere [20, 22]. In brief, new antibacterial agents are more likely to achieve superior efficacy than are comparator drugs when used to treat infections caused by organisms resistant to the comparator drugs. However, patients with infections caused by organisms resistant to standard comparator drugs are excluded from enrollment in clinical trials. Therefore, new antibacterial agents typically cannot be tested in the very patients in whom they are likely to achieve superior efficacy, compared with comparator drugs. It is not surprising, therefore, that all recent trials of antibacterial agents for HABP and/or VABP have been noninferiority studies [23, 24]. Situations exist in which superiority trials of antibacterial agents for HABP and/or VABP would be both feasible and desirable (discussed further below); however, in most instances, clinical trials of new antibacterial agents for the treatment of HABP and/or VABP are likely to be noninferiority trials.
According to guidance documents from the International Congress on Harmonization (ICH) [25, 26] and the FDA , noninferiority trials are appropriate only when a comparator drug has been established previously to be superior in efficacy to placebo or no therapy for the disease in question (ie, the historical evidence of sensitivity to drug effect standard). Furthermore, the clinical contexts in which the efficacy of the comparator drug was previously established must be relevant to the planned noninferiority trial (ie, the constancy assumption standard). Unfortunately, as is true of other severe infections [20, 22], no placebo-controlled studies of antibacterial agents for the treatment of HABP and/or VABP are available, because antibacterial agents became available in an era before the widespread use of placebo-controlled studies. Furthermore, active antibacterial agents were already being used to treat HABP and/or VABP before the FDA designation of these infections as antibiotic indications. The lack of placebo-controlled studies complicates justification of noninferiority margins for new antibacterial agents for the treatment of HABP and/or VABP.
To evaluate evidence about the acceptability of a noninferiority design for clinical trials of HABP and/or VABP, Sorbello et al  from the FDA conducted an extensive search of the literature from the period 1969–2008. They focused on studies of delayed initiation of effective antibacterial therapy for HABP and/or VABP as a proxy for placebo or no therapy data. A substantial number of studies have evaluated the impact of delayed initiation of effective therapy for HABP and/or VABP [28–39]. Sorbello et al  reviewed the subset of these studies that most closely reflected the patient age and severity of illness in recent registration studies of antibacterial agents for the treatment of HABP and/or VABP. Their analysis revealed a ≥29% absolute reduction in mortality among patients with HABP and/or VABP treated with active antibacterial therapy (ie, therapy to which the etiological organism was susceptible in vitro), compared with when initial antibacterial therapy was inactive for the organism causing the infection (ie, therapy to which the etiological organism was resistant in vitro).
The primary limitations of this estimate are the reliance on meta-analysis of non-randomized studies of delayed initiation of active therapy and the absence of placebo-controlled trials . Nevertheless, Sorbello et al  used conservative random-effects methods to analyze the data. Furthermore, the estimate of antibacterial efficacy based on delayed initiation of effective antibacterial therapy is likely to be inherently conservative, because the duration of delay in initiation of effective therapy in the analyzed studies was typically 1–3 days. It seems probable that the mortality rate associated with HABP and/or VABP episodes that remained untreated during the entire duration of illness would be substantially higher, compared with the mortality associated with a <72 h delay in initiation of effective antibacterial therapy.
A specific concern about the analysis discussed at the workshop was that delayed initiation of effective therapy might be more likely to occur in more severely ill patients with a higher mortality rate due to their underlying diseases, compared with patients who received initially effective antibacterial therapy. However, initial discordant therapy is most likely to occur when patients are infected with MDR pathogens. As indicated in the consensus ATS and IDSA guidelines on the management of nosocomial pneumonia  and as summarized in the current supplement [41–44], baseline disease severity does not correlate with risk that HABP and/or VABP is caused by MDR pathogens. Instead, the factors associated with infection due to MDR pathogens and, thus, associated with increased risk of receipt of initially ineffective antibacterial therapy include prior exposure to antibiotics and exposure to environments in which MDR organisms are present (discussed further below).
The reliability of the aforementioned estimate of efficacy of antibacterial therapy for HABP and/or VABP, compared with placebo or no therapy, is substantiated by other data. For example, in accordance with the analysis by Sorbello et al , an independent analysis of the literature on delayed initiation of effective antibacterial therapy for HABP and/or VABP included the results of all identified studies of delayed initiation of effective therapy . Thus, this second evaluation serves as a useful sensitivity analysis of the estimate of antibacterial efficacy derived by Sorbello et al , based on their more focused analysis. The broader, random-effects meta-analysis found a ≥33% reduction in mortality when initial antibacterial therapy was effective, compared with when it was ineffective —a result similar to that generated by the more focused meta-analysis of Sorbello et al . The concordance of the broader analysis, which incorporated more studies with more variation in underlying disease severity and patient age, provides reassuring evidence that the estimate of the mortality benefit of effective antibacterial therapy for HABP and VABP is robust.
Additional evidence that the estimate of antibacterial efficacy for nosocomial pneumonia is robust is provided by natural history studies of untreated pseudomonal nosocomial pneumonia [46, 47]. These studies found that ~60% of such patients died without therapy, similar to the meta-analytic estimates of mortality related to delayed initiation of antibacterial therapy in more recent studies.
Historical literature identified after the workshop lends further credence to the substantial efficacy of antibacterial therapy for nosocomial pneumonia. For example, in 1952, Kassowitz and Muscato  published data from >74,000 admissions over 20 years to a pediatric hospital to determine the efficacy of antibacterial therapy for the treatment of pulmonary infections. The period analyzed spanned the pre- and immediate postantibiotic era. With a specific focus on the subset of patients who developed nosocomial pneumonia (termed “secondary pneumonia”), the mortality rate was >50% every year before 1936. In 1936, immediately after the availability of sulfonamide therapy, mortality rates decreased to ~20%, reflecting an absolute 30% reduction in mortality resulting from sulfonamide therapy; other studies showed that sulfonamide therapy was substantially less effective than penicillin therapy . Furthermore, Glew et al  evaluated the impact of effective versus ineffective therapy on mortality in 25 patients with pneumonia caused by Acinetobacter species. The mortality associated with pneumonia treated with effective antibiotics was 14%, compared with an 82% mortality rate among patients treated with ineffective antibiotics. Finally, the magnitude of efficacy of antibiotics for the treatment of HABP and/or VABP appeared to be similar to the magnitude of efficacy of antibiotics for treatment of the most severe forms of CAP reviewed at the previous workshop and in subsequent proceedings [20, 50, 51].
These collective data, derived from multiple independent sources, provide considerable, robust evidence of the accuracy of the estimate of the minimal effect size of antibacterial therapy for HABP and/or VABP. A conservative estimate is that effective antibacterial therapy results in a 30% absolute reduction in mortality associated with HABP and/or VABP, compared with placebo or no therapy. The large effect size and the robustness of the analyses supporting the estimate clearly indicate that noninferiority studies are acceptable for antibacterial agents for the treatment of HABP and/or VABP.
As mentioned, establishment of superior efficacy of a new antibacterial agent is made difficult by study exclusion of patients infected by organisms resistant to the study comparator drug(s). Furthermore, placebo-controlled superiority trials of HABP and/or VABP cannot be conducted because of the high mortality associated with the disease and the availability of effective antibacterial therapy for most cases. However, there are specific circumstances for the treatment of HABP and/or VABP in which superiority of a new agent should be feasible to achieve and in which superiority trials may be preferred to a noninferiority design.
The marked increase in the incidence of HABP and/or VABP caused by XDR or truly pan–drug-resistant gram-negative bacilli has created a situation in which superiority, compared with relatively ineffective standard therapy, can be tested ethically and appropriately in a clinical trial. When HABP and/or VABP is caused by organisms resistant to virtually all other agents, the standard of care is to treat the infection with the antibacterial agents to which the pathogen remains susceptible (eg, colistin), because no other therapy is available for such infections. Because of the lack of alternative therapy and the low efficacy of current standard of care in this context [52–58], a superiority trial testing the efficacy of a promising experimental antibacterial agent, compared with standard therapy, for HABP and/or VABP caused by XDR gram-negative bacilli would be ethical, appropriate, feasible to have approved by institutional review boards, and desirable to advance the science and clinical therapy of these infections.
Superiority testing of antibacterial agents for the treatment of HABP and/or VABP would also be desirable in the context of adjunctive therapy to improve outcomes of infection. Such a study would compare standard-of-care therapy plus the novel adjunctive therapy with standard-of-care therapy plus placebo. The comparator arm should include placebo to enable blinding of the study. The addition of a new antibacterial agent to an existing regimen to improve outcome of infection can only be tested in a superiority study, because achievement of noninferiority in that context would not constitute evidence of efficacy of the new agent. Some examples of new agents that would be appropriate for testing in the context of adjunctive therapy plus available adjunctive therapy are (1) inhalational agents targeting MDR or XDR organisms, (2) new systemic agents with spectra of activity focusing on certain MDR or XDR organisms, or (3) immunomodulatory adjunctive therapy.
Precise design features of superiority studies in this context were not discussed extensively at the workshop. Important issues to consider in designing such studies are (1) whether patients should be enrolled during the empirical therapy stage, with narrowing of the evaluable population after microbiological confirmation, or after microbiological confirmation of the MDR and/or XDR organism causing the infection; (2) the acceptability of a standard-of-care control regimen to the FDA and other regulators, because of the innate variability that can be found in such approaches and the need to have a well-justified rationale for selection of the comparator regimen; and (3) the complexities of blinding for such a superiority study. Because of these questions, the societies recommend that a follow-up workshop be convened to discuss the design and conduct of registration studies of agents active against MDR and/or XDR pathogens.
Because of the lack of efficacy of most antibacterial agents for HABP and/or VABP caused by XDR or pan–drug-resistant organisms, consideration should be given to the potential use of historical controls in a clinical trial of a new agent with activity against such organisms. The possibility of historically controlled superiority studies in this context was not discussed on the record at the workshop but has been the focus of subsequent dialogue related to the workshop and is explicitly mentioned as a possibility in a relevant ICH guidance .
Specifically, the ICH E10 guidance indicates that, “in unusual cases…it may be possible to use a similar group of patients previously studied as a historical control” for clinical trials [59, p 7]. The guidance emphasizes that, if a historical control group is to be used for a clinical trial, the control subjects should be selected from a “well-documented population of patients …on the basis of particular characteristics that make them similar to the treatment group” [60, p 30]. The guidance continues, “The inability to control bias restricts use of the [historical] control design to situations in which the effect of treatment is dramatic and the usual course of the disease highly predictable. In addition, use of [historical] controls should be limited to cases in which the end points are objective and the impact of baseline and treatment variables on the end point is well-characterized” [60, p 30].
In accordance with the ICH E10 guidance, the reliable, high mortality rate associated with untreated or ineffectively treated HABP and/or VABP enables potential use of historical controls for a clinical trial of an experimental antibacterial agent against HABP and/or VABP caused by XDR or pan–drug-resistant gram-negative bacilli. The ICH E10 guidance specifies criteria to be incorporated in the historical controls to elevate the rigor of the study to the level necessary for registration clinical trials. The guidance specifies that “[historically] controlled trials are most likely to be persuasive when the study end point is objective, when the outcome on treatment is markedly different from that of the external control and a high level of statistical significance for the treatment-control comparison is attained, when the covariates influencing outcome of the disease are well characterized, and when the control closely resembles the study group in all known relevant baseline, treatment (other than study drug), and observational variables” [61, p 32].
Prospective establishment of a robust and well-characterized observational cohort of patients with HABP and/or VABP caused by XDR or pan–drug-resistant gram-negative bacilli could fulfill the rigorous criteria specified in the ICH E10 guidance on historically controlled studies. For example, such a database could be constructed by enrolling the prospective observational cohort that will serve as the historical control proximate to the planned initiation of the experimental arm of the study, such that the patients ultimately enrolled in the experimental arm are demonstrably similar to those in the observational cohort serving as the historical control subjects. Furthermore, pre-specified analysis of baseline patient characteristics and covariates that predict mortality could be planned between the historical control subjects and the experimental arm to validate the similarity of the populations. The experimental arm most likely would consist of open-label administration of the experimental drug to the second cohort. The prespecified primary efficacy outcome of the study would be all-cause mortality as the most objective measure possible, with the experimental arm tested for superiority against the historical control subjects.
The societies emphasize that active dialogue (eg, by means of a follow-up workshop) regarding clinical trial designs for the study of infections caused by organisms for which there is limited (or no) effective antibacterial therapy would be greatly beneficial. The possibility of historically controlled studies in this context should be a focus of discussion.
As discussed above, the data supporting a substantial treatment effect size of initial effective, compared with ineffective, antibacterial therapy for HABP and/or VABP are based entirely on estimates of all-cause mortality. On the basis of the precedent established for CABP , a decrease in survival benefit of >10% with effective antibacterial therapy for the treatment of HABP and/or VABP is clinically unacceptable. Because of the substantial treatment effect of active antibacterial therapy (ie, absolute reduction in mortality of ≥30%), a 10% absolute margin of noninferiority can be justified and is appropriate for all-cause mortality as a primary efficacy end point in a noninferiority clinical trial of antibacterial therapy for HABP and/or VABP.
Multiple speakers at the workshop emphasized that adjudication of attributable mortality is problematic and frequently inaccurate for HABP and/or VABP, in the context of which underlying diseases and comorbidities are common. Therefore, the majority of workshop participants believed that all-cause mortality should be evaluated in lieu of attributable mortality. However, some workshop members believed that attributable mortality was a more clinically relevant end point.
The optimal timing in the course of a HABP and/or VABP registration trial at which mortality is evaluated was the subject of considerable discussion among the workshop panel members. The primary advantage of an earlier (eg, 14 days) analysis of mortality is the potential to eliminate from analysis late deaths related primarily to progression of underlying disease or to development of intercurrent events unrelated to the original pneumonia. In addition, the pathogenesis of HABP and/or VABP is primarily an aspiration event, and patients could continue to aspirate and, therefore, be at risk of early recurrence not because of failure of the initial course of therapy. However, the consensus of the workshop panel was that analysis of all-cause mortality at a later time (ie, 28–30 days) was more appropriate for trials of HABP and/or VABP for several reasons. First, recent registration trials that have formed the basis for the determination of the magnitude of antibiotic efficacy for the disease have shown a continual increase in mortality over the entire 30-day period after study enrollment. Second, modern critical care can artificially prolong the time to death; therefore, the time of death may vary by several weeks, based on decisions about the duration of supportive care before withdrawal of care from moribund patients. With an earlier analysis for all-cause mortality, there is a risk of obscuring true differences in mortality rates because of continued life support through the period of the earlier analysis despite eventual withdrawal of care. Finally, it was emphasized that HABP and/or VABP can result in initiation of complex physiological and inflammatory cascades (eg, systemic inflammatory response syndrome and acute respiratory distress syndrome) that continue to affect mortality among patients even after resolution of active infection. Therefore, changes in mortality occurring after 14 days may reflect a true modulatory effect of an experimental drug relative to control drugs on HABP- and/or VABP-induced physiological or inflammatory cascades. Nevertheless, some workshop participants favored a shorter, 14-day mortality end point, which could potentially eliminate confounding causes of death at later times.
With the assumption that a noninferiority trial design would use a mortality end point, vigorous debate at the workshop revolved around how to adjudicate the outcome of a patient who is experiencing therapy failure clinically and for whom salvage antibacterial agents were administered after randomization. Of note, this concern is distinct from that raised by use of adjunctive antibacterial therapy during study drug treatment that has overlapping activity with the study medication or that raised by concomitant therapy administered for a distant site infection (both discussed below).
Many of the panel members at the workshop believed that a patient given salvage antibacterial agents after randomization should be considered as having experienced clinical failure from the perspective of the primary efficacy end point. In contrast, others argued that adjudicating such a patient as experiencing clinical failure introduced subjectivity to the end point analysis and would run the risk of invalidating the statistical justification of the noninferiority margin, which is based on all-cause mortality data, without consideration of subjective determination of disease progression or clinical failure. The latter panel members argued instead that patients receiving salvage therapy should be adjudicated on the basis of all-cause mortality on a strict ITT basis, irrespective of the use of salvage therapy.
The major advantage of not adjudicating a patient receiving salvage antibacterial therapy who experiences clinical failure is the maintenance of a pure all-cause mortality primary efficacy end point. A strict all-cause mortality end point is totally objective, which somewhat mitigates the potential for a non–double-blinded study design to introduce unmeasured bias in end point adjudication. Therefore, if a double-blinded study design is problematic because of characteristics of the study or comparator drugs (eg, different dose administration schedules and colored intravenous solutions), a primary outcome measure of all-cause mortality based on initial randomization, irrespective of use of salvage antibacterial therapy, could be a useful mechanism to mitigate bias.
The problem of not adjudicating use of salvage antibacterial therapy as failure arises if such use is not balanced between the 2 study arms. An extreme example of this point was discussed at the workshop. If noninferiority were achieved for the primary efficacy end point of all-cause mortality but 90% of the salvage antibacterial agent use was in the experimental arm, it would be difficult to accept a conclusion that the experimental drug was not unacceptably worse than the comparator. During the workshop, representatives from the FDA agreed that such a study result would raise considerable concern during regulatory review.
The ICH E10 guidance document emphasizes that “the determination of the margin in a noninferiority trial is based on both statistical reasoning and clinical judgment” [25, p 15]. In this context, adjudication of salvage antibacterial therapy as equivalent to death for analysis creates problems with statistical justification of the noninferiority margin for the study. On the other hand, use of salvage antibacterial therapy is an indicator of clinical failure of the therapy to which that patient was assigned. Clinically, it would not be acceptable to use a drug that was clearly inferior in efficacy, simply because effective salvage therapy was available for the patient after progression during receipt of the previous therapy. The fact that the decision to add salvage antibacterial therapy is not strictly objective creates concerns about statistical bias in end point analysis, but it is consistent with standard clinical practice. Therefore, not adjudicating the use of salvage therapy as a failure runs the risk of making the results of the clinical trial irrelevant to clinical practice.
Reconciliation of these competing statistical and clinical concerns is problematic to achieve. Indeed, more so than any other issue discussed at the workshop, the decision regarding how to adjudicate patients who receive salvage antibacterial therapy after randomization cannot be made clearly on the basis of ICH guidance, because either position can be justified by either statistical or clinical reasoning. In light of the equipoise on this issue, it is prudent to consider both options as acceptable if certain measures are taken to protect the integrity of the study and its interpretation.
On the basis of the aforementioned considerations, the societies agreed to an acceptable compromise on this issue. Option 1 is adjudication on a strict ITT basis of all-cause mortality, without consideration of postrandomization salvage therapy to indicate failure. This method is statistically advantageous, but runs the risk of making the trial results less relevant to standard clinical practice. This approach is clearly preferred for studies that cannot be double-blinded. If this strategy is used, the statistical analysis plan should account for the impact of institution of salvage therapy by other analyses (eg, by prospectively planned comparison of use in both arms).
The second option is adjudication of failure on the basis of all-cause mortality or the postrandomization addition of salvage therapy. This method may be statistically less desirable, but it is more clinically relevant than the first option. If this strategy is used, both the patient and the observer (ie, the assessor who determines that salvage therapy is necessary) must be blinded. If feasible by study design (see discussion on blinding below), blinding of other study personnel and clinical teams should be strongly considered. Furthermore, irrespective of study blinding, prospectively defined objective criteria should be included in the protocol that indicate the factors that should trigger use of salvage antibacterial therapy. With use of either option, the protocol should specify the reason that such nonstudy therapy was used, so that a prospective analysis of the factors driving the nonstudy therapy in both arms can be conducted.
Because of the severity of illness and frequent comorbidities in patients with HABP and/or VABP, use of other antibacterial and nonantibacterial therapies is frequently required for appropriate clinical management. Standardization of nonantibacterial therapies is an important feature of study design, albeit challenging because of differences in standards of care nationally and internationally. Nonetheless, standard-of-care therapy must be delivered in both the experimental and the control arms of a HABP and/or VABP study . Such therapy includes timely initiation of antibacterial therapy, deescalation of therapy on the basis of microbiology, proper dosages and duration of antibacterial therapy, and proper mechanical ventilation management for patients with VABP.
Adjunctive antibacterial therapy also presents challenges [62, 63]. In many patients, effective therapy of HABP and/or VABP requires >1 agent to achieve the necessary spectrum of activity (discussed further below). Another difficulty arises when adjunctive therapy is required for a distant site infection, such as a urinary tract infection, as opposed to the primary indication of HABP and/or VABP. Because of the frequency with which intercurrent infections unrelated to pneumonia occur in patients with HABP and/or VABP, exclusion of all such patients from the primary analysis population is impractical. However, the spectrum of activity and duration of adjunctive antibacterial therapy for infections unrelated to pneumonia should be kept as narrow as possible. The frequency of such antibacterial use in each study arm should be assessed, and if a difference is observed, a sensitivity analysis should be performed to elucidate the impact of such therapy on the primary end point.
The societies strongly and unanimously believe that it is essential to incorporate clinical components in the primary efficacy end point to make HABP and/or VABP clinical trials relevant to clinical practice. Unfortunately, little historical evidence was available to serve as a basis for justifying a noninferiority margin for any end point other than all-cause mortality. Two data-sets available at the workshop that could enable an estimate of antibiotic efficacy for clinical end points focused on defervescence and resolution of hypoxemia. Specifically, Vidaur et al  published Kaplan-Meier curves of time to resolution of fever in patients with pseudomonal VABP treated initially with appropriate versus inappropriate antibacterial therapy. The effect size of defervescence in the context of initial appropriate versus inappropriate antibacterial therapy was substantial, both from a time-to-event perspective (ie, comparing areas under the curves) and by dichotomous analysis of defervescence at specified times. For example, on day 7, the proportion of febrile patients in the initially ineffective therapy group was 50% higher on an absolute basis than the proportion of febrile patients in the effective therapy group (~65% vs ~15%). This magnitude of benefit of effective antibacterial therapy on defervescence in the context of VABP is similar to that previously summarized for CABP .
The relevance and complexity of using defervescence as a marker for clinical response to therapy has been discussed previously in the context of end points for CABP . Furthermore, duration of fever has been shown to be important as a marker of resolution of VABP. For example, using data from a recent, large, randomized, controlled trial of patients with VABP, Shorr et al  reported that, by multivariable analysis, persistence of fever was the only factor associated with clinical failure in patients who survived infection. Therefore, defervescence is a relevant clinical end point for HABP and/or VABP.
The only other clinical end point identified at the workshop that described an antibacterial treatment effect size was improvement of the ratio of partial pressure of arterial oxygen to the fraction of inspired oxygen (PaO2:FIO2), a marker of patient oxygenation status. In a prospective study by Luna et al , resolution of VABP over time was analyzed. The authors found that patients receiving initial effective antibacterial therapy had faster rates of improvement in PaO2:FIO2 than did patients who received initial ineffective antibacterial therapy. This difference was found in both a Kaplan-Meier time-to-improvement comparison and by comparison of dichotomous outcomes at a de-fined time. Specifically, on day 3 after the diagnosis of VABP, the PaO2:FIO2 decreased by 26% in patients treated with initial ineffective antibacterial therapy and increased by 3% in patients treated with effective antibacterial therapy (point estimate of the difference, 29%). Furthermore, improvement in the PaO2:FIO2 has been shown to be an independent predictor of successful treatment of HABP and/or VABP , indicating the clinical relevance of the ratio as a marker for disease status. In its previous draft guidance for noninferiority trials of CABP, the FDA noted:
The treatment effect for an end point such as clinical failure would likely be larger than that seen with a mortality end point. It is reasonable to assume that some of the patients in present-day trials would progress to death in the absence of rescue therapy. If the definition of clinical failure (including death) were applied to a historically conducted study or clinical trial, the clinical failure end point would be at least as great as the observed mortality. Thus, the treatment effect based on mortality in historical studies or clinical trials can be extrapolated to a composite end point in a present-day trial that includes both mortality and clinical failure [67, pp 24–25].
The societies concur with this logic, both for studies of CABP and for studies of HABP and/or VABP. The societies emphasize that limiting trials to a mortality end point is not consistent with standard clinical practice. Physicians routinely assess response to antibacterial therapy by evaluating clinical biomarkers, such as resolution of fever, normalization of WBC count, improvement in oxygenation, and successful extubation of patients receiving mechanical ventilation. Failure to consider the impact of antibacterials on such end points decreases the clinical relevance of the study and creates a risk that results of registrational studies will not extrapolate well to postapproval use of approved agents. Therefore, the societies strongly endorse additional research to allow use of clinical primary end points in future noninferiority trials. Specifically, analysis of the impact of discordant antibacterial therapy should allow documentation of the magnitude of treatment effect on these clinical biomarkers. Such investigations should be a priority research focus. When such results become available, they should be incorporated rapidly into acceptable clinical trial designs for noninferiority trials of HABP and/or VABP.
Hierarchical end point testing was previously discussed during the CAP workshop [20, 68]. Hierarchical testing is particularly advantageous for trials of HABP and/or VABP, because it enables sequential assessment of both noninferiority and superiority primary end points in the same trial. Multiple primary end points are generally not appropriate for a clinical trial because of the concern of multiple comparisons testing. However, hierarchical testing obviates concern about multiple comparisons, because the end points are tested sequentially rather than concurrently. Specifically, end points are prospectively ranked such that the most important end point is tested first, and subsequent end points are tested only if significance is achieved with the preceding end point. Therefore, a trial could test for noninferiority in all-cause mortality for the primary efficacy end point, and if non-inferiority is achieved, it can proceed to test for superiority in clinical end points (such as clinical response or resolution of signs and symptoms of disease, the standard primary end point used in HABP and/or VABP trials until recently [23, 24]).
If hierarchical primary end point testing is used in a clinical trial, hierarchical order should reflect loss of available information at each step in the hierarchy . For example, in a trial assessing both all-cause mortality and clinical end points, mortality must be the first end point tested, because nonsurvivors are not available for assessment of clinical end points . If the initial mortality end point does not meet statistical significance, the trial fails the primary end point, and subsequent end points in the hierarchy cannot be considered as primary end points. In the latter scenario, subsequent end points should either not undergo statistical testing or, if testing does occur, results should be considered as secondary, hypothesis-generating data rather than confirmatory end points.
The FDA recently released a draft guidance on the conduct of CABP clinical trials . That guidance emphasizes the importance of establishing a microbial diagnosis in patients enrolled in noninferiority clinical trials of CABP. The need for a confirmed microbial diagnosis in patients enrolled in noninferiority clinical trials for HABP and/or VABP is even more important than that for CABP. Specifically, noninferiority trials carry a significant risk of a false-positive result (ie, failing to show a difference between 2 therapies, thereby establishing noninferiority) if substantial numbers of patients in either arm do not have the disease being studied. Nonbacterial causes of pulmonary infiltrates in hospitalized patients (eg, atelectasis, pulmonary contusions, noninfectious acute respiratory distress syndrome, viral pneumonia, pulmonary embolism, and alveolar hemorrhage) are common, are frequently indistinguishable from bacterial pneumonia, and will not respond to either the experimental or the comparator antibacterial agents used in a clinical trial. If substantial numbers of such patients were to be enrolled in a clinical trial of antibacterial agents for HABP and/or VABP, an equal lack of efficacy in both arms could result in falsely apparent noninferiority. Therefore, culture-confirmed bacterial infection is critical for the integrity of a noninferiority study of HABP and/or VABP. Furthermore, enriching enrollment for patients infected with a bacterial pathogen will likely enrich for more ill patients, which is necessary to ensure constancy to the treatment effects seen in previous HABP and/or VABP studies .
For the aforementioned reasons, most workshop panel members agreed that the primary efficacy analysis should be conducted in a microbiologically confirmed population, in accordance with the recently released CABP guidance. More specifically, a mMITT population should be used, with efficacy analysis restricted to patients who receive at least 1 dose of study drug (the MITT population). Some panel members believed that coprimary analysis populations should be evaluated, including both the ITT and the mMITT populations.
An additional concern in HABP and/or VABP noninferiority trials is potential enrollment of a patient infected with an organism resistant to all protocol-specified regimens. Inclusion of patients infected with organisms resistant to all therapies in the primary efficacy analysis potentially decreases assay sensitivity of a noninferiority study. Inclusion of patients for whom neither therapeutic arm is likely to be effective makes noninferiority to the comparator regimen easier to achieve, even though neither therapy is more effective than placebo in this context. Therefore, patients infected by such an organism should be considered to be nonevaluable for the mMITT primary efficacy end point (but not for the safety ITT population).
One complexity is the lack of availability of established susceptibility breakpoints for the investigational agent, particularly if that agent has not been approved previously for another indication. In this case, whether a cultured microorganism is susceptible to protocol-specified therapy may be determined on the basis of the previously approved protocol-specified agents that have established susceptibility breakpoints (whether adjunctive therapy in the investigational arm or in the comparator arm), rather than on susceptibility to the investigational agent.
A microbiologic end point is a logical primary efficacy end point for HABP and/or VABP studies, but a variety of factors limit the possibility of such an end point . Distinguishing persistent colonization from a persistent pathogen is often not possible when assessing postbaseline respiratory cultures . Imputing microbiological eradication (ie, inability to obtain a proper specimen for follow-up culture because the patient is improved and no longer producing sputum) provides no additional useful information, compared with the information that is already available in a clinical response assessment. Serial quantitative cultures have the potential to ameliorate some of these limitations. However, obtaining follow-up invasive cultures is not standard of care and may expose the patient to risk of a procedure without altering the clinical course of the infection. Furthermore, thresholds for quantitative culture positivity are not well defined and may vary by microorganism. Therefore, evidence of microbiological eradication is not appropriate as a primary efficacy end point for a HABP and/or VABP study.
In selecting clinical enrollment criteria to be used in a HABP and/or VABP study, the goal is to increase the pretest probability of eventual culture-confirmed pneumonia. Combinations of appropriate clinical and radiographic criteria can be used to select patients more likely to be evaluable in the mMITT population. Clinical criteria relevant to the diagnosis of HABP and/or VABP are hospitalization for ≥48 h (or ventilation for ≥48 h for VABP); a new, progressive, or persistent pulmonary in-filtrate on chest radiograph (read as consistent with or likely indicative of pneumonia by a radiologist); and at least 2 of the following signs: (1) temperature <36°C or ≥38.3°C, (2) WBC count <5000 cells/μL or >10,000 cells/μL; or (3) purulent sputum or endotracheal aspirate [41, 42]. These clinical and radiographic criteria are sensitive but not specific for establishing the diagnosis of HABP or VABP [42, 71, 72]. Nevertheless, these criteria are useful because the combination of clinical and radiographic criteria increase the pretest probability of disease [73, 74], thereby improving the positive predictive power of subsequent, confirmatory microbiology cultures for diagnosis of HABP or VABP. Therefore, the aforementioned clinical and radiographic criteria are appropriate inclusion criteria for HABP and/or VABP studies.
The Clinical Pulmonary Infection Score (CPIS) as a diagnostic tool for HABP and/or VABP was discussed extensively at the workshop. The CPIS is calculated from clinical and radiographic criteria very similar to the aforementioned enrollment criteria (ie, temperature, WBC count, radiographic findings, and tracheal secretions) but also includes estimates of hypoxemia (PaO2:FIO2) and respiratory culture results . Whereas the CPIS is somewhat more objective than the 3 individual clinical criteria, subjectivity remains inherent in the calculation of the CPIS, especially with regard to radiographic interpretation and quantification of tracheal secretions.
A CPIS ≥6 has been proposed to support the diagnosis of HABP or VABP . However, data supporting the accuracy of the CPIS alone to establish a HABP or VABP diagnosis are mixed, and similar to the clinical criteria, the CPIS is most accurate for diagnosis when combined with microbiologic confirmation of infection [41, 42, 76]. Twenty-two percent of patients with a CPIS <6 on day 1 can have their CPIS increase to ≥6 by day 3, usually with the addition of microbiologic culture results . Therefore, requiring the CPIS to be ≥6 at enrollment may exclude up to one-quarter of patients who would be evaluable for the primary efficacy end point. Of note, the CPIS performs particularly poorly for patients with trauma and/or burns [78, 79], who comprise an increasingly important population of patients with HABP and/or VABP, because the incidence of these infections remains high in these contexts.
Because of the similarity of the information on which standard clinical and radiographic criteria and the CPIS are based, use of either clinical and radiographic criteria or the CPIS for enrollment criteria is reasonable. In either case, the purpose of these criteria is to increase the pretest probability of HABP and/or VABP; they must be used in combination with microbiologic confirmation to determine which patients are evaluable for the primary efficacy end point.
To ensure constancy with the historical studies used to justify the noninferiority margin for the primary efficacy end point, enrichment of the enrolled population for patients with relatively severe disease is necessary. The overall target all-cause mortality rate in the control arm should be 15%–20%. Therefore, calculation of a severity-of-illness scoring system is necessary as part of the study enrollment criteria to enrich for sufficiently ill patients.
Factors that define severe HAP and/or VAP have been characterized . Such risk factors include admission to the ICU, respiratory failure (ie, the need for mechanical ventilation or need for >35% oxygen to maintain oxygen saturation >90%), multilobar pneumonia or cavitation, or evidence of severe sepsis or septic shock. Factors associated with an increased risk of mortality include prolonged mechanical ventilation before pneumonia, serious comorbidities, high Acute Physiology and Chronic Health Evaluation (APACHE) II score (ie, ≥11 points), severe pneumonia, age >60 years, a high-risk pathogen, and delayed initiation of appropriate therapy . Inclusion of these factors in enrollment criteria alone or as part of a disease severity scoring system (discussed below) would enable the study to achieve the target all-cause mortality of 15%–20% in the control arm.
Numerous disease severity scoring systems for HABP and/or VABP were discussed at the workshop, including the Simplified Acute Physiology Scoring, APACHE (II or III), the Therapeutic Intervention Scoring System, the Mortality Prediction Model, the Sequential Organ Failure Score, the Multiple Organ Dysfunction Score, and the Predisposition Insult Response Organ dysfunction system . No clear consensus emerged from the workshop panel on the optimal choice for a severity-of-illness scoring system for a clinical trial of HABP and/or VABP. It was also noted that fewer pediatric disease severity scoring systems have been investigated or validated in neonates, infants, and children. The overwhelming consensus of the panel was that a disease severity scoring system should be used as an enrollment criterion for HABP and/or VABP studies. The choice of scoring system and the cutoff (both high and low) that should be used for the enrollment criterion should be determined by the study sponsor in consultation with the FDA and other regulatory agencies.
Gram staining of a deep respiratory specimen may be useful at baseline for inclusion or exclusion of certain patients from enrollment, thereby enriching the mMITT population for the primary efficacy end point. For example, in a recent prospective study, Gram staining of bronchoscopically obtained specimens had a 90% sensitivity and 96% negative predictive value for VABP . Similarly, in another study, results of Gram stain of either bronchoscopically or nonbronchoscopically obtained respiratory tract samples improved the diagnostic accuracy of the CPIS for VABP . Incorporation of Gram stain results into the CPIS enabled early detection of 85% of patients subsequently confirmed to have VABP and enabled exclusion of 70% of those who did not have confirmed VABP. Therefore, a negative result of Gram stain of a sample obtained by bronchoscopy would be a useful tool to exclude patients from enrollment to enrich the mMITT population. In a systematic review, Klompas  reported that the positive likelihood ratio of Gram stain of a sample obtained by bronchosopy (but not by less invasive means) was high for VABP. Therefore, a positive result of Gram stain of a bronchoscopically obtained specimen could be useful in enriching patients for those likely to have VABP.
Gram stain results may also be important to include or exclude patients infected with organisms likely to be susceptible or resistant to the experimental therapy. For example, in a study of an investigational agent with a purely gram-negative spectrum, the observation of only gram-positive cocci on an adequately prepared and interpreted Gram stain of a deep respiratory specimen could be a useful exclusion criterion. Alternatively, in a study of an investigational agent with a purely gram-positive spectrum, the finding of gram-positive cocci on the Gram stain can be used to enrich the trial for patients who are likely to have HABP and/or VABP caused by methicillin-resistant Staphylococcus aureus (MRSA).
Gram staining of deep lower respiratory tract specimens is also useful, because it provides information about leukocytes. The finding of <50% neutrophils by cell count analysis in a lavage specimen (either bronchoscopic or nonbronchoscopic) has a negative likelihood ratio of 0.05:0.1 for the diagnosis of VABP . Therefore, the presence of <50% neutrophils in a lower respiratory tract specimen could be used as an exclusion criterion for enrollment (assuming that the information becomes available shortly after the specimen is obtained), thereby enriching for patients who meet the mMITT criteria.
At many health care centers, Gram stain results are available within a short period and could be used as part of enrollment criteria to enrich the mMITT population. However, at many other health care centers (particularly, international centers), Gram stain results do not become available until the subsequent day, precluding their use as an enrollment criterion. Ultimately, the consensus of the workshop panel was that the decision regarding requirement for deep respiratory specimen Gram stain as an enrollment criterion should be made by the study sponsors, who can weigh the risks and benefits of its use for specific studies.
Finally, the potential for use of procalcitonin level as a diagnostic and/or enrollment criterion for HABP and/or VABP studies was discussed at the workshop [41, 70]. The relatively high negative predictive value of low procalcitonin level could make it useful for exclusion of bacterial infection. Therefore, use of a low baseline procalcitonin level to exclude patients who are unlikely to have a positive lower respiratory tract bacterial culture result may be reasonable, again enriching for patients more likely to be evaluable in the mMITT population.
There was general agreement that the primary efficacy end point should be analyzed in the mMITT population. Therefore, all evaluable patients must have a positive bacterial culture result. Nevertheless, microbiologic results are typically not available at the time of patient enrollment, and use of culture results as an enrollment criterion is, therefore, not practical. Instead, results of culture of specimens obtained at enrollment determine which patients to include in the mMITT population for the primary efficacy end point. Patients found to have a negative culture result should be considered to be nonevaluable for the primary efficacy end point (although they should be included in the ITT safety population). Experts at the workshop emphasized that dropping patients from the evaluable population after randomization is statistically acceptable in this context, because the microbiologic study on which the decision is based is not a postrandomization event (ie, the culture is performed at baseline, before initiation of any study treatment).
The definition of a positive culture result enabling inclusion of the patient in the mMITT population should be considered carefully in the study protocol to exclude cultures positive for nonpathogenic organisms. For example, specifying that a positive culture result requires moderate-to-heavy growth, by semi-quantitative or quantitative culture methods, of ≥1 organism known to be causative of HABP and/or VABP (eg, gram-negative bacilli, S. aureus, Hemophilus influenzae, Streptococcus pneumoniae, and Streptococcus milleri) may be reasonable and was done in a recent multicentered, randomized trial of VABP .
One of the most contentious foci of discussion at the workshop was the proper technique for confirmation of the microbiologic etiology of VABP. Aside from debate about the degree to which bronchoscopically obtained culture specimens are superior in specificity to deep endotracheal aspirate specimens [74, 83–86], concern was expressed regarding the feasibility of obtaining quantitative, bronchoscopic specimens for culture from all patients in multinational clinical trials enrolling participants at dozens, if not hundreds, of sites worldwide [62, 63]. Ethical considerations also exist for routine invasive techniques for sample obtainment, such as bronchoscopy, in pursuit of a pathogen, especially for pediatric patients enrolled in HABP and/or VABP studies. This discussion reflected the lack of consensus of English-language national treatment guidelines on nosocomial pneumonia, regarding the need for bronchoscopic cultures, compared with noninvasive culture strategies, to diagnose nosocomial pneumonia in clinical practice . The guidelines achieve consensus that a lower respiratory tract culture must be performed to support the diagnosis. However, the method by which such a culture specimen should be obtained differs among the various guidelines. Nevertheless, on the basis of published data indicating superior diagnostic accuracy, numerous panel members strongly preferred that quantitative cultures be used, regardless of whether the samples are obtained bronchoscopically, by a standardized method of mini-bronchoalveolar lavage , blind nonbronchoscopic obtainment of samples from distal airways [88, 89], or deep endotracheal aspiration . Furthermore, some panel members believed strongly that bronchoscopically obtained quantitative culture specimens were preferred to those obtained by other methods.
In summary, the greater accuracy of quantitative culture of bronchoscopically obtained samples for the diagnosis of VABP must be weighed against the degree of invasiveness and feasibility because of the limited availability of quantitative cultures for HABP and/or VABP studies conducted at numerous sites internationally. Many panel members, but not all, concluded that carefully obtained deep endotracheal aspirate specimens may reflect a reasonable compromise between diagnostic accuracy and study feasibility.
The method for obtaining deep respiratory culture specimens in the context of VABP should be prospectively defined, and such specimens should be obtained by trained, experienced personnel. For example, a deep endotracheal aspirate requires that the suction catheter be advanced until resistance is met; only then should the specimen be taken. This method is not the usual technique for clearing secretions from proximal airways. Consideration may also be given to use of an external sterile suction catheter and suction trap rather than use of the in-line suction catheter, as was done in a multicenter, randomized comparison of quantitative bronchoscopically obtained culture samples and nonquantitative endotracheal aspirate samples .
If a patient with HABP undergoes bronchoscopy for clinical purposes, positive culture results for samples obtained through bronchoscopy would be appropriate for evaluation in the mMITT population of a clinical trial. However, most patients with HABP do not undergo bronchoscopy, and many workshop panel members believed that an invasive procedure that was not otherwise clinically indicated could not be mandated specifically for the purpose of obtaining specimens adequate for inclusion in the m-MITT population of a clinical trial. These panel members believed that, for patients with HABP who cannot undergo bronchoscopy, positive culture results for semiquantitatively expectorated sputum samples are an alternative basis for inclusion in the mMITT primary efficacy population. Such sputum cultures should meet prespecified cytologic criteria (eg, ≥ 25 polymorphonuclear leukocytes ± < 10 squamous epithelial cells per high-power field [90–92]). For pediatric patients with HABP, obtainment of appropriate expectorated samples is not realistic, further complicating the accurate identification of pediatric patients with HABP who are microbiologically evaluable.
There was considerable controversy over the proper methods used to obtain a deep respiratory culture samples for patients enrolled in HABP and/or VABP studies. For patients with either HABP or VABP, major emphasis should be placed on obtaining high-quality, deep respiratory samples for culture, irrespective of the method of obtainment. Prespecified protocols and criteria should be included in the clinical protocol to ensure the adequacy of the specimens, and the samples should be obtained by experienced, trained personnel.
Other means to obtain microbiologic confirmation of infection include positive pleural fluid culture results, positive blood culture results in the context of clinical and radiographic evidence of HABP and/or VABP, and urinary antigen testing. The societies underscore the need for advances in molecular diagnostic testing for establishing the microbiologic etiology of HABP and VABP. These advanced molecular diagnostic techniques could be used to establish the microbial etiology of HABP and VABP in clinical trials when such technologies become available and are validated.
The acceptability of enrolling patients with either HABP or VABP in the same clinical trial was discussed at the workshop. Four predominant factors were central to consensus on this issue.
The first concern regarding enrollment of patients with HABP or VABP in the same clinical trial was the difference between patient drug exposure during HABP and that during VABP [63, 93]. An important subset of patients with VABP exhibit higher drug clearance and, therefore, lower antibacterial drug exposure, than do the majority of patients with HABP; both renal and hepatic clearance can be higher than expected, resulting in a bimodal distribution of exposure [63, 93].
The second factor affecting the appropriateness of combining patients with HABP and VABP in a single study is the microbiological etiology of the diseases. Although some differences in microbiology (eg, less S. aureus and MRSA, in particular, in patients with VABP) exist, in general, the microbial etiologies of the 2 types of infection have been similar in recent series [16, 41, 94–96]. Specifically, nonfermenting gram-negative bacilli, including MDR gram-negative bacilli, such as Pseudomonas and Acinetobacter species, cause a substantial proportion of both HABP and VABP. Key factors predicting whether MDR pathogens are the cause of infection include duration of hospitalization before the onset of infection (ie, <5 days imparts low risk and ≥5 days imparts higher risk), exposure to antibiotics during the preceding 90 days, or exposure to environments rich in MDR pathogens (eg, prior hospitalization, residence in nursing home, or receipt of dialysis or home infusion therapy). These factors predict MDR organisms equally for HABP and VABP. Therefore, a key factor determining the necessary antibacterial spectrum of both the experimental drug and the comparator regimen is not whether patients with both HABP and VABP are included, but whether there is presence or absence of individual patient risk factors for MDR organisms, such as the aforementioned factors and those mentioned in the ATS and IDSA guidelines on treatment of nosocomial pneumonia .
Third, the need to establish a microbiologic diagnosis for evaluable patients in the mMITT population may be problematic for a combined HABP and/or VABP study. Deep respiratory tract culture samples are readily obtainable from patients with VABP. However, an adequate deep expectorated sputum culture sample may be difficult to obtain from most patients with HABP. Excluded patients in a combined study are likely to be disproportionately patients with HABP. Practically, the time commitment and cost of an excluded patient may drive many investigators and sponsors to emphasize VABP enrollment.
Finally, the difference in mean severity of illness between patients with HABP and patients with VABP is an important consideration regarding whether studies should enroll both patient subsets. On average, patients with VABP are more severely ill and have higher predicted mortality rates, compared with patients with HABP . Nevertheless, some patients with VABP (eg, young individuals without comorbidities who suffer trauma) may have a lower mortality rate than may certain subsets of patients with HABP (eg, those treated in an ICU). Furthermore, patients with HABP treated in the ICU have severe disease with substantial risk of death, more akin to the typical mortality rate associated with VABP. Thus, enrollment of patients with HABP or VABP in a single study would require mechanisms to monitor the appropriate severity of illness and balance of severity of illness in the 2 randomization arms.
Considering the aforementioned factors, the potential for substantive differences between patients with HABP and patients with VABP exists. Therefore, enrollment of patients with HABP or VABP in the same study would only be feasible if these factors were accounted for in the study protocol. Specifically, 3 factors must guide the choice of enrollment of patients with HABP and/or VABP in a clinical trial. First, robust enabling data must be available to support the design of the study protocol for the definitive study. Specifically, data must be available to enable rational selection of a dose that provides adequate therapy, taking into consideration both drug exposure and susceptibility of likely organisms. For a study seeking to enroll patients with both HABP and VABP, the enabling data must provide a basis for a dosing rationale for both patient populations for the study drug. Second, patients must have microbiologic confirmation of disease for inclusion in the mMITT primary efficacy population. Finally, the severity of illness needs to be substantial for the total enrolled population, to provide constancy for the mortality rates in the historical studies used to justify the margin for a noninferiority study. Use of a severity-of-illness scoring system as an enrollment criterion (discussed further below) and potentially restricting or enriching enrollment for patients in the ICU could enable patients with HABP or VABP of similar disease severity to be enrolled in the same study.
In summary, noninferiority studies of nosocomial pneumonia could focus on HABP and/or VABP. In practice, patients with VABP will be easier to enroll in clinical trials, because positive, deep respiratory tract culture samples are easier to obtain from patients with VABP than they are from patients with HABP, and patients with VABP are more severely ill, on average, than are patients with HABP. However, advances in molecular diagnostics may make enrollment of patients with HABP more facile in the coming years. Combination HABP and VABP studies would be more complex to justify, because of the need for enabling data to support dose selection for patients with both HABP and VABP.
Finally, clear consensus existed at the workshop that patients with ventilator-associated tracheobronchitis, in the absence of radiographically confirmed pneumonia [41, 97], should not be enrolled in studies of antibacterial therapy for HABP and/or VABP. Clinical trials of tracheobronchitis for the purpose of establishing an indication for the treatment of this disease could be considered in the future, as understanding of the pathophysiology and clinical features of this disease become better understood.
The panel members at the workshop emphasized the need to use adequate and appropriate antibacterial therapy for all patients enrolled in studies of HABP and/or VABP [41, 42, 63, 98]. In general, individual antibacterial agents and specific combinations of agents, as well as dose and duration of therapy, that are recommended by the ATS and IDSA consensus guidelines on the treatment of HABP and/or VABP are appropriate for comparator drugs .
A major complicating factor is the variability of approved antibacterial drugs and especially their dosing regimens worldwide [62, 63]. Drug and dosing regimens should be standardized as much as possible in the protocol, despite variations in factors affecting pharmacokinetics (eg, weight and renal function). Ultimately, the selection of comparator regimens should take into consideration local microbiology surveillance data at participating study sites, such that local investigators are not forced by the study protocol to use inadequate antibacterial therapy for anticipated pathogens. To match appropriate therapy to likely MDR organisms, the protocol should specify different levels of intensity of comparator therapy and adjunctive therapy in the experimental arm on the basis of the presence or absence of the aforementioned risk factors for infection by an MDR organism [40–42].
Although comparator agents that have been previously approved for the specific indication under study have traditionally been used in noninferiority studies, the increasing prevalence of MDR and XDR pathogens makes the selection of an appropriate comparator for HABP and/or VABP studies increasingly difficult. For the treatment of infection with XDR pathogens that are resistant to all other options, it may be necessary to allow use of comparator treatments that do not have an approved indication for the treatment of HABP and/or VABP (eg, colistin and tigecycline). Furthermore, no comparator drug with activity against gram-negative bacilli has been approved for the treatment of HABP or VABP in pediatric populations; the only antibiotic approved for nosocomial pneumonia in children, linezolid, has no activity against gram-negative bacilli. Again, in trials of pediatric HABP and/or VABP, a protocol to specify unapproved comparator drugs may be necessary. In general, de-escalation of empirical combination therapy should be mandated by the protocol on the basis of microbiologic test results.
One of the most complex decisions in a noninferiority trial design for HABP and/or VABP pertains to which adjunctive therapy should be allowed per protocol in the experimental arm [41, 42, 62]. A guiding principle is that the safety of patients enrolled in clinical trials cannot be compromised. Furthermore, study enrollment and clinical relevance are affected negatively if protocol-determined regimens deviate from national treatment guidelines. Therefore, clinical trial design should be consistent with best practices. Because of the established increase in mortality when ineffective antibacterial therapy is initiated for the treatment of HABP and/or VABP, it is imperative that the initial empirical therapy in the experimental arm has activity against the infecting pathogens. Thus, failure to use combination comparator therapy for patients at risk of MDR pathogens is unacceptable. Adjunctive therapy also must be allowed per protocol for most experimental drugs for 2 primary reasons. First, the spectrum of most drugs does not include all the categories of pathogens relevant to HABP and/or VABP (ie, gram-positive cocci to include MRSA, gram-negative bacilli, MDR and XDR gram-negative bacilli, and anaerobes). Second, even if the experimental drug exhibits in vitro activity against each of the general categories of the likely organisms causing HABP and/or VABP, a very high probability of activity against individual microbial isolates (eg, ≥90% of likely isolates) must be shown, or a second agent should be added to increase the likelihood that initial therapy will be effective against likely isolates.
For experimental drugs with activity limited to gram-positive cocci, including MRSA, adjunctive therapy with an agent with activity limited to gram-negative bacilli is desirable. However, this approach is not always feasible or in the patient’s best interests. For example, because many anti–gram-negative agents have some anti–gram-positive activity, aztreonam has been the preferred adjunctive agent in a number of studies. Unfortunately, resistance to this compound has reached substantial levels in Pseudomonas aeruginosa, and aztreonam is typically not effective against Acinetobacter species, an increasing cause of MDR HABP and/or VABP. Therefore, emphasis on aztreonam for gram-negative coverage in studies focusing on MRSA pneumonia is inappropriate and dangerous. Combination adjunctive therapy focused on MDR and/or XDR gram-negative bacilli should be used as indicated by the ATS and IDSA guidelines .
For experimental drugs with exclusive anti–gram-negative activity, adjunctive therapy for gram-positive cocci is required . If the experimental drug is likely to treat virtually all (eg, >90%) strains circulating at the local site, including MDR and XDR strains, addition of a second gram-negative agent may not be necessary. If the experimental drug is not likely to treat virtually all strains of gram-negative bacilli, addition of a second gram-negative agent must be considered, based on ATS and IDSA criteria . Empirical therapy with 2 drugs active against certain gram-negative bacilli is the standard of care for specific patient populations . Thus, a newly approved antibacterial agent for HABP and/or VABP would be used empirically in conjunction with a second agent in patients at risk of MDR organisms, as was done during its registrational clinical trials. Therefore, addition at baseline of a second agent with activity against gram-negative bacilli does not necessarily affect the integrity of analysis of the efficacy of the experimental drug if use of combination gram-negative therapy was equally applied to the experimental and comparator arms of the randomized study and if adjunctive therapy was terminated promptly after microbiologic confirmation of susceptibility becomes available. Of note, addition of a second agent with activity against gram-negative bacilli is yet another reason why double-blinding of the study should be conducted, because open-label use of the experimental drug could lead to bias in selection of patients requiring a second gram-negative agent. Preplanned analysis of the frequency of addition of a second agent with gram-negative activity would provide reassurance that the protocol-specified criteria for a second agent were applied evenly to both arms. In all cases, adjunctive therapy should be eliminated or narrowed as much as possible immediately after availability of microbiologic confirmation of the etiological agent(s).
The most complicated scenarios arise for experimental drugs that have activity against both gram-negative bacilli and gram-positive cocci not including MRSA (eg, imipenemcilastatin) and for agents with a limited spectrum of activity against one or a few specific types of gram-negative bacilli that are common causes of HABP and/or VABP (eg, a drug or biological with exclusive activity for MDR and/or XDR Pseudomonas or Acinetobacter species, but not other organisms). The former situation is complicated because the adjunctive antibacterial therapy targeting MRSA is likely to have overlapping activity with the experimental drug against non-MRSA gram-positive organisms (eg, methicillin-susceptible S. aureus or streptococci). Patients determined to be infected with MRSA would be excluded from the mMITT population because of the absence of activity of the experimental drug against MRSA. However, patients determined to be infected with methicillin-susceptible S. aureus or streptococci would be included in the mMITT population. For double gram-negative bacilli coverage, addition of an adjunctive agent with activity against MRSA does not necessarily affect the integrity of analysis of the efficacy of the experimental drug against other gram-positive organisms if such adjunctive MRSA therapy was applied equally to the experimental and comparator arms and if adjunctive therapy was promptly terminated after microbiologic confirmation of susceptibility became available.
For agents with a limited spectrum of activity against one or a few specific types of gram-negative bacilli, the mMITT population should be limited to the organisms for which the therapy has activity to avoid confounding effects of additional adjunctive therapy. Such an agent might be more appropriately studied in a superiority study of adjunctive, combination therapy versus monotherapy for the targeted organism.
In all cases, adjunctive therapy should be eliminated or narrowed as much as possible immediately after availability of microbiologic confirmation of the etiologic agent(s). A prespecified analysis of the duration of adjunctive therapy in both study arms would provide reassurance about the comparability of narrowing of therapy in both study arms.
Final susceptibility interpretive criteria are not established for an investigational agent until after phase 3 data become available. Furthermore, susceptibility testing for an investigational agent may need to be conducted at a central laboratory, because clinical laboratories may not have the capacity to test susceptibilities for nonapproved drugs. Therefore, results of susceptibility testing for an investigational agent may not be available in real time during treatment of the patient and, even when available, may not be interpretable with respect to definitive breakpoints until after the end of the phase 3 study.
Even for commercially available adjunctive or comparator therapies, susceptibility testing results may not return for 48–72 h. Because of this delay, in blinded studies, an acceptable approach has been for the investigator to determine treatment discontinuation primarily on the basis of the patient’s response to therapy and not on the basis of susceptibility data. For example, in situations in which the isolated pathogen appears to be resistant to both of the treatment regimens, a salutary clinical and radiographic response would ethically allow continuation of blinded study therapy. By contrast, a patient infected by such an organism who experiences clinical failure should have study treatment discontinued (but they should not be withdrawn from the study), regardless of the susceptibility pattern.
In contrast to CABP, for which a published study suggested a treatment effect of even a single dose of antibacterial therapy before enrollment in a clinical trial , no such data are available on the impact of prior therapy for HABP and/or VABP. The microbiology of HABP and/or VABP is clearly distinct from that of CABP, with HABP and/or VABP typically caused by MRSA or gram-negative bacilli that are more refractory to eradication than are CABP pathogens. S. pneumoniae and H. influenzae infrequently (<5%) cause HABP, and when they do, it is usually in the context of early-onset (<5 days) disease . Underlying disease and comorbidities are, on average, more numerous and severe for the hospitalized population with HABP and/or VABP, tending to make microbial eradication more difficult than for CABP. Finally, VABP occurs in the setting of a foreign body (the artificial airway), making bacterial eradication far less likely after a single day of therapy. Therefore, the consensus of the workshop panel members was that a single day (not dose) of prior appropriate antibiotic therapy is unlikely to significantly affect cure rates for HABP and/or VABP. Patient enrollment before initiation of nonstudy antibiotic therapy, if possible, is recommended, but ≤24 h of prior therapeutic drug exposure should be allowed per protocol for studies of HABP and/or VABP.
Randomization should enable balance in important baseline characteristics between study arms. Nevertheless, stratification for factors known to affect the likelihood of treatment success provides an additional layer of security that the 2 study arms will be balanced for these key factors. Stratification during enrollment is recommended for risk factors for infection due to a MDR and/or XDR organism, as elaborated elsewhere and in the ATS and IDSA guidelines , and for factors increasing disease severity and/or mortality risk, as discussed above .
If patients with both HABP and VABP are enrolled in the same study, the primary stratification should be by disease type (HABP vs VABP). Most panel members also believed that patients should be stratified by a disease severity scoring system to ensure adequate balance between the arms of the study. The scoring system and cutoff values to be used for stratification should be chosen by the sponsor.
There was consensus among the workshop participants that studies of HABP and/or VABP should be double-blinded (patient and observer). Minimization of all forms of bias is crucial in a noninferiority trial, and blinding of the observer is necessary to minimize bias. Blinding of the clinical care team and any end point adjudicators is also desirable, if possible.
Nevertheless, complexities of study blinding are likely to be encountered in HABP and/or VABP studies. Some possible comparator or adjunctive antibacterial agents require monitoring of serum concentrations (eg, vancomycin and aminoglycosides), and many antibacterial agents require dose adjustment for renal dysfunction, which is common in patients enrolled in HABP and/or VABP studies. Adjustments of dose in these contexts require unblinded study personnel to evaluate results of drug concentrations and renal function. Such unblinded personnel should not participate in any other aspect of study conduct or end point assessment, aside from appropriate alteration of drug doses. Furthermore, drug concentrations should not be placed in the patient’s medical record to avoid unblinding the patient’s assigned study arm.
Other complications to study blinding are the use of multiple antibacterial agents with varied administration schedules in the control arm and as adjunctive therapy and the potential for antibacterial agents used as comparators to differ from those used as adjunctive therapy in the experimental arm. Double-dummy designs should be used, if possible, for dosing regimens that differ between the control and experimental arms, although the additional fluid volume required may limit feasibility. Colored infusion solutions also complicate blinding and may require colored tubing or opaque tubing sleeves to maintain blinding.
The societies support inclusion of pediatric patients in HABP and/or VABP research protocols, if possible, because of the need to define appropriate therapy for these patients. Complexities of pediatric studies of HABP and/or VABP are discussed further in this supplement , with an acknowledgment that invasive diagnostic techniques may not be widely used at study enrollment for neonates, infants, and children, requiring some degree of extrapolation of drug exposure and/or efficacy data from adult populations. Collection of adequate safety data for each pediatric age group, from extremely low birth weight premature infants to adolescents, remains an important goal for pediatric investigations. Inclusion of children earlier in the overall drug evaluation programs than currently exists for HABP and/or VABP registration trials is also important, because MDR pathogens exist in hospitalized pediatric populations. Postponing the start of a pediatric program until the conclusion of large phase 3 adult studies results in an unacceptable delay in providing essential information to clinicians on medically needed drugs for children .
Recent clinical trials of HABP and/or VABP have enrolled at sites in multiple countries on multiple continents [62, 63]. The complexities of conducting such studies and the resources required to enroll patients at such sites are considerable. Indeed, it was estimated at the workshop that recent studies of HABP and/or VABP cost $60,000–$80,000 per patient enrolled, resulting in phase 3 trial program costs of >$75 million per study [62, 63]. The consensus of the panel was that it was simply not feasible to conduct a HABP and/or VABP study strictly in the United States because of limited numbers of eligible patients and especially because of (1) highly restrictive and complex protocol entry requirements [62, 63], which limit potential patient and investigative site participation; (2) recent changes in reimbursement for patients with nosocomial infections, which could lead to underreporting of HABP and/or VABP cases ; and (3) the likely reluctance of severely ill patients and families to participate. Because of these factors, enrollment would be impossible to complete solely in the United States within a reasonable period. Therefore, it is necessary that studies of HABP and/or VABP be allowed to enroll patients internationally.
International enrollment adds complexity to study protocols for a variety of reasons [62, 63], including (1) variations in local microbiology that require prespecification of a sufficiently broad comparator antibacterial regimen to be effective at all sites ; (2) variations in standard of care and, thus, availability of microbiologic techniques and other laboratory data; and (3) variations in quality of data that can be gathered and abstracted from study sites. Such factors must be considered by the study sponsor when selecting study sites. Several of the workshop participants emphasized that the reported frequency of HABP and/or VABP is decreasing in medical ICUs and that emphasis should be placed on recruiting patients from trauma and/or surgical and burn ICUs to improve enrollment rates [80, 97].
Noninferiority trials are particularly susceptible to issues of study integrity [25, 26]. Collection of inadequate data, enrollment of incorrect patients, improper randomization, and myriad other potential issues in study conduct all increase the risk of incorrectly rejecting the null hypothesis and establishing noninferiority of an experimental drug that is actually less effective than the comparator regimen. Specifically for HABP and/or VABP, experienced study sites are highly desirable, as are sites with a high level of medical technology and training where preferred microbiologic techniques (eg, bronchoscopic and/or quantitative cultures) can be used, similar adjunctive management of critically ill patients can be reliably performed, and other crucial elements of study conduct can be assured. An established network of clinical trial sites would improve the quality of study data, enable timely enrollment of patients, and result in a significantly higher proportion of patients being enrolled in the United States, helping to ensure that the data from the study are relevant to the US population. The need for such a clinical trial network, based on similar concerns, has been discussed elsewhere . The societies reiterate the need for such a network to help support conduct of clinical trials for HABP and/or VABP, as for other diseases.
Although the major focus of the workshop was on the design of individual HABP and/or VABP clinical trials, the panel discussed the core components of a clinical trial program, because successful development of new drugs for patients with HABP or VAP requires that both the scientific and regulatory requirements and the regulatory indication are clearly defined for each trial .
An essential feature of such a program is a robust set of enabling data before initiation of phase 3 trials [63, 93]. Relevant enabling data include prior therapeutic experience with the class, preclinical data (eg, in vitro drug-susceptibility testing and activity in animal pneumonia models), and clinical data (eg, pharmacokinetic-pharmacodynamic modeling for target attainment in plasma and, when possible, in the lung, and possibly phase 2 data on HAP and/or VAP, especially for novel antibacterial classes). Although the state of the art allows prediction of efficacious dosing regimen(s), important physiological factors altering drug exposure and unexpected distributions of infecting pathogens or drug-susceptibility profiles can be problematic.
Because of the scientific and logistical issues associated with the study of HABP and/or VABP (as discussed at the workshop), data from one respiratory indication could be used to inform regulatory decisions about another. For example, because a CABP draft guidance has been issued by the FDA, one paradigm for registration could be the successful conduct of a noninferiority trial on both moderate-to-severe CABP and HABP or VABP to support an indication for pneumonia, including both community and nosocomial cases. As was discussed at the workshop, there is precedence at the FDA for granting a second indication to a drug on the basis of one successful clinical trial if that drug had previously been granted a related indication on the basis of the results of 2 successful clinical trials. Therefore, successful completion of 2 clinical trials of CABP and 1 clinical trial of HABP and/or VABP could lead to a general pneumonia indication. Furthermore, the enabling data (ie, preclinical in vitro and animal model data, clinical pharmacokinetic-pharmacodynamic data, and early-phase clinical data) are generally predictive of antibacterial efficacy in phase 3 clinical trials. Therefore, if a development program had strong enabling data, granting of an FDA indication for the treatment of pneumonia after successful completion of a trial for CABP and a trial for HABP and/or VABP would be reasonable.
The societies that cosponsored the HABP and/or VABP workshop advocate for patients and their health care providers. The positions presented here are not motivated by advocacy for industry. The convergence of lack of antibiotic development and increasing rates of antibiotic resistance in lethal bacterial pathogens , particularly organisms that cause HABP and VABP, has created a dangerous public health problem. As physicians and public health advocates, the workshop panel emphasizes that patients need new drugs for HABP and/or VABP. Furthermore, because a mean period of ≥10 years is required to complete development of a new drug, strengthening of the antimicrobial pipeline now is essential to meet anticipated needs in ≥1 decade. An important step to enhance the discovery and development of new antibiotics is clarification of FDA guidance for future clinical trials of antibacterial agents for HABP and/or VABP.
The current uncertainty in acceptable designs for clinical trials of HABP and/or VABP contributes to disincentives in the discovery and development of new drugs for these diseases. After a related workshop on CABP, the FDA released a guidance document that provided clear directions for conduct of trials of CABP. The societies desire similar approval and dissemination of clear and defensible guidelines for future clinical trials of new antibacterial agents for the treatment of HABP and/or VABP.
Drs Edward Cox (US Food and Drug Administration), Henry Masur (National Institutes of Health), John Bartlett (Infectious Disease Society of America), Richard Wunderink (American College of Chest Physicians), Michael Niederman (American Thoracic Society), and Philip Barie (Society of Critical Care Medicine).
Drs Helen Boucher, John Bradley, John Edwards, David Gilbert, Louis Rice, Michael Scheld, Brad Spellberg, and George Talbot.
We thank Dr John Powers and Dr Eric Brass for helpful discussions.
Potential conflicts of interest. B.S. has received consulting fees from Pfizer, Merck, Basilea, Arpida, Theravance, Advanced Life Sciences, and The Medicines Company; research support from Astellas, Gilead, Enzon, Novartis, Merck, and Pfizer; and speaker’s honoraria from Merck, Pfizer, and Astellas and owns shares in NovaDigm Therapeutics. Through Talbot Advisors, G.T. has provided consultative services to Actelion, Advanced Life Sciences, Avera, Bausch & Lomb, Calixa, Cempra, Cerexa, Cubist, Ipsat, Middlebrook, Nabriva, PTC, Rib-X, Shire, Targanta, Tetraphase, Theravance, ViroPharma, and Wyeth and owns shares in Calixa and Mpex.
Supplement sponsorship. This article was published as part of a supplement entitled “Workshop on Issues in the Design of Clinical Trials for Antibacterial Drugs for Hospital-Acquired Pneumonia and Ventilator-Associated Pneumonia,” sponsored by the US Food and Drug Administration, Infectious Diseases Society of America, American College of Chest Physicians, American Thoracic Society, and the Society of Critical Care Medicine, with financial support from the Pharmaceutical Research and Manufacturers of America, AstraZeneca Pharmaceuticals, and Forest Pharmaceuticals.