|Home | About | Journals | Submit | Contact Us | Français|
The purpose of this analysis was to evaluate a novel strategy for reporting adverse events in the Pediatric Heart Network’s randomized surgical trial of systemic–pulmonary artery shunt versus right ventricle–pulmonary artery conduit in infants with hypoplastic left heart syndrome. The strategy was developed to align the reporting process with the needs of a surgical trial while maintaining participant safety.
Adverse event reporting was analyzed for 2 groups of study subjects: those randomized to a trial arm during a period in which a standard adverse event reporting system was used (period 1) and those randomized after institution of a system that focused serious adverse event reporting on 6 sentinel events (period 2). The analysis encompassed the period from randomization (Norwood surgery) to hospital discharge from stage II surgery. Adverse event rates were compared using a Poisson regression model for the number of events per subject.
From period 1 to period 2, the rate of serious adverse events requiring expedited reporting decreased as expected (0.42 vs 0.14/subject/month of follow-up; P < .001). Subjects with a serious (sentinel) adverse event in period 2 had a significantly higher rate of death and cardiac transplantation.
The new adverse event reporting system successfully targeted subjects at highest risk, while decreasing the administrative burden associated with adverse event reports. This methodology may be of benefit in trials evaluating surgical or device-based interventions and in critically ill populations where many common clinical events would qualify as serious adverse events in the context of a drug trial.
Little has been published on adverse event (AE) reporting in pediatric research. There is even less experience with AE reporting in clinical trials in congenital heart disease research, and there is no regulatory guidance specific to conducting clinical research in critically ill children. Recent publications have standardized the definitions of a number of complications and AEs for patients undergoing treatment for congenital and pediatric heart diseases,1,2 but few, if any, publications examine strategies for reporting and grading the severity of these complications and AEs in pediatric cardiac surgical trials.
The Pediatric Heart Network (PHN) was established by the National Heart, Lung, and Blood Institute (NHLBI), National Institutes of Health (NIH), in 2001 to conduct multi-center studies in children with congenital and acquired heart disease.3 Early PHN protocols4,5 effectively used standard principles for reporting AEs that adhered to the criteria for defining serious AEs (Table 1). These general principles are established by the International Conference on Harmonisation, a body of clinical trial guidance derived from the collaborative efforts of the United States, European Union, and Japan and implemented by the federal Office of Human Research Protections and the Food and Drug Administration. 6,7 Coding dictionaries or classification systems are used to augment this guidance and provide a standardized approach to reporting AEs in clinical studies. The PHN used the Common Terminology Criteria for Adverse Events (CTCAE), version 3.0, developed by the National Cancer Institute, NIH. These principles and tools provided standard definitions of AEs and serious adverse events (SAEs), a system of severity scoring, and reporting guidelines, designed particularly for drug trials.
The limitations of the standard AE reporting approaches became apparent during the PHN’s Infant Single Ventricle Trial, which compared treatment with enalapril to placebo in critically ill infants with single ventricle physiology.8,9 There were not enough appropriate pediatric CTCAE codes to cover the complexity of this trial population, and the CTCAE severity scale was not developed for trials in critically ill infants. Discussions, training, and close monitoring for the duration of the trial were implemented to refine our approach to AE reporting.
Then in May 2005, the PHN launched the Single Ventricle Reconstruction (SVR) Trial, a randomized trial comparing a systemic–pulmonary artery shunt versus a right ventricle–pulmonary artery conduit in infants with hypo-plastic left heart syndrome undergoing initial surgical palliation. 10,11 The SVR trial began with a standard AE reporting framework modified on the basis of lessons learned in the Infant Single Ventricle Trial, but it soon became clear that an AE reporting system based on the standard strategy for drug studies was inappropriate for this surgical trial. The specific challenge was that many clinical events that are common in the postoperative period qualify as serious AEs in the context of a drug trial; thus the standard reporting paradigm resulted in excessive reporting while providing a disproportionately small amount of useful safety information. Moreover, the large number of reports generated considerable administrative workload for individual investigators, study coordinators, and institutional review boards (IRBs)/research ethics boards as well as the NIH, the data coordinating center, the medical monitor, and the Data and Safety Monitoring Board (DSMB).
To enhance the ability to recognize clinically meaningful events, PHN investigators turned to the sentinel event concept. A sentinel event, as defined by the Joint Commission on Accreditation of Healthcare Organizations, is an unexpected occurrence involving death or serious physical or psychological injury, or the risk thereof, that signals the need for immediate investigation and response.12 In a research context, sentinel perinatal events have been identified that predict patterns of brain injury.13
The purpose of this analysis was to compare two AE reporting methods used in the SVR trial to assess the accuracy of predicting the most severe adverse outcomes in the study population and the associated changes in reporting burden.
The design of the SVR trial has been described8 and the main results reported.9 Neonates with hypoplastic left heart syndrome and other single right ventricle malformations were enrolled at 8 main PHN clinical sites and 7 SVR auxiliary sites. Subjects were randomized to receive a Norwood procedure with either a modified right Blalock–Taussig shunt or right ventricle–pulmonary artery conduit to provide pulmonary blood flow. The primary end point was death or transplant at 12 months. The protocol for the SVR trial was approved by each center’s IRB, and written informed consent was obtained from a parent or guardian.
The SVR trial protocol underwent review by an independent DSMB appointed by NHLBI to oversee all PHN studies and was subsequently approved by the IRBs at all sites, including the PHN Data Coordinating Center (DCC). AEs were recorded and submitted by the site team via a secure, web-based electronic data capture system to the DCC, with immediate reporting of SAEs to the DCC, NHLBI, and the independent PHN medical monitor, a pediatric cardiologist with expertise in cardiac critical care. Sites reported AEs/SAEs to their IRBs as dictated by local requirements. The DSMB reviewed study data every 6 months, and a summary of this review was sent to the local IRB to ensure that all participating centers were informed of any pertinent safety findings.
SAEs were initially defined in the SVR trial as events that met the criteria in Table 1. In this phase of the SVR trial, May 2005 to January 2007, referred to here as period 1, each AE and SAE was recorded and submitted to the DCC. The events were categorized in standard fashion as to the degree of relatedness to the trial intervention. The site investigator categorized the response to the AE as surgical or catheter intervention, medical therapy, or none; the clinical outcome was characterized as resolved or stabilized.
A revised approach, developed by the SVR Adverse Events Subcommittee, was implemented from February 2007 to trial end in October 2009 (period 2). The SVR AE subcommittee included congenital heart surgeons, pediatric cardiologists and cardiac intensivists, study coordinators, the AE coordinator from the DCC, and NHLBI staff. The subcommittee reviewed the limited published literature and obtained information from pediatric investigators who had been involved in previous operative and perioperative surgical trials.14–16 An AE framework was established consistent with the concept of the sentinel event in patient safety literature.
Six “sentinel” SAEs, considered sufficiently serious for the SVR trial population, were identified (Table 2) and triggered expedited reporting. The frequency of these events was known from published retrospective outcome reports on neonates with hypoplastic left heart syndrome after Norwood palliation,17–19 so there was a precedent for determining the number of reported events. Following standard practice, in addition to these 6 events, any other event that the site principal investigator considered to be an SAE was also reported in expedited fashion. Death, 1 of the 6 sentinel SAEs, was also a primary study end point and required a separate reporting format to the DSMB to preserve the statistical integrity of the final trial data analysis.
During period 2, all events that were not sentinel SAEs were captured in the trial database as complications rather than AEs, using a standardized code list organized by organ system that was developed for this purpose (Table 3). Complications were recorded on an abbreviated case report form, which replaced the detailed AE reporting forms previously used. The SVR trial protocol was amended to reflect these changes, approved by the DSMB and NHLBI, and approved by the IRB at each site. Training sessions were held to familiarize study personnel with the new procedures, and the DCC AE coordinator worked individually with sites to ensure a smooth transition on the effective date of the change.
Data analysis was performed to evaluate the effectiveness of the new approach in achieving the goals of facilitating the detection of important safety end points and decreasing the volume of expedited reporting of events. Data were analyzed for 2 groups of study subjects: those randomized during period 1 (May 2005–January 2007) and those randomized during period 2 (February 2007–October 2009). The clinical period of interest for this analysis was from randomization (Norwood surgery) up to hospital discharge from stage II surgery. AEs occurring after the predefined follow-up window were not included. The data of subjects who had not been discharged from the stage II surgery within the period window were considered incomplete and also excluded from analysis. The number of SAEs, AEs, and complications was determined from the case report forms. SAE, AE, and complication incidence rates were compared, accounting for varying follow-up time in each AE reporting period, to normalize the raw numbers of events. A Poisson regression model for the number of events per subject was used to compare the SAE rates of the 2 time periods. In addition to comparing SAE rates, the new reporting system was validated by assessing the rates of death or transplant in the 2 time periods among those for whom an SAE was reported. Event rates in this report are based on the complete trial data set. NHLBI staff participated in the analysis and interpretation of the data for this article.
During periods 1 and 2, a total of 247 and 224 subjects were randomized and followed up as described earlier. Average follow-up time during period 1 was 3.9 ± 2.3 months (median, 4.3 months) compared with 4.7 ± 2.3 months (median, 4.8 months) during period 2; the difference results from normalizing the raw numbers of events. In period 1, 81% of subjects had an SAE reported, compared with 48% in period 2 (Table 4). From period 1 to period 2, the number of reported SAEs decreased by 64%, and the SAE incidence decreased by 67%. This represented a significant drop (by design) in the SAE rate (0.42 vs 0.14 events/subject/month of follow-up; P < .001). The incidence of complications in period 2 was 1.0 event/subject/month of follow-up.
The 1-year death/transplant rate associated with patients who had a sentinel SAE reported under the new system was 1.4 times higher (66% vs 46%; P = .001) than the rate associated with patients who had an SAE reported under the original system, which included a broader range of clinical AEs. The 1-year death/transplant rates were similar (5% vs 3%; P = .484) for patients who had only a nonserious AE (period 1 designation) and those who had only a complication (period 2 designation).
Because the number of events reported as SAEs decreased 64% from 406 in period 1 to 147 in period 2 (Table 4), the number of case report forms submitted to the DCC also decreased by two-thirds. The information requested in the SAE forms did not change from period 1 to period 2.
Standard AE reporting practices for drug trials were inadequate in this randomized cardiovascular surgical trial of fragile neonates. The standard perioperative course for these patients includes many clinical states that would represent SAEs in the CTCAE scoring system. This raised concerns about the ability to detect any single critical safety signal in the large stream of reported data and resulted in a significant reporting burden.
We found that changing to a sentinel SAE reporting strategy allowed us to identify those patients with the highest risk of death or transplant, as exemplified by the fact that rate of death and transplant among patients with a sentinel SAE was significantly higher than in patients with an SAE under the standard reporting system. This finding was an expected outcome of the change in reporting, because under the new system, we were distilling SAE reporting to represent only the most serious, life-threatening events.
When comparing patients who had only nonserious events (called AEs in period 1 and complications in period 2), we found that these 2 groups were homogeneous with regard to risk, based on their similar rates of death and transplant. Additionally, the events classified as SAEs in period 1, for which the classification changed to “complication” in period 2, were more similar to nonserious AEs than to SAEs, providing further verification that we had successfully focused the SAE reporting process on the highest-risk events.
The new system also allowed study coordinators and investigators to focus their time and efforts on reporting and responding to the most severe events. The 64% reduction in the volume of SAEs reported, with no change in the information requested on SAE reports, suggests that study coordinator time for reporting SAEs could have been reduced by as much as two-thirds, although this was not measured directly. Other perceived efficiencies were gained, including a decrease in reporting burden to local IRBs and decreased review time by IRBs, the medical monitor, and the DSMB.
Changing to a system of specified sentinel events also, by design, decreased the variability in what was reported as an SAE across sites. Site investigators, typically congenital heart surgeons or pediatric cardiac intensivists, were understandably accustomed to thinking in terms of a normal postoperative course, as opposed to what would be normal for a healthy nonhospitalized neonate. This led to some sites reporting events or procedures such as gastrostomy tube placement or diaphragm plication as SAEs, while other sites considered the same event or procedure to be part of typical postoperative care and thus did not report it. During period 1, a considerable amount of time was spent attempting to harmonize the types of events reported. This problem was solved with the sentinel SAE strategy used in period 2, because sites were reporting on a small number of predefined events.
There is a large body of literature devoted to patient safety and clinical error prevention designed to minimize morbidity and mortality and to evaluate quality of care. Ensuring the safety of subjects in clinical trials has received less attention. AE reporting is one of the main tools to assess safety, but the good clinical practice principles that govern the conduct of clinical research were developed primarily for drug trials. Furthermore, most clinical trials have been conducted in adults. It was not until approximately 1 decade ago that pediatric research began to be emphasized as a matter of policy by the NIH and by Congress with the passage of the Food and Drug Administration Modernization Act of 1997, the Best Pharmaceuticals for Children Act (2002),20 and the Pediatric Research Equity Act (2003).21 We believe that our approach may represent a useful paradigm for future surgical trials, as well as for trials of critically ill participants in general. This approach has already been adopted by another NHLBI-funded trial of therapeutic hypothermia in children after cardiac arrest22 (personal communication from Victoria Pemberton, December 28, 2008).
This was not a prospectively designed comparison, but rather reflects a retrospective evaluation of a practical change made in AE reporting procedures during the conduct of a clinical trial. Therefore, we do not know whether unmeasured factors could have influenced our results and thus the generalizability of the findings. Nonetheless, our results consistently support the benefits of the change. Furthermore, there is no evidence from these data or other trial monitoring data that there were any untoward effects of the reporting change.
In a randomized surgical trial in neonates with complex congenital heart disease, expedited reporting of AEs based on existing methods was found to be impractical. An overwhelming volume of data was generated, which not only increased the chance of error, but also risked obscuring true indicators of concern. Thus, a novel methodology was developed to effectively and efficiently capture appropriate events. This new technique relies on monitoring a small number of clearly defined sentinel SAEs and resulted in comparable identification of clinically relevant events while decreasing the administrative reporting burden. This methodology may be of benefit in trials evaluating surgical or device-based interventions and in critically ill populations where many common clinical events would qualify as SAEs in the context of a drug trial. Additional research on this methodology as well as on pediatric AE reporting in general is needed.
Supported by U01 grants from the National Heart, Lung, and Blood Institute (HL068269, HL068270, HL068279, HL068281, HL068285, HL068292, HL068290, HL068288, HL085057).
We thank Christine Kang, MDiv, for assisting in the preparation and submission of this manuscript.
Network Chair.University of Texas Southwestern Medical Center, Lynn Mahony
Data Coordinating Center. New England Research Institutes, Lynn Sleeper (PI), Sharon Tennstedt (PI), Steven Colan, Lisa Virzi, Patty Connell,* Victoria Muratov, Lisa Wruck,* Minmin Lu, Dianne Gallagher, Anne Devine,* Julie Schonbeck, Thomas Travison,* David F. Teitel
Core clinical site investigators.Children’s Hospital Boston, Jane W. Newburger (PI), Peter Laussen, Pedro del Nido, Roger Breitbart, Jami Levine, Ellen McGrath, Carolyn Dunbar-Masterson, John E. Mayer, Jr, Frank Pigula, Emile A. Bacha, Francis Fynn-Thompson; Children’s Hospital of New York, Wyman Lai (PI), Beth Printz,* Daphne Hsu,* William Hellenbrand, Ismee Williams, Ashwin Prakash,* Seema Mital,* Ralph Mosca,* Darlene Servedio,* Rozelle Corda, Rosalind Korsin, Mary Nash; Children’s Hospital of Philadelphia, Victoria L. Vetter (PI), Sarah Tabbutt,* J. William Gaynor (Study Co-Chair), Chitra Ravishankar, Thomas Spray, Meryl Cohen, Marisa Nolan, Stephanie Piacentino, Sandra DiLullo, Nicole Mirarchi; Cincinnati Children’s Medical Center, D. Woodrow Benson (PI), Catherine Dent Krawczeski, Lois Bogenschutz, Teresa Barnard, Michelle Hamstra, Rachel Griffiths, Kathryn Hogan, Steven Schwartz,* David Nelson, Pirooz Eghtesady; North Carolina Consortium: Duke University, East Carolina University, Wake Forest University, Page A. W. Anderson (PI)—deceased, Jennifer Li (PI), Wesley Covitz, Kari Crawford,* Michael Hines, James Jaggers,* Theodore Koutlas, Charlie Sang, Jr, Lori Jo Sutton, Mingfen Xu; Medical University of South Carolina, J. Philip Saul (PI), Andrew Atz, Girish Shirali, Scott Bradley, Eric Graham, Teresa Atz, Patricia Infinger; Primary Children’s Medical Center and the University of Utah, Salt Lake City, Utah, L. LuAnn Minich (PI), John A. Hawkins, Michael Puchalski, Richard V. Williams, Peter C. Kouretas, Linda M. Lambert, Marian E. Shearrow, Jun A. Porter; Hospital for Sick Children, Toronto, Brian McCrindle (PI), Joel Kirsh, Chris Caldarone, Elizabeth Radojewski, Svetlana Khaikin, Susan McIntyre, Nancy Slater; University of Michigan, Caren S. Goldberg (PI), Richard G. Ohye (Study Chair), Cheryl Nowak; Children’s Hospital of Wisconsin and Medical College of Wisconsin, Nancy S. Ghanayem (PI), James S. Tweddell, Kathleen A. Mussatto, Michele A. Frommelt, Peter C. Frommelt, Lisa Young-Borkowski
Auxiliary sites.Children’s Hospital Los Angeles, Alan Lewis (PI), Vaughn Starnes, Nancy Pike; The Congenital Heart Institute of Florida (CHIF), Jeffrey P. Jacobs, MD (PI), James A. Quintessenza, Paul J. Chai, David S. Cooper, J. Blaine John, James C. Huhta, Tina Merola, Tracey Grifith; Emory University, Kirk Kanter (PI), William Mahle, Joel Bond,* Jeryl Huckaby; Nemours Cardiac Center, Christian Pizarro, Carol Prospero, Julie Simons, Gina Baffa, Wolfgang A. Radtke; University of Texas Southwestern Medical Center, Ilana Zeltzer (PI), Tia Tortoriello,* Deborah McElroy, Deborah Town
Angiography core laboratory. Duke University, John Rhodes, J. Curt Fudge
Echocardiography core laboratories. Children’s Hospital of Wisconsin, Peter Frommelt; Children’s Hospital Boston, Gerald Marx.
Genetics core laboratory. Children’s Hospital of Philadelphia, Catherine Stolle
Protocol Review Committee. Michael Artman (Chair), Erle Austin, Timothy Feltes, Julie Johnson, Thomas Klitzner, Jeffrey Krischer, G. Paul Matherne
Data and Safety Monitoring Board, John Kugler (Chair), Rae-Ellen Kavey,* Executive Secretary; David J. Driscoll, Mark Galantowicz, Sally A. Hunsberger, Thomas J. Knight, Holly Taylor, Catherine L. Webb
ClinicalTrials.gov number, NCT00115934.
Disclosures: Authors have nothing to disclose with regard to commercial support.
*No longer at the institution listed.