|Home | About | Journals | Submit | Contact Us | Français|
Kaiser Permanente (KP) has a long-standing commitment to conduct research and report publicly. Simultaneously, it faces a different imperative: harnessing information systems to leverage internal improvements in outcomes, efficiency, and costs. Now that KP HealthConnect, the KP electronic health record, is fully implemented, research challenges at KP are moving away from issues of data access and toward the mechanisms through which raw data create meaningful clinical knowledge that is based on rigorous research. In this report we describe a model for research—the Northern California Division of Research Perinatal Research Unit—that leverages internal and external resources to fulfill these twin missions.
The word Irish is seldom coupled with the word civilization.
—How the Irish Saved Western Civilization, by Thomas Cahill
The word improvement is seldom coupled with the word research.
—Gene Nelson, MD, Dartmouth University
The principal investigator (PI) shares sensitive outcomes data in a meeting with the neonatology chiefs. The graphs clearly show that facilities A and C admit infants with suspected infections at rates that are two to three times higher than those for facility B, despite infant populations that are similar in terms of birth weight, severity of illness, and mortality. Data are stratified using the Score for Neonatal Acute Physiology (SNAP), forestalling a frequent response: “But my babies are sicker!” The meeting is challenging but goes well, establishing a pattern that continues to date: high-quality outcomes data, often combined with algorithms derived from federally funded research projects based at Kaiser Permanente (KP), are shared with clinicians, who respond by changing their admission criteria. Over time, admissions decrease by almost 8%, with no increase in neonatal mortality or morbidity. In addition, infants and their families are spared the disruption and stress associated with unnecessary admissions and separations. Reduced costs with potentially better care, along with an impressive publication record in the peer-reviewed literature—how did KP acquire the ability to have this type of dialogue and enable these kinds of informed, data-driven, evidence-based, operations decisions?
Now that KP HealthConnect, the KP electronic health record, is fully implemented, research challenges at KP are moving away from issues of data access and toward the mechanisms through which raw data create meaningful clinical knowledge that is based on rigorous research. Studies have documented that an average of 17 years elapses between the creation of clinical knowledge and its general use at the front line of care.1 The average time for the entire cycle of knowledge creation—from research idea through funding, data collection, analysis, conclusions, publication, and finally, to broad dissemination—is even longer (Figure 1).
This article explores a model of research and operations analysis that has proven to be very effective: the Northern California Division of Research's Perinatal Research Unit (PRU). This hybrid research model combines the best of traditional research capabilities with a rapid operations research function. As KP strives to improve outcomes by bringing research to bear directly on operational questions this unit provides an example of how KP can shorten the “time to using knowledge” cycle and effectively change clinical practice.
The mission of the PRU is threefold: research, reporting, and ad hoc analysis. PRU staff provide rapid-cycle summary and benchmarking data, as well as an excellent setting for conducting research. Among routine PRU outputs are annual data sets sent to the State of California on behalf of the six Northern California neonatal intensive-care units (NICUs), required for NICU certification by California Children's Services. The group also generates analyses in response to ad hoc queries from clinicians, using the full array of available data at KP.2–4 Eileen Walsh, RN, MPH, PRU Project Manager, notes, “We take someone's ‘I wonder …,’ quantify it, and answer it accurately in a way that has meaning and can be generalized to our entire neonatal population.” Answers to operationally important questions posed by one NICU are often directly applicable to other units—and often serve as the starting point for manuscripts and federal grant applications.
This hybrid research model combines the best of traditional research capabilities with a rapid operations research function.
The interdisciplinary staff at the PRU includes a PI, a project manager, a statistician or analyst, programmers, a research assistant, and other staff. The unit also partners with investigators at other institutions, including the University of California, San Francisco; the University of California, Santa Cruz; Harvard University; and the University of Pennsylvania. The PRU's strong research team, with analytic and statistical expertise, is critical to the unit's success.
The PI, Gabriel Escobar, MD, leads research activities and sets unit priorities. He brings several attributes to his role. Although he is a seasoned and successful traditional researcher, he also has the mind-set of a practicing hospital-based physician. As a clinician, he understands which questions are most pressing for operations and is driven to find answers. Straddling the worlds of research and operations, he is uniquely able to translate between them. His goals have always been to 1) improve the frontline delivery of care at KP and 2) conduct rigorous research. Because Dr Escobar is a physician who is translating research and embedding it into operations, his activities embody the concept of how research and quality improvement can be integrated into the broader KP community.
The work of the PRU is tightly integrated with the decision making of the neonatal chiefs in Northern California, a small specialty group whose visionary leadership has created a highly effective learning community. According to Allen Fischer, MD, Northern California's Regional Director of Neonatology, the value of the PRU is that “their efforts inform our action. When we consider a change in practice, we ask the PRU, ‘What does the literature look like? What do KP outcomes look like?’” The PRU supports neonatologists as they work together to identify new practices by analyzing and showing them local data (baseline and postintervention), gathered from their own nurseries.
Figure 2 describes the interactions between the key groups involved in exploring and identifying changes in practice. Ideas for clinical practice research can come from the Neonatal Chief's Group, the KP HealthConnect NICU/Newborn Governance Team, or the Neonatology Journal Club or “collaboratory,”5 whereby a community of practice uses shared data to improve knowledge and results. The Journal Club meets online one evening per month and draws an audience that includes neonatologists from Southern California and Hawaii as well as invited speakers from multiple universities. The needs of these groups drive much of the PRU's work, and the Northern California Nursery Directors' group, of which Dr Escobar is a sitting member, actively participates in setting PRU priorities.
Beyond the support that Dr Escobar provides to neonatologists to identify needed changes in clinical practice, he brings a hands-on approach to implementing changes in the NICUs. Dr Fischer says that Dr Escobar focuses on the question “‘How do you package new information so that it changes practice?’ As a practicing physician, he needs decision support himself, so he understands how to make it work for others.” Practice changes are facilitated through influence, as most of the key players in this process lack the line authority to mandate practice changes. However, widespread involvement of practicing clinicians with the PRU facilitates buy-in.
The work of the PRU to improve operations has also been furthered through Dr Escobar's relationships with regional operations leadership. He shares data and findings, as well as their implications, with senior leadership; in turn, senior leaders have consistently supported the PRU. A number of his colleagues have noted that Dr Escobar “cares about what leadership cares about.” He concurs: “Senior leaders tend to be interested in things that move the whole system, and that's what I am interested in.” For example, Philip Madvig, MD, Associate Executive Director of The Permanente Medical Group (TPMG), sees the potential for the approach pioneered by Dr Escobar to bring value to medical specialties beyond neonatology. He and Donald Dyson, MD, Associate Executive Director of TPMG, are very supportive of closing the gap between emerging knowledge about effective practices and subsequent physician adoption.
The PRU has demonstrated effectiveness in three dimensions: clinical research, operations analysis and improvement, and leveraging resources.
With 92 peer-reviewed publications and 2945 citations during a period of 15 years, the PRU is considered an authority on newborn treatment management in many areas. For example, the National Institute of Child Health and Human Development (NICHD) views KP as a resource on data on newborns because of the principal PRU database, the Neonatal Minimum Data Set (NMDS). The NMDS is an application based on SAS statistical software (Cary, NC) populated with a standard set of data gathered on all infants admitted to Northern California NICUs. It contains a total of more than 46,000 infant records dated from 1993 to 2009; since 2000, enrollment averages 2969 infants per year. Full data collection for an infant who spends at least 72 hours in the NICU comprises approximately 150 data elements corresponding to maternal and infant demographics, maternal history, intrapartum and delivery details, NICU diagnoses and procedures, severity of illness, and disposition. Reliability of the NMDS data is enabled by ownership and strict quality control by the PRU and its partners. Although the NMDS is a state-of-the-art database and the defining product of the PRU, it sits with and is linked to a wide variety of KP data resources. Project-specific data sets are created for externally funded research studies.
PRU researchers are invited participants at NICHD conferences on jaundice and late-preterm infants. In a recent issue of Pediatrics focusing on jaundice, PRU researchers published a key article6 and were cited in five of six remaining articles,7–11 as well as in editorials. The official Centers for Disease Control Guidelines for Prevention of Perinatal Group B Streptococcal Disease cite a study based on a PRU population on “rule out sepsis.” The PRU's H index (a measure of research productivity and impact) is 30,12 and its funding rate is roughly 50%.
Another stream of research conducted during a multiyear period centers on an issue that has not received rigorous research attention in the academic literature: respiratory distress in babies born at 34 weeks' gestation or later. PRU work in this area resulted in the “Big Babies Breathing Hard” project. Central to this work is the Richardson score, which permits a rapid, quantitative assessment of the severity and the prognosis of respiratory distress. Additional areas of contribution are neonatal sepsis (identification, prevention, clinical management, outcomes), effects of maternal substance abuse during pregnancy, risk factors for rehospitalization among newborns, hyperbilirubinemia (effectiveness of screening and treatment strategies), and longitudinal outcomes for NICU survivors. An extensive bibliography can be found at http://dor-ent1.kaiser.org/staff/investigators/escobar.htm (password protected).a
An equally important area of focus for the PRU is operations analysis and improvement. Driven by clinician questions, this work may or may not result in a publication, but it directly contributes to improved patient outcomes. PRU research on operational questions is conducted with the same data and the same resources for analytic rigor as clinical research and often uses the knowledge gained from traditional clinical research projects. The NMDS database, the PRU staffing structure, and strong analytic expertise all enable analysis.
The queries that PRU receives from operations run the gamut from a low level of analytic complexity (eg, a clinician who requests simple counts of number of ventilated babies younger than 32 weeks' gestation) to moderately complex (eg, a query about appropriate referral of mothers at risk of delivering late-preterm, multiple-gestation infants) and very complex (eg, research studying the effects of neonatal nosocomial infection on hospital length of stay and mortality). The response time to queries depends on the level of complexity. Most simple queries can be answered within days. More complex queries (eg, “Are the outcomes for respiratory distress in full-term infants the same across our units?”) often lead to more elaborate answers13,14 and sometimes lead Dr Escobar to submit a formal grant proposal. Because publishing can take a longer time, Dr Escobar accelerates knowledge sharing by circulating draft manuscripts internally with the Nursery Directors, instead of waiting for publication (up to two years).
Under a long-standing collaboration with the University of California, San Francisco, the PRU has also played an important role in how KP clinicians manage neonatal hyperbilirubinemia. Working with a nationally recognized jaundice expert, Thomas Newman, MD, the PRU initially contributed considerable data, consultation support, and paper tools to efforts by the Chiefs of Pediatrics and Nursery Directors to implement the American Academy of Pediatrics clinical practice guideline for hyperbilirubinemia. These efforts continue, but the PRU is now shifting its emphasis to KP HealthConnect, where it played a major role in developing and implementing an automated hyperbilirubinemia assessment tool embedded in the electronic health record. Currently in beta testing, this tool will be rolled out to the Northern California Region this year.
Ongoing support for PRU activities comes from KP and external grants. Between 2000 and 2009, Kaiser Foundation Hospitals and Health Plan provided an average of approximately $600,000 per year in direct support for the NMDS database, whereas TPMG provided approximately $200,000 per year in support for programming and consultation. Since 2000, the PRU has also averaged $1 million per year in external funding from the federal government, foundations, and industry. Clearly, grantors have benefited from Dr Escobar's operational insights and relationships, and KP has benefited from the research conducted for external sources. Everyone wins.
Although the NMDS database was initially expensive to build and maintain, it provides value to KP on multiple levels. Ready access to this database—with pilot data ready in days—makes the PRU very competitive in securing external grants. The results of research using the NMDS have answered clinical questions and provided real benefit to KP in “rule out sepsis,” jaundice,15 dehydration,16 outcomes13,14,17 for late-preterm infants (Escobar GJ [PI]: Sepsis and critical illness in babies at 34 weeks gestation and longer. Study in progress; funded by the National Institute for General Medical Sciences), and respiratory distress in full-term babies.
Serving the two masters of research and operations is not without challenges. The benefits to KP of the PRU's ready and rigorous response to clinical questions are clear, but the close association between research and operations may also raise concerns. For instance, allowing operations leaders to influence the choice of research topics may be perceived as impinging on a PI's intellectual integrity. However, Dr Escobar finds interesting clinical research topics and contributes in meaningful ways without jeopardizing his reputation or the integrity of the research. Both the PI and KP operational leaders have a common goal (improved clinical care), so there is much less potential conflict of interest than in other, more conventional relationships, such as between researchers and pharmaceutical companies.
Equally important is the PRU's policy of transparency about research results. Results from studies approved by the institutional review board are always submitted for publication, even if they make clinicians uncomfortable. This was the case with an Escobar study on neonatal “sepsis workup” that found that only 78% of newborns who met the study's definition for “critical illness” had been treated with systemic antibiotics.3,18 Initial reaction to this finding was consternation, but after the results were confirmed by a repeat audit, the neonatologists took a different stance: They changed the guideline for such infants. On the other hand, because the PRU aims at systematic knowledge, it handles incidental findings during the course of a study in a different manner. For example, a study-required record review might reveal an instance of apparently inappropriate care. Because the PRU is not charged with local quality assurance, this information is not published—nor is it suppressed. In these situations, the PRU simply refers the case to the appropriate facility authority (usually, the Nursery Director).
In the end, no attempt at any kind of organizational censorship of the published study has occurred, and the Neonatology Chiefs' culture of transparency allows them to improve care and to establish a precedent for performance improvement.
The PRU brings to knowledge creation an effective balance between rigor of methodology and speed.
A number of factors differentiate the PRU model and contribute to its success:
Not all research units engage in answering clinical and operations questions with such agility, nor are many regional operations analysis units set up to manage the subtle and clinical nature of some of these questions. The PRU brings to knowledge creation an effective balance between rigor of methodology and speed. Rigorous research work and research capability are leveraged to inform questions of day-to-day clinical practice in a more timely way.
The work of the PRU is enabled by aligned interests and strong partnerships with clinicians, medical leadership, and regional executive operations leadership. The PRU has made a substantial contribution to Northern California neonatology's reputation as among the best in the state. The model has now been extended to adult hospital care in Northern California, and a similar model is being explored in Southern California.
The PRU has developed a synergistic approach to using internal and external funding sources while effectively meeting the requirements of both. Dr Escobar has maintained the highest standards of scholarship while exploring issues that directly affect operations. Most importantly, the leadership, transparency, and partnership demonstrated by Northern California research, operations, and neonatology have resulted in demonstrably better care and outcomes for the mothers and babies of KP Northern California.
The author(s) have no conflicts of interest to disclose.
We are sincerely grateful for the open access to information and insights provided by Gabriel Escobar, MD, and Eileen Walsh, RN, MPH. We also appreciate the time and wisdom contributed by senior TPMG leadership. Finally, we thank the physicians and staff of the Northern California perinatal services in their continuing efforts to improve care for our patients.
Katharine O'Moore-Klopf, ELS, of KOK Edit provided editorial assistance.
a Those outside the Kaiser Permanente organization may send questions concerning investigators to gro.resiak.rod@ksedplehrod.