|Home | About | Journals | Submit | Contact Us | Français|
The ability to measure health system quality has become a priority for governments, the private sector, and the public. Quality indicators (QIs) refer to clear, measurable items related to outcomes. The use of QIs can initiate local quality improvement and track changes in quality over time as interventions are implemented.
We identified existing evidence-based indicators of quality pediatric orthopaedic care and evaluated published QIs that may be applicable to pediatric orthopaedic care.
Using five standard search engines we searched the literature using terms such as “quality indicators,” “orthopaedic surgery,” and “pediatric.” Study selection was performed in a stepwise manner, first by title, then abstract, and then full-text review. Of the 604 citations identified, 13 articles were selected for inclusion. Eight papers included only pediatric patients.
The most commonly reported indicator was mortality followed by postoperative complications. Reoperation and readmission rates were also reported along with patient-centered QIs, although with less frequency.
Although mortality and postoperative complications were the most frequently reported QIs, concern for their applicability was raised because of their relative infrequency in pediatrics. Patient-centered QIs appear to be the most useful tools reported, although their use is somewhat limited in the published literature. Although there are benefits and drawbacks to all reported QIs, patient-centered and surgeon-defined outcomes along with cost-effectiveness have important roles in evaluating the quality of pediatric orthopaedic care.
Health system improvement has become a priority for governments as well as the public. In 2004, unintentional and intentional injuries were estimated to cost Canadians $19.8 billion with almost 10% of this a result of falls in patients younger than 19 years of age . Although the economic burden of injury is documented, it is still unclear as to what impact quality of care has on the overall health of patients. Currently, almost one-third of children with time-sensitive orthopaedic problems will have a problem with diagnosis, treatment, or delay in treatment before arriving at a tertiary pediatric center . Various national and regional bodies in Canada and the United States now mandate reporting on health system quality [1, 18, 23, 32]. This has created a pressing need for published information on best practices to study and report on health system quality.
The assessment of surgical performance has evolved considerably over the past decades through the reporting of quality indicators. Quality indicators (QIs) refer to well-defined, structural, and process measures that are associated with outcomes. Although adult surgery has been the initial focus of public reporting efforts, pediatric surgery is increasingly included [6, 15, 27, 34]. Some efforts have been made in congenital heart surgery , juvenile arthritis [3, 13] and trauma [29, 40] to examine outcomes. The Agency for Healthcare Research and Quality has commissioned a report on pediatric QIs and although postoperative hemorrhage and hematomas were included, no specific orthopaedic indicators were investigated . Postoperative infection is believed to be important; however, standard hospital-based surveillance systems often fail in their ability to detect postdischarge surgical site infections . There is a movement to implement a pilot project to assess mortality and perioperative morbidity for inpatient surgical procedures in children through the administrative database Children’s National Surgical Quality Improvement Program . However, this program and others are not designed to address pediatric surgical care specifically and have reported challenges and barriers to implementation .
There is increasing evidence that the evaluation of pediatric surgical programs for quality is more complicated than the simple comparison of mortality rates or case volume, and currently, well-defined metrics specific to the pediatric population are scarce in the medical literature as a whole [31, 36]. Although the many unique qualities of pediatric surgical care provide challenges to the development of a quality improvement program, QIs can provide an opportunity to initiate local quality improvement and track changes in quality over time as interventions are implemented [4, 22]. Research focused on developing, evaluating, or improving QIs relevant to the pediatric population is critical.
The aim of this systematic review is to assess the existing evidence-based indicators of quality of care in pediatric orthopaedics. First, we aim to identify published QI through a survey of the literature. Second, we evaluate existing QIs by reporting on their use in pediatric orthopaedic care. Third, we make recommendations for the future of QIs in pediatric orthopaedic care.
A search of the literature using the MEDLINE, Cochrane Database of Systematic Reviews, CENTRAL (Cochrane Central Register of Controlled Trials), and the Journal of Pediatric Orthopedics databases was performed using key words and MeSH headings. Relevant articles were included in this review if they described or used any quality-of-care indicator in pediatric orthopaedic surgery or any orthopaedic surgical specialty. Articles that did not describe or use a quality-of-care indicator in their outcome measures or did not include any pediatric patients or patients specific to an orthopaedic specialty were excluded from this review. There were no restrictions based on language, publication date, or study design. The search was conducted in April 2011 and a comprehensive list of studies was compiled in bibliographic software.
The MEDLINE search strategy was: “quality indicator$.mp. or Quality Indicators, Health Care/ “Quality of Health Care”/ or Quality Assurance, Health Care/ or “Outcome and Process Assessment (Health Care)”/ or quality measure$.mp. or Quality Indicators, Health Care/ or performance improvement.mp. or best practice.mp. AND Orthopedic Procedures/ or Orthopedics/ or orthopaedic$.mp.”
Two of us (CB, AK) critically appraised the studies and independently extracted the data from each study. Study selection was performed in a stepwise manner, first by title, then abstract, and then full-text review. Bibliographies of the identified articles were studied to include any missed articles.
The search strategy generated 604 citations (Fig. 1). Thirteen articles met our inclusion criteria and are reviewed in this article (Table 1) [2, 9–11, 14, 16, 17, 21, 24, 26, 30, 33, 34]. Eight papers specifically looked at pediatric patients and included no adults. Two papers were specifically focused on orthopaedics alone and the remaining 11 papers included some orthopaedics patients plus patients from at least one other specialty. Most study designs were either a systematic review or survey (five of 13 articles) or a retrospective review of administrative databases (five of 13 articles). Only one paper was a prospective intervention study evaluating the impact of the World Health Organization Safety Checklist on major complications.
The most common indicator cited was mortality (six of 13 articles). Of these six articles, four of the papers specifically looked at pediatric patients.
Postoperative complications were the second most commonly cited QIs in our review (five of 13 articles); however, further study and validation were recommended. Only one of these papers was specifically looking at orthopaedic surgery and this paper was a short communication reporting that teaching hospitals have better outcomes than nonteaching hospitals . The need for risk adjustment was reported along with the difficulty and added complexity in performing this in pediatric patients.
Reoperation and readmission rates were the next most frequently reported QIs [9, 10, 16] along with patient-centered QIs [2, 17, 21]. The Chambers and Clarke and Haynes et al. articles were neither pediatric nor orthopaedic-specific so it is difficult to delineate what the reoperation or readmission rates would be for these specific populations. However, Cox and Clarke reported that 42 of 398 children (12.1% of fractures) were readmitted on a further 47 occasions in connection with their original injury . Beal et al. used the Institute of Medicine national healthcare quality report framework that includes a separate patient-perspective domain . In addition, Beal et al. report that almost one-third (32.1%) of measures within the healthcare quality domain were related to patient-centeredness. The vast network of Shriners Hospitals for Children treats orthopaedic problems and burns and began a systemwide outcome management effort to validate their quality of care using a patient and family perception of care inpatient survey . No pediatric subjects were included in the Idvall report .
In a recent Policy Statement, the American Academy of Pediatrics (AAP) summarized efforts on quality measurement and reinforced that current efforts often failed to recognize the unique needs of children and their families . The AAP advocates the use of pediatric measurement data, including public reporting on validated pediatric measures that are appropriately constructed for quality improvement . The Canadian Pediatric Society (CPS) has also made quality pediatric care a priority , albeit to a lesser extent than the AAP. In a 2009 Position Statement that broadly addressed the role of pediatricians in the health needs of youth, the CPS stated that children and youth who are hospitalized should have access to quality, specialized pediatric expertise that meets their needs . Following these position statements, we conducted a systematic review to identify and evaluate existing published QIs in pediatric orthopaedic care.
Our review is limited by a number of factors. First, we did not search using Google Scholar. One major advantage of Medline is that it is readily updated not only with printed literature, but also with literature that has been presented online in an early version before print publication by various journals . Second, our search did not include a review of all published and unpublished literature and did not include databases that are not peer-reviewed [6, 12]. A scoping review is a consideration for the future. Third, the strength of our recommendations is based on the strength of the results. The articles identified in this search strategy were largely retrospective reviews of administrative databases, other systematic reviews, or surveys. Health record databases are largely administrative and retrospective. This can be problematic because the data depend on medical record coders, whose major focus is billing . One large national database in Canada is the Canadian Hospitals Injury Reporting and Prevention Program (CHIRPP), a computerized information system that collects and analyzes data on injuries at participating hospitals since 1990 . Although more than 80% of the more than 1.5 million records in the CHIRPP data set capture pre-event information on pediatric injuries, the data that are collected are mainly used to establish injury prevention guidelines and do not contain any measures of quality of care or patient-specific outcomes .
Mortality, postoperative complications, and readmission rates and reoperation rates were the most commonly reported QIs in pediatric orthopaedics despite the fact that they have been adapted from the adult literature and are less than ideal in the pediatric population. For example, despite the use of prophylactic antibiotics a minimum of 60 minutes before incision in adult surgery, this may not be appropriate as a quality measure in pediatric surgery because prophylactic antibiotics are often not given as a result of the lack of supporting evidence . In the papers that cited mortality, most authors concluded mortality was not a useful indicator. Although experts in this field rank mortality as a potentially very useful indicator and one that is also very likely to prompt action for improving quality of care, this is one QI that is not particularly sensitive to pediatric use because of its relative rarity in children’s hospitals and because death is not often linked to a preventable error or complication. In a 1-year review of all pediatric orthopaedic patients, Cox and Clarke reported a zero mortality rate . The article by Sedman et al. on 1.92 million discharges across 4 years of data from children’s hospitals in 31 states reported that the majority of mortality cases were very complex cases for which death was a nonpreventable rather than a preventable complication .
Many unique qualities of pediatric surgical care provide challenges to the development of a quality improvement program. There is marked heterogeneity in outcomes of surgical procedures in children, some resulting from the physiological changes in function, which are a part of normal development and growth . As well, a positive relationship between volume and outcome has been reported for various orthopaedic procedures [31, 36]. However, most procedures in pediatrics have much lower volume than adult procedures and mortality rates are so low that statistical analysis and modeling is challenging if not impossible . In addition, limitations exist in applying this type of information without risk adjustment . Risk adjustment involves accounting for patients’ characteristics that influence quality measures but are not under the control of the provider . Risk adjustment is also difficult in pediatrics.
Cost-effectiveness was notably absent as an identified measure of quality in our review. Cost-effectiveness analysis (CEA) provides an important tool by which policymakers may assess and potentially increase the return on healthcare investments . Although CEA is most useful for the evaluation of new technologies such as in knee and hip arthroplasties, there may be a place for CEA in pediatrics surgery [5, 28].
Our review of the literature looking for specific QIs that may be applicable to pediatric orthopaedics highlights that although there appears to be a trend toward performing patient-centered outcomes, there is a long way to go to be able to reliably evaluate the quality of pediatric surgical programs. Patient-centered outcomes capture the impact of a patient’s overall well-being much better than traditional, surgeon-defined outcomes such as mortality or infection rates . Generic health utilities have limited value in pediatric quality of care evaluation because of they are ill-equipped to account for the developmental changes resulting from normal growth and many are only appropriate for older children.
Although there are benefits and drawbacks to all reported QIs, patient-centered and surgeon-defined outcomes along with cost-effectiveness have important roles in evaluating the quality of pediatric orthopaedic care. When we look specifically at the subspecialty of pediatric orthopaedics, there is a dearth of reliable or applicable measures to address quality of care on the whole. The next step in quality measurement will require targeting specific areas for further literature review to identify any available clinical practice guidelines or evidence to be able to begin to develop some disease-specific measures that can then be assessed for feasibility. Pediatric orthopaedic care cannot be improved until we can better measure quality and identify areas to target for improvement. If there is to be a commitment to providing the highest quality and safest health care for infants, children, adolescents, and young adults, quality measurement must be made a priority.
Each author certifies that he or she has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.
The University of Calgary Department of Surgery Research Prize and the University of Calgary COREF Grant provided educational grants to complete this research.