Search tips
Search criteria 


Logo of corrspringer.comThis journalToc AlertsSubmit OnlineOpen Choice
Clin Orthop Relat Res. 2012 April; 470(4): 1124–1132.
Published online 2011 September 13. doi:  10.1007/s11999-011-2060-2
PMCID: PMC3293946

Quality Indicators in Pediatric Orthopaedic Surgery: A Systematic Review

Angeliki Kennedy, MSc,1 Christina Bakir, MD,1 and Carmen A. Brauer, MD, MSc (Health Econ), FRCSCcorresponding author1,2



The ability to measure health system quality has become a priority for governments, the private sector, and the public. Quality indicators (QIs) refer to clear, measurable items related to outcomes. The use of QIs can initiate local quality improvement and track changes in quality over time as interventions are implemented.


We identified existing evidence-based indicators of quality pediatric orthopaedic care and evaluated published QIs that may be applicable to pediatric orthopaedic care.

Search Strategy

Using five standard search engines we searched the literature using terms such as “quality indicators,” “orthopaedic surgery,” and “pediatric.” Study selection was performed in a stepwise manner, first by title, then abstract, and then full-text review. Of the 604 citations identified, 13 articles were selected for inclusion. Eight papers included only pediatric patients.


The most commonly reported indicator was mortality followed by postoperative complications. Reoperation and readmission rates were also reported along with patient-centered QIs, although with less frequency.


Although mortality and postoperative complications were the most frequently reported QIs, concern for their applicability was raised because of their relative infrequency in pediatrics. Patient-centered QIs appear to be the most useful tools reported, although their use is somewhat limited in the published literature. Although there are benefits and drawbacks to all reported QIs, patient-centered and surgeon-defined outcomes along with cost-effectiveness have important roles in evaluating the quality of pediatric orthopaedic care.


Health system improvement has become a priority for governments as well as the public. In 2004, unintentional and intentional injuries were estimated to cost Canadians $19.8 billion with almost 10% of this a result of falls in patients younger than 19 years of age [38]. Although the economic burden of injury is documented, it is still unclear as to what impact quality of care has on the overall health of patients. Currently, almost one-third of children with time-sensitive orthopaedic problems will have a problem with diagnosis, treatment, or delay in treatment before arriving at a tertiary pediatric center [37]. Various national and regional bodies in Canada and the United States now mandate reporting on health system quality [1, 18, 23, 32]. This has created a pressing need for published information on best practices to study and report on health system quality.

The assessment of surgical performance has evolved considerably over the past decades through the reporting of quality indicators. Quality indicators (QIs) refer to well-defined, structural, and process measures that are associated with outcomes. Although adult surgery has been the initial focus of public reporting efforts, pediatric surgery is increasingly included [6, 15, 27, 34]. Some efforts have been made in congenital heart surgery [20], juvenile arthritis [3, 13] and trauma [29, 40] to examine outcomes. The Agency for Healthcare Research and Quality has commissioned a report on pediatric QIs and although postoperative hemorrhage and hematomas were included, no specific orthopaedic indicators were investigated [1]. Postoperative infection is believed to be important; however, standard hospital-based surveillance systems often fail in their ability to detect postdischarge surgical site infections [25]. There is a movement to implement a pilot project to assess mortality and perioperative morbidity for inpatient surgical procedures in children through the administrative database Children’s National Surgical Quality Improvement Program [27]. However, this program and others are not designed to address pediatric surgical care specifically and have reported challenges and barriers to implementation [35].

There is increasing evidence that the evaluation of pediatric surgical programs for quality is more complicated than the simple comparison of mortality rates or case volume, and currently, well-defined metrics specific to the pediatric population are scarce in the medical literature as a whole [31, 36]. Although the many unique qualities of pediatric surgical care provide challenges to the development of a quality improvement program, QIs can provide an opportunity to initiate local quality improvement and track changes in quality over time as interventions are implemented [4, 22]. Research focused on developing, evaluating, or improving QIs relevant to the pediatric population is critical.

The aim of this systematic review is to assess the existing evidence-based indicators of quality of care in pediatric orthopaedics. First, we aim to identify published QI through a survey of the literature. Second, we evaluate existing QIs by reporting on their use in pediatric orthopaedic care. Third, we make recommendations for the future of QIs in pediatric orthopaedic care.

Search Strategy and Criteria

A search of the literature using the MEDLINE, Cochrane Database of Systematic Reviews, CENTRAL (Cochrane Central Register of Controlled Trials), and the Journal of Pediatric Orthopedics databases was performed using key words and MeSH headings. Relevant articles were included in this review if they described or used any quality-of-care indicator in pediatric orthopaedic surgery or any orthopaedic surgical specialty. Articles that did not describe or use a quality-of-care indicator in their outcome measures or did not include any pediatric patients or patients specific to an orthopaedic specialty were excluded from this review. There were no restrictions based on language, publication date, or study design. The search was conducted in April 2011 and a comprehensive list of studies was compiled in bibliographic software.

The MEDLINE search strategy was: “quality indicator$.mp. or Quality Indicators, Health Care/ “Quality of Health Care”/ or Quality Assurance, Health Care/ or “Outcome and Process Assessment (Health Care)”/ or quality measure$.mp. or Quality Indicators, Health Care/ or performance or best AND Orthopedic Procedures/ or Orthopedics/ or orthopaedic$.mp.”

Two of us (CB, AK) critically appraised the studies and independently extracted the data from each study. Study selection was performed in a stepwise manner, first by title, then abstract, and then full-text review. Bibliographies of the identified articles were studied to include any missed articles.

The search strategy generated 604 citations (Fig. 1). Thirteen articles met our inclusion criteria and are reviewed in this article (Table 1) [2, 911, 14, 16, 17, 21, 24, 26, 30, 33, 34]. Eight papers specifically looked at pediatric patients and included no adults. Two papers were specifically focused on orthopaedics alone and the remaining 11 papers included some orthopaedics patients plus patients from at least one other specialty. Most study designs were either a systematic review or survey (five of 13 articles) or a retrospective review of administrative databases (five of 13 articles). Only one paper was a prospective intervention study evaluating the impact of the World Health Organization Safety Checklist on major complications.

Fig. 1
The study selection trial flow diagram is shown.
Table 1
Literature review for quality indicators in pediatric orthopaedics


The most common indicator cited was mortality (six of 13 articles). Of these six articles, four of the papers specifically looked at pediatric patients.

Postoperative complications were the second most commonly cited QIs in our review (five of 13 articles); however, further study and validation were recommended. Only one of these papers was specifically looking at orthopaedic surgery and this paper was a short communication reporting that teaching hospitals have better outcomes than nonteaching hospitals [26]. The need for risk adjustment was reported along with the difficulty and added complexity in performing this in pediatric patients.

Reoperation and readmission rates were the next most frequently reported QIs [9, 10, 16] along with patient-centered QIs [2, 17, 21]. The Chambers and Clarke and Haynes et al. articles were neither pediatric nor orthopaedic-specific so it is difficult to delineate what the reoperation or readmission rates would be for these specific populations. However, Cox and Clarke reported that 42 of 398 children (12.1% of fractures) were readmitted on a further 47 occasions in connection with their original injury [10]. Beal et al. used the Institute of Medicine national healthcare quality report framework that includes a separate patient-perspective domain [2]. In addition, Beal et al. report that almost one-third (32.1%) of measures within the healthcare quality domain were related to patient-centeredness. The vast network of Shriners Hospitals for Children treats orthopaedic problems and burns and began a systemwide outcome management effort to validate their quality of care using a patient and family perception of care inpatient survey [21]. No pediatric subjects were included in the Idvall report [17].


In a recent Policy Statement, the American Academy of Pediatrics (AAP) summarized efforts on quality measurement and reinforced that current efforts often failed to recognize the unique needs of children and their families [39]. The AAP advocates the use of pediatric measurement data, including public reporting on validated pediatric measures that are appropriately constructed for quality improvement [39]. The Canadian Pediatric Society (CPS) has also made quality pediatric care a priority [8], albeit to a lesser extent than the AAP. In a 2009 Position Statement that broadly addressed the role of pediatricians in the health needs of youth, the CPS stated that children and youth who are hospitalized should have access to quality, specialized pediatric expertise that meets their needs [8]. Following these position statements, we conducted a systematic review to identify and evaluate existing published QIs in pediatric orthopaedic care.

Our review is limited by a number of factors. First, we did not search using Google Scholar. One major advantage of Medline is that it is readily updated not only with printed literature, but also with literature that has been presented online in an early version before print publication by various journals [12]. Second, our search did not include a review of all published and unpublished literature and did not include databases that are not peer-reviewed [6, 12]. A scoping review is a consideration for the future. Third, the strength of our recommendations is based on the strength of the results. The articles identified in this search strategy were largely retrospective reviews of administrative databases, other systematic reviews, or surveys. Health record databases are largely administrative and retrospective. This can be problematic because the data depend on medical record coders, whose major focus is billing [34]. One large national database in Canada is the Canadian Hospitals Injury Reporting and Prevention Program (CHIRPP), a computerized information system that collects and analyzes data on injuries at participating hospitals since 1990 [7]. Although more than 80% of the more than 1.5 million records in the CHIRPP data set capture pre-event information on pediatric injuries, the data that are collected are mainly used to establish injury prevention guidelines and do not contain any measures of quality of care or patient-specific outcomes [7].

Mortality, postoperative complications, and readmission rates and reoperation rates were the most commonly reported QIs in pediatric orthopaedics despite the fact that they have been adapted from the adult literature and are less than ideal in the pediatric population. For example, despite the use of prophylactic antibiotics a minimum of 60 minutes before incision in adult surgery, this may not be appropriate as a quality measure in pediatric surgery because prophylactic antibiotics are often not given as a result of the lack of supporting evidence [16]. In the papers that cited mortality, most authors concluded mortality was not a useful indicator. Although experts in this field rank mortality as a potentially very useful indicator and one that is also very likely to prompt action for improving quality of care, this is one QI that is not particularly sensitive to pediatric use because of its relative rarity in children’s hospitals and because death is not often linked to a preventable error or complication. In a 1-year review of all pediatric orthopaedic patients, Cox and Clarke reported a zero mortality rate [10]. The article by Sedman et al. on 1.92 million discharges across 4 years of data from children’s hospitals in 31 states reported that the majority of mortality cases were very complex cases for which death was a nonpreventable rather than a preventable complication [33].

Many unique qualities of pediatric surgical care provide challenges to the development of a quality improvement program. There is marked heterogeneity in outcomes of surgical procedures in children, some resulting from the physiological changes in function, which are a part of normal development and growth [2]. As well, a positive relationship between volume and outcome has been reported for various orthopaedic procedures [31, 36]. However, most procedures in pediatrics have much lower volume than adult procedures and mortality rates are so low that statistical analysis and modeling is challenging if not impossible [9]. In addition, limitations exist in applying this type of information without risk adjustment [36]. Risk adjustment involves accounting for patients’ characteristics that influence quality measures but are not under the control of the provider [19]. Risk adjustment is also difficult in pediatrics.

Cost-effectiveness was notably absent as an identified measure of quality in our review. Cost-effectiveness analysis (CEA) provides an important tool by which policymakers may assess and potentially increase the return on healthcare investments [5]. Although CEA is most useful for the evaluation of new technologies such as in knee and hip arthroplasties, there may be a place for CEA in pediatrics surgery [5, 28].

Our review of the literature looking for specific QIs that may be applicable to pediatric orthopaedics highlights that although there appears to be a trend toward performing patient-centered outcomes, there is a long way to go to be able to reliably evaluate the quality of pediatric surgical programs. Patient-centered outcomes capture the impact of a patient’s overall well-being much better than traditional, surgeon-defined outcomes such as mortality or infection rates [28]. Generic health utilities have limited value in pediatric quality of care evaluation because of they are ill-equipped to account for the developmental changes resulting from normal growth and many are only appropriate for older children.

Although there are benefits and drawbacks to all reported QIs, patient-centered and surgeon-defined outcomes along with cost-effectiveness have important roles in evaluating the quality of pediatric orthopaedic care. When we look specifically at the subspecialty of pediatric orthopaedics, there is a dearth of reliable or applicable measures to address quality of care on the whole. The next step in quality measurement will require targeting specific areas for further literature review to identify any available clinical practice guidelines or evidence to be able to begin to develop some disease-specific measures that can then be assessed for feasibility. Pediatric orthopaedic care cannot be improved until we can better measure quality and identify areas to target for improvement. If there is to be a commitment to providing the highest quality and safest health care for infants, children, adolescents, and young adults, quality measurement must be made a priority.


Each author certifies that he or she has no commercial associations (eg, consultancies, stock ownership, equity interest, patent/licensing arrangements, etc) that might pose a conflict of interest in connection with the submitted article.

The University of Calgary Department of Surgery Research Prize and the University of Calgary COREF Grant provided educational grants to complete this research.


1. AHRQ Quality Indicators. Pediatric Quality Indicators Overview. February 2006. Agency for Healthcare Research and Quality, Rockville, MD. Available at: Accessed December 6, 2010.
2. Beal AC, Co JT, Dougherty D, Jorsling T, Kam J, Perrin J, Palmer RH. Quality measures for children’s health care. Pediatrics. 2004;113:199–209. [PubMed]
3. Bordley WC. Outcomes research and emergency medical services for children: domains, challenges, and opportunities. Ambul Pediatr. 2002;2:306–310. doi: 10.1367/1539-4409(2002)002<0306:ORAEMS>2.0.CO;2. [PubMed] [Cross Ref]
4. Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, Radford MJ, Krumholz HM. A qualitative study of increasing beta-blocker use after myocardial infarction: why do some hospitals succeed? JAMA. 2001;285:2604–2611. doi: 10.1001/jama.285.20.2604. [PubMed] [Cross Ref]
5. Brauer CA, Neumann PJ, Rosen AB. Trends in cost-effectiveness analyses in orthopedic surgery. Clin Orthop Relat Res. 2007;457:42–48. [PubMed]
6. Brien SE, Lorenzetti DL, Lewis S, Kennedy J, Ghali WA. Overview of a formal scoping review on health system report cards. Implement Sci. 2010;5:1–12. doi: 10.1186/1748-5908-5-2. [PMC free article] [PubMed] [Cross Ref]
7. Canadian Hospitals Injury Reporting and Prevention Program (CHIRPP). Injury data were obtained from the database of the Canadian Hospitals Injury Reporting and Prevention Program (CHIRPP), Public Health Agency of Canada. Available at: Accessed April 17, 2011.
8. Canadian Pediatric Society. A model of paediatrics: rethinking health care for children and youth. Position Statement (CPS 2009–01). Paediatr Child Health. 2009;14:319–325. [PMC free article] [PubMed]
9. Chambers M, Clarke A. Measuring readmission rates. BMJ. 1990;301:1134–1136. doi: 10.1136/bmj.301.6761.1134. [PMC free article] [PubMed] [Cross Ref]
10. Cox PJ, Clarke NM. Improving the outcome of paediatric orthopaedic trauma: an audit of inpatient management in Southampton. Ann R Coll Surg Engl. 1997;79:441. [PMC free article] [PubMed]
11. Croft AM, Lynch P, Smellie JS, Dickinson CJ. Outpatient waiting times: indicators of hospital performance? J R Army Med Corps. 1998;144:131–137. [PubMed]
12. Falagas ME, Pitsouni EI, Malietzis GA, Pappas G. Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses. FASEB J. 2008;22:338–342. doi: 10.1096/fj.07-9492LSF. [PubMed] [Cross Ref]
13. Feldman DE, Bernatsky S, Houde M. The incidence of juvenile rheumatoid arthritis in Quebec: a population data-based study. Pediatr Rheumatol. 2009;7:20. doi: 10.1186/1546-0096-7-20. [PMC free article] [PubMed] [Cross Ref]
14. Gallagher C. Applying quality improvement tools to quality planning: pediatric femur fracture clinical path development. J Healthcare Q. 1994;16:6–14. doi: 10.1111/j.1945-1474.1994.tb00705.x. [PubMed] [Cross Ref]
15. Guttman A, Razzaq A, Lindsay P, Zagorski B, Anderson GM. Development of measures of the quality of emergency department care for children using a structured panel process. Pediatrics. 2006;118:114–123. doi: 10.1542/peds.2005-3029. [PubMed] [Cross Ref]
16. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat AH, Dellinger EP, Herbosa T, Joseph S, Kibatala PL, Lapitan MC, Merry AF, Moorthy K, Reznick RK, Taylor B, Gawande AA. Safe Surgery Saves Lives Study Group, A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–499. doi: 10.1056/NEJMsa0810119. [PubMed] [Cross Ref]
17. Idvall E, Hamrin E, Sjöström B, Unosson M. Patient and nurse assessment of quality of care in postoperative pain management. Qual Saf Health Care. 2002;11:327–334. doi: 10.1136/qhc.11.4.327. [PMC free article] [PubMed] [Cross Ref]
18. Kirby MJL (Chair), LeBreton M. The Health of Canadians—The Federal Role (Final Report). The Standing Senate Committee on Social Affairs, Science, and Technology, October 2002. Available at: Accessed December 6, 2010.
19. Kuhlthau K, Ferris TG, Iezzoni LI. Risk adjustment for pediatric quality indicators. Pediatrics. 2004;113:210–216. doi: 10.1542/peds.113.3.e249. [PubMed] [Cross Ref]
20. Lacour-Gayet F, Clarke DR. the Aristotle Committee, The Aristotle method: a new concept to evaluate quality of care based on complexity. Curr Opin Pediatr. 2005;17:412–417. doi: 10.1097/01.mop.0000165361.05587.b9. [PubMed] [Cross Ref]
21. Lanford A, Clausen R, Mulligan J, Hollenback C, Nelson S, Smith V. Measuring and improving patients’ and families’ perceptions of care in a system of pediatric hospitals. J Quality Improve. 2001;278:415–429. [PubMed]
22. Mazankowski D (Chair). A Framework for Reform: Report of the Premier’s Advisory Council on Health. Alberta Health and Wellness, February 2002. Available at: Accessed December 6, 2010.
23. McDonald KM, Davies SM, Haberland CA, Gepper JJ, Ku A, Romano PS. Preliminary assessment of pediatric health care quality and patient safety in the United States using readily available administrative data. Pediatrics. 2008;122:e416. doi: 10.1542/peds.2007-2477. [PubMed] [Cross Ref]
24. Michelson J. Improved detection of orthopaedic surgical site infections occurring in outpatients. Clin Orthop Relat Res. 2005;433:218–224. doi: 10.1097/01.blo.0000150666.06175.6b. [PubMed] [Cross Ref]
25. Moore JD., Jr Teaching hospitals have better outcomes. Modern Healthcare. 1999;29:34. [PubMed]
26. Morrato EH, Dillon P, Ziegler M. Surgical outcomes research: a progression from performance audits, to administrative databases, to prospective risk-adjusted analysis—how far have we come? Curr Opin Pediatr. 2008;20:320–325. doi: 10.1097/MOP.0b013e3283005857. [PubMed] [Cross Ref]
27. Novak EJ, Vail TP, Bozic KJ. Advances in orthopaedic outcomes research. J Surg Orth Adv. 2008;17:200–203. [PubMed]
28. Osler TM, Vane DW, Tepas JJ, Rogers FB, Shackford S, Badger GJ. Do pediatric trauma centers have better survival rates than adult trauma centers? An examination of the National Pediatric Trauma Registry. J Trauma. 2001;50:96–101. doi: 10.1097/00005373-200101000-00017. [PubMed] [Cross Ref]
29. Pike I, Piedt S, Warda L, Yanchar N, Macarthur C, Babul S, Macpherson AK. Developing injury indicators for Canadian children and youth: a modified-Delphi approach. Inj Prev. 2010;16:154. doi: 10.1136/ip.2009.025007. [PubMed] [Cross Ref]
30. Rogers SO. The holy grail of surgical quality improvement: process measures or risk-adjusted outcomes? Am Surg. 2006;72:1046–1050. [PubMed]
31. Romanow R. Building on Values: The Future of Health Care in Canada (Final Report). Health Canada, November 2002. Available at: Accessed December 6, 2010.
32. Scanlon MC, Harris JM, 2nd, Levy F, Sedman A. Evaluation of the agency for healthcare research and quality pediatric quality indicators. Pediatrics. 2008;121:e1723. doi: 10.1542/peds.2007-3247. [PubMed] [Cross Ref]
33. Sedman A, Harris JM, 2nd, Schulz K, Schwalenstocker E, Remus D, Scanlon M, Bahl V. Relevance of the Agency for Healthcare Research and Quality patient safety indicators for children’s hospitals. Pediatrics. 2005;115:135–145. doi: 10.1542/peds.2004-1870. [PubMed] [Cross Ref]
34. Shaller D. Implementing and using quality measures for children’s health care: perspectives on the state of the practice. Pediatrics. 2004;113:217–227. doi: 10.1542/peds.113.3.e217. [PubMed] [Cross Ref]
35. Showstack J. Improving quality of care in orthopedic surgery. Arthritis Rheum. 2003;48:289–290. doi: 10.1002/art.10751. [PubMed] [Cross Ref]
36. Skaggs DL, Roy AK, Vitale MG, Pfiefer C, Baird G, Femino D, Kay RM. Quality of evaluation and management of children requiring timely orthopaedic surgery before admission to a tertiary pediatric facility. J Pediatr Orthop. 2002;22:265–267. doi: 10.1097/00004694-200203000-00027. [PubMed] [Cross Ref]
37. SMARTRISK. The Economic Burden of Injury in Canada. Toronto: SMARTRISK; 2009.
38. Steering Committee on Quality Improvement and Management and Committee on Practice and Ambulatory Medicine Pediatrics Principles for the development and use of quality measures. Pediatrics. 2008;121:411–418. doi: 10.1542/peds.2007-3281. [PubMed] [Cross Ref]
39. Tepas JJ. The National Pediatric Trauma Registry: a legacy of commitment to control of childhood injury. Semin Pediatr Surg. 2004;13:126–132. doi: 10.1053/j.sempedsurg.2004.01.009. [PubMed] [Cross Ref]
40. Welke KF, Diggs BS, Karamlou T, Ungerleider RM. Measurement of quality in pediatric cardiac surgery: understanding the threats to validity. ASAIO J. 2008;54:447–450. doi: 10.1097/MAT.0b013e318185daa3. [PubMed] [Cross Ref]

Articles from Clinical Orthopaedics and Related Research are provided here courtesy of The Association of Bone and Joint Surgeons