Several agencies, including the Agency for Healthcare Research and Quality, have argued that indicators needed to be developed for specific populations, including paediatrics.28
We concluded that a robust TT, validated for paediatric use, was necessary for evaluating AEs and conducting a national paediatric study in Canada. Unlike the IHI-GTT where individual modules were validated separately, the CPTT has been validated for use as a single tool for all paediatric populations.
Similar to the other national studies of AEs, use of our tool calls for a two-stage review process: first, nurses use the trigger tool to identify charts likely to have an AE, followed by physicians who review the triggered charts for AEs. However, in order to establish the sensitivity of the nurse review, all charts in this study were reviewed independently for triggers (by nurses) and for AEs (by physicians and nurses). This process is the first of its kind to establish a false-negative rate for triggered charts, by identifying patients with AEs whose charts were not triggered. The number of false-negative charts may be reduced through education highlighting use of the ‘other’ trigger by the nurses. Nurse and physician review of all charts for AEs also demonstrated differences between these groups in designating AEs. Nurses and physicians assessed the charts using somewhat different methods. Nurses reviewed charts using the CPTT and assessed all charts in terms of the NCC-MERP Harm Scale and the presence of AEs,29
whereas physicians reviewed all charts for AEs only. These different methods mean their results are not strictly comparable. However, the physicians read each case from admission to discharge, and based on these findings, it appears that physicians may be better qualified to assess the presence of AEs. This difference between nurse and physician assessment of AEs is consistent with previous findings,30 31
but these findings should be explored further.
There are inherent limitations in TT methodology. The utility of any chart review is subject to the quality of the documentation.32
Like other screening tests, TTs have a high sensitivity and relatively low specificity. However, their purpose is to cull charts unlikely to have an AE, leaving behind for physician review only those charts with a high probability of having an AE. The IRR of identifying AEs through chart review is variable; previous studies reported κ scores ranging from −0.077 to 0.66.1–9 16 19 32–35
Combining ratings from more than one physician19 36
and monitoring reviewer performance during the study with personal feedback may improve IRR.30
However, a recent report suggests that a review process involving two physicians per chart is no more reliable than only one in detecting AEs.36
Additional training that includes a consensus approach on the number of AEs in the training charts has been used successfully.35
Finally, previous national studies of AEs have focused on those characterised by significant harm, that is NCC-MERP categories F–I, underestimating the total burden of harm by disregarding events with temporary harm requiring intervention only (category E events). Although achieving high IRR for category E can be challenging,35
capturing these events in future studies will result in a more comprehensive view of the nature of iatrogenic injury, and permit comparisons with recent studies of AEs in children and youth that have used the expanded definition.17 18
Despite these limitations, the use of a broad paediatric population sample to validate the CPTT permits its use in other acute care paediatric institutions. With its final refinements, the validated CPTT offers the first comprehensive evidence-based tool for assessing harm in hospitalised children and youth.