|Home | About | Journals | Submit | Contact Us | Français|
Improvement in outcomes after vaginal hysterectomy (VH) requires accurate identification of complications. We hypothesized that coded data, commonly used to determine morbidity, would miss more complications than chart review.
Medical records of women who underwent VH from January 2004 through December 2005 were reviewed for cardiac or respiratory arrest, congestive heart failure, pulmonary edema, pulmonary embolism, urinary tract infection, ureteral obstruction, hemorrhage, and delirium. Complications were identified with use of coded data, in which diagnoses were classified with a modification of the Hospital Adaptation of the International Classification of Diseases.
Records of 712 patients were reviewed. Of the 161 complications identified, 158 (158/161; 98.1%) were identified through chart review and 48 (48/161; 29.8%) through coded data. Codes captured all diagnoses of cardiac arrest, respiratory arrest, and pulmonary embolism but missed other complications.
Codes captured life-threatening complications, but other complications were underestimated or missed entirely. Reliance on coded data for outcome assessments can be misleading and should be combined with other methods to maximize validity.
Improvement of care among women undergoing vaginal hysterectomy (VH) requires an accurate assessment of postoperative complications. Although VH is associated with low morbidity (1), adverse events do occur that are potentially preventable. Perioperative complications are increasingly identified through administrative data, comprising information from discharge summaries; procedural codes; International Classification of Diseases, Ninth Revision (ICD-9) codes; and diagnostic codes from billing data. This approach is most applicable in large-volume, multicenter studies in which data abstraction through chart review would be cumbersome. However, previously published studies indicate that the use of administrative data leads to incorrect assumptions about actual clinical processes (2–4). Moreover, this underestimation of complications negatively affects the ability to adequately counsel patients about expected postoperative outcomes.
Since the measurement of outcomes from quality improvement processes must be reliable and accurate when prospective, real-time methods are used, we sought to compare the accuracy of administrative data with the accuracy of chart review in identifying perioperative complications. We hypothesized that chart review provides a more accurate outcome assessment of perioperative morbidity.
This study was a planned secondary analysis of data from women who underwent VH for a benign indication from January 2004 through December 2005. In the original cohort, women were excluded if they underwent any nonvaginal surgery (eg, diagnostic laparoscopy) or had a preoperative diagnosis of malignancy. After the Mayo Clinic Institutional Review Board approved the study, the complete medical records of adult women who underwent VH were reviewed for perioperative complications that occurred within 9 weeks after the VH.
Key components of the electronic medical record consisted of the preoperative consultation, operative notes, inpatient and anesthesia records, direct postoperative patient communications, and postoperative examination through 9 weeks after the operation. Preoperative and intraoperative variables affecting morbidity were reviewed. Specific complications of interest that were abstracted during the chart review included the following: cardiac or respiratory arrest, congestive heart failure, pulmonary edema, pulmonary embolism, urinary tract infection, ureteral obstruction, hemorrhage, and delirium.
For comparison, coded data from within 9 weeks after the VH were obtained from the institution’s Medical Index. The Medical Index is an institutional resource used to identify patients for epidemiologic and clinical research, statistical analysis, administrative reporting, and quality control. This resource is created by the coding and classification of diagnoses from a patient’s medical record with a modification of the Hospital Adaptation of the International Classification of Diseases, second edition (HICDA). These diagnoses are derived from clinical “problems” that are identified in the “Impression-Report-Plan” section of clinical notes and discharge summaries. The HICDA codes for the specific complications of interest were independently identified. The ICD-9 codes corresponding to these HICDA codes are summarized in Table 1. Comorbid conditions that were present on admission were excluded from postoperative diagnoses. Records identifying a complication through review of either charts or administrative data were abstracted again to verify clinical information.
Data were analyzed using SAS statistical software, version 9.0 (SAS Institute Inc, Cary, North Carolina). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for the coded classification of perioperative complications, with chart review serving as the criterion standard.
Of the 903 women who underwent VH from January 1, 2004, through December 31, 2005, a total of 736 met the inclusion criterion for this study (ie, they had an operation for a benign condition). Review of the medical documentation showed that 712 women had complete records from the index surgery through 9 weeks after the operation. Of the 161 complications identified, 158 (158/161; 98.1%) were identified through chart review and 48 (48/161; 29.8%) through administrative data.
Coded data identified all instances of cardiac arrest (n=1), respiratory arrest (n=1), and pulmonary embolism (n=2) that were identified by chart review, with a calculated sensitivity, specificity, PPV, and NPV of 100% (Table 2). However, coded classification did not capture other types of complications, with sensitivity ranging from 0% to 75%. For example, coded classifications were less likely to identify conditions associated with fluid overload, namely congestive heart failure (2 patients; sensitivity, 0%) and pulmonary edema (5 patients; sensitivity, 20%). In all instances of fluid overload, chart review indicated aggressive medical management or transfer to the intensive care unit for telemetry and diuretics. Postoperative hemorrhage was coded for 1) a postoperative decrease in the hemoglobin level of 3 g/dL or more from the preoperative laboratory values or 2) clinical symptoms resulting in transfusion (6 of 25 patients; sensitivity, 24%). Urinary tract infections were identified if patients had symptoms (dysuria, urgency, frequency, etc) with or without a urine culture associated with cystitis and received antibiotics, but urinary tract infections were administratively coded for only 24 of 108 patients (sensitivity, 22%).
Among women who underwent VH during the 2-year period of the present study, all life-threatening complications, such as cardiac arrest, respiratory arrest, and pulmonary embolism, were identified through coded classifications. As we hypothesized, however, other specific complications (eg, pulmonary edema, congestive heart failure, urinary tract infection, and postoperative hemorrhage) were underreported or missed entirely through review of administrative data.
Accurate reporting of complications is imperative for evaluating quality improvement efforts, documenting the maintenance of standard of care, and providing the basis for informed consent (5,6). In the gynecologic surgery literature, administrative data have routinely been used to generate morbidity data for all these purposes. Brown et al (7) used ICD-9 codes to determine that the overall morbidity in more than 200,000 women undergoing prolapse surgery was 16%. In addition, Myers and Steege (8) evaluated more than 100,000 women who underwent hysterectomy during a 7-year period and also found a 16% complication rate. Similarly, Sung et al (9) reported perioperative morbidity of 14% in 264,000 women undergoing urogynecologic surgery in 1,000 hospitals (from the Nationwide Inpatient Sample).
In the present study, for example, the risk of 1 of these specific complications was 6% as identified through administrative data, but it was 22% through chart review, a difference of 16%. Given that the morbidity outcomes in the present study were similar to those in the literature, studies that used administrative data could have underestimated perioperative morbidity, which may actually be as high as 32%. This could have a profound impact on surgical practice, since performance measures and quality improvement initiatives are intimately associated with morbidity outcomes.
The present study had specific limitations. Not all outcome measures could be assessed, and some important variables may have been excluded. Another limitation is that the extensive electronic medical record at Mayo Clinic is unique and allows for relatively easy access to a patient’s entire medical record. This infrastructure is conducive to a study of this type and magnitude, which would be considerably more challenging with a paper-based system. Similarly, there was complete medical record ascertainment for 712 women through the postoperative evaluation, including follow-up patient contact. The coding process did not capture diagnoses generated from telephone conversations with patients. This might contribute to the identification of more postoperative complications within a chart review, but there was no way to capture these complications exclusively through coded classifications.
Administrative data are increasingly being used to identify outcome data, whether for research, health care policy, or reimbursement, but their accuracy is variable. The process for generating ICD-9 codes allows for considerable variation between the clinical information and the medical diagnosis codes (10). This variation is partly due to the documentation provided in the charting (eg, how detailed the notes are and how medical issues are described and defined in the medical record) (11). Other factors accounting for ICD-9 code and chart review discrepancies are that codes are limited to diagnoses that affect duration of hospitalization and treatment, and a maximal number of codes can be generated (12).
Reliably and efficiently identifying outcomes is clearly an important focus for the future of health care. When ICD-9 codes solely are used to generate outcome data, consideration must be given to the validity of the information to identify all relevant conditions and complications. Although chart review may synergistically augment administrative data to increase identification of perioperative complications, chart review is subject to human error. Iezzoni and colleagues (13) explored the feasibility of using administrative data to identify postoperative complications among adult medical and surgical patients. With the use of administrative data in combination with demographic, clinical, and hospital characteristic variables, their logistic regression model had improved predictive ability (C statistic, 0.64–0.70) for predicting complications.
In the present study, the codes were assigned correctly in charts where the clinical scenario was well documented before dismissal, as reflected by the 99% to 100% specificity of the coding classification. However, codes may not have been assigned for specific complications owing to incomplete or unclear medical documentation. For example, excessive intraoperative blood loss requiring a transfusion would have been identified as hemorrhage by the reviewer, yet if hemorrhage was not listed in either the operative note or the dismissal summary, it would not have been coded as such. Detailed clinical documentation ultimately leads to accurate diagnosis code assignment, which falls on the shoulders of the individual physician. This is an excellent opportunity for quality improvement on a level that has been underemphasized.
Ideally, tracking perioperative complications would occur prospectively, paralleling the efforts of the National Surgical Quality Improvement Program (NSQIP) (14). In this way, data would be directly analyzed for trends in safety and quality within and between institutions. Furthermore, quality improvement measures could be evaluated within the same system, making the application functional and accurate.
The authors acknowledge Dr Rita Wang, who assisted with data abstraction and maintenance of the database.
Supported in part by research grant RA-30582 from the National Institutes of Health, U.S. Public Health Service.
Disclosure Information: Nothing to disclose.
Presented at the 29th Annual Scientific Meeting of the American Urogynecologic Society, Chicago, IL, September 2008.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.