Results of this study suggested the feasibility and validity of the automatic note search strategy in identifying CCI comorbidities in the EMR. Our results indicate that the sensitivities of the automatic note search strategy were considerably better than ICD-9
codes to search for all the CCI variables except one, leukemia, for which the sensitivity of both search strategies was 100%. The specificity and the NPV of the automatic note search strategy were also equal or superior to the ICD-9
codes search in all the CCI comorbidities. In addition, our results confirmed the findings of previous studies on the reliability and accuracy of electronic search; for example, Alsara et al26
also reported that electronic query resulted in accurate and highly efficient data extraction.
The CCI is being widely used by health care researchers to predict short-term (30 days) and long-term (1 year) mortalitiy in ICU patients.27-30
To compare the meaningful differences in patients' outcomes, it is essential to balance the baseline comorbidity conditions. The CCI is one of the most commonly used tools to measure the baseline comorbidities before ICU admission. A recent study performed by Christensen et al31
discussed the important role of the CCI combined with administrative data in predicting short- and long-term mortality for ICU patients. Although D'Hoore et al32
described the CCI index as a resourceful way to perform risk adjustment from administrative databases, Poses et al4
reported enhanced discrimination of inpatient mortality using the CCI index. Currently, an ICD-9
code search is frequently used to automatically extract CCI comorbidities.10,33,34
However, the ICD-9
–coded administrative databases lack a clinical definition for diagnoses, causing variability in coding practices.35
Our results revealed that ICD-9
codes underreport the comorbidities that substantiate the finding of the previous studies.19,20,36,37
The underreporting could be attributable to extra emphasis on the procedures and complications on admission, compared with the comorbidities, for monetary reasons.20
Romano et al38
also found that the CCI comorbidities were not accurately defined in ICD-9
codes, which produced interobserver variations in ICD-9
codes assigned to the comorbidities. Although automatic searches using ICD-9
codes to identify comorbidities has been used in many research projects, the lack of accuracy in criteria used by the staff who code medical records may differ from physicians' criteria in diagnosing a medical condition, which significantly limits the broad use of this method. The automatic note search strategies were derived from the algorithm-incorporated keyword and program for a query within the specific note section. This approach enhanced the use of the patient database query and tremendously reduced the time when compared with the manual medical record review (mean time taken to manually review 1 patient note for the CCI comorbidities ranged from 5 to 10 minutes). The implementation of an electronic strategy to extract information is not only useful for research purposes but also may be helpful for the treatment of patients.39
Because an automated digital algorithm provides accurate information about a patient's comorbidities, it will help physicians to recognize comorbidity information early and might help in better treatment. Comorbidities act as a prognosticating factor for patient survival and treatment-related outcomes. Patients with higher CCI scores are at increased risk for readmissions and hospitalizations; thus, using automated digital algorithms to identify comorbidities early will certainly be an important factor and might well be used for early palliative consultations if needed in the future. The high sensitivity and specificity of the automated digital algorithm make it an important tool for physicians and investigators in accurately estimating comorbidities and might help in making early decisions and avoiding medical errors.
Another alternative search strategy to identify comorbidities is the Systematized Nomenclature of Medicine–CLINICAL Terms (SNOMED-CT). Although this method produced better performance than the ICD-9
code search strategy, there were also significant limitations for broad use in clinical research.40
Chiang et al41
suggested that SNOMED-CT coding is imperfect and unreliable and requires physician training and repeated testing. Furthermore, SNOMED-CT does not satisfactorily distinguish the exact terms at the clinical interface level for the study template at the current stage.42
Our search strategies also had certain limitations. First, performance of the automated digital algorithm and coding of the CCI is dependent on the quality of the database and consistency of the text entries, which limits the applicability of this approach to units with this database or one similar. However, the logic and the free text search concept could be generalized to other institutions; it provides potential for diffusion of the method at sites willing to replicate the programming effort because the medical documentation training is similar across the country. Because electronic clinical notes are becoming a standard feature of the modern era, our approach will become more generalizable. Second, we only focused on a pertinent section (medical and surgical history) of clinical notes to search for comorbidities, which might have caused us to miss some information provided in other note sections, although the same validation process can be extended to other sections of clinical notes because the concept remains the same. Third, the data can be missed because of errors or corruption in the data warehouse.43
However, this will only account for a small proportion of the database. Fourth, some of the CCI comorbidities definitions are outdated. Since the original CCI was developed in 1987, medicine has undergone a vast amount of change. Certain diseases, such as AIDS, no longer have the same relative risk of mortality as when the CCI was developed. Similarly, criterion for untreated thoracic and abdominal aneurysm 6 cm or larger for diagnosis of peripheral vascular disease needs reassessment. The latest guidelines advocate surgery when the aneurysm is 5.5 cm or larger.44
However, we could refine our search strategy to identify variables according to any new definitions by modifying the algorithm for new definitions. Finally, because of the retrospective nature of the study, we only included documented comorbidity in the definite diagnostic criteria.
In conclusion, CCI comorbidities can be correctly identified using the automated digital algorithm. The combination of good sensitivity, specificity, and easy calculation should encourage physicians to implement the automated digital algorithm in their clinical practice and medical research.