|Home | About | Journals | Submit | Contact Us | Français|
Acute lung injury (ALI) is a serious postoperative complication with limited treatment options. A preoperative risk prediction model would assist both clinicians and scientists interested in ALI. The objective of this investigation was to develop a surgical lung injury prediction (SLIP) model to predict risk of postoperative ALI based on readily available preoperative risk factors.
This is secondary analysis of a prospective cohort investigation including adult patients undergoing high-risk surgery. Preoperative risk factors for postoperative ALI were identified and evaluated for inclusion in the SLIP model. Multivariate logistic regression was used to develop the model. Model performance was assessed with the area under the Receiver Operating Characteristics Curve and the Hosmer and Lemeshow Goodness-of-fit test.
Out of 4,366 patients, 113 (2.6%) developed early postoperative ALI. Predictors of postoperative ALI in multivariate analysis which were maintained in the final SLIP model included high-risk cardiac, vascular, and thoracic surgery, diabetes mellitus, chronic obstructive pulmonary disease, gastroesophageal reflux disease, and alcohol abuse. The SLIP score discriminated patients who developed early postoperative ALI from those who did not with an area under the Receiver Operating Characteristic Curve (95% CI) of 0.82 (0.78 – 0.86) and was well calibrated (Hosmer Lemeshow p = 0.55). Internal validation using 10-fold cross-validation noted minimal loss of diagnostic accuracy with a mean +/− standard deviation area under the Receiver Operating Characteristic Curve of 0.79 +/− 0.08.
Using readily available preoperative risk factors, we developed the SLIP scoring system to predict risk of developing early postoperative ALI.
Postoperative respiratory complications are important causes of perioperative morbidity and mortality.1–3 Postoperative acute lung injury (ALI) and its most severe form, the Acute Respiratory Distress Syndrome (ARDS), are particularly severe pulmonary complications with an estimated mortality exceeding 45% in certain surgical populations.4,5 ALI has recently been identified as the most common cause of postoperative respiratory failure.6 Although appropriate ventilator7 and fluid strategies8 may improve outcomes, morbidity and mortality remain unacceptably high.
Mechanistically, postoperative ALI results from injury to the alveolar epithelium and/or capillary endothelium with associated alterations in the innate immune system, 9 activation of the coagulation cascade,10 and generation of reactive oxygen species11. The resulting influx of alveolar fluid leads to a severe impairment in gas exchange with hypoxemia and respiratory failure requiring ventilatory support. The surgical insult and various perioperative health care delivery factors (e.g. ventilator management, fluid and transfusion strategies) may impact all three of the major pathways involved in ALI pathogenesis. Importantly, patient comorbidities, medications, and other pertinent exposures can potentially impact the host response to these perioperative events as well.
In contrast to the numerous investigations of ALI treatment, surprisingly little emphasis has been placed on the important issue of ALI prevention. This is largely because studies evaluating ALI are typically performed in the intensive care unit, enrolling patients with established lung injury who are beyond the therapeutic window of potential prevention or early treatment strategies. The perioperative period is an attractive alternative environment for studying ALI mechanisms and testing prevention and early treatment strategies as the timing of the intraoperative insults contributing to postoperative ALI are predictable. This allows enrollment of subjects before the development of ALI and before the major intraoperative insults which portend risk for ALI occur. However, we cannot currently predict who is at risk of this serious postoperative complication at the onset of the surgical procedure. This precludes the enrollment of appropriate at-risk populations into prevention studies and prevents the full and appropriate study of ALI mechanisms in a clinical setting. Furthermore, with an incidence estimated at 3%,6 testing prevention strategies in unselected patient populations is both inefficient and prohibitively expensive.
Previous studies using administrative data have identified certain demographic and surgical factors associated with high risk of postoperative respiratory failure, but neither the incidence of ALI nor specific risk factors for this most important postoperative respiratory complication were reported.1,12,13 Recent clinical studies attempted to identify patients at high risk of ALI in non-selected (medical and surgical) high risk patients,14,15 but no model exists for the preoperative prediction of postoperative ALI. The objective of this study was to develop a surgical lung injury prediction (SLIP) model for predicting risk of postoperative ALI/ARDS based on readily available preoperative patient and procedural factors.
Following approval by the Mayo Clinic, Rochester, Minnesota Institutional Review Board, a secondary analysis of a prospective cohort study was used to identify risk factors associated with the development of early postoperative ALI/ARDS. These variables were then used to construct a SLIP score to estimate risk of developing postoperative ALI/ARDS. The Standards for Reporting of Diagnostic Accuracy guidelines were used for reporting our study results.16
Study participants were identified from a previous prospectively collected database of consecutive patients undergoing elective surgery at the Mayo Clinic from November 2005 to August 2006. The objective of this initial investigation was to determine the incidence and survival of ALI associated postoperative respiratory failure and its association with intraoperative ventilator settings, specifically tidal volume. Details of the study population have been previously described.6 Briefly, participants were included if mechanically ventilated for > 3 hours during general anesthesia for the following procedures: (1) all cardiac and aortic vascular surgeries, (2) non-cardiac thoracic surgeries, including esophageal and pulmonary surgeries, (3) all major abdominal surgeries, including laparoscopic procedures (excluding appendectomies and other lower abdominal procedures such as hernia repairs) and laparoscopic gastric bypasses, (4) spine surgeries (performed by either orthopedic surgeons or neurosurgeons), (5) surgical procedures on the hips and knees, (6) cystectomies, (7) neurosurgical procedures (excluding ventriculoperitoneal shunts, stereotactic and peripheral nerve surgeries) and (8) head and neck surgeries. Patients were excluded if: (1) they denied permission to utilize their health information for research, (2) they were less than 18 years old, or (3) they had prevalent major risk factors for lung injury, respiratory failure or they previously required mechanical ventilation, including the following: (a) mechanically ventilated prior to surgery, (b) trauma, sepsis, aspiration, shock, acute congestive heart failure, idiopathic interstitial pneumonias with diffuse bilateral infiltrates on chest radiography, pneumonia or respiratory failure at any point before surgery, during the hospitalization associated with the surgical procedure of interest (c) underwent emergency surgery, (d) had previous high risk surgery during the study period (no patient was included more than once), (e) had a history of sleep apnea or neuromuscular disease requiring continuous positive airway pressure for postoperative respiratory failure or (f) required re-intubation or need for mechanical ventilation for re-operation. The exclusion of patients with prevalent risk factors for ALI was performed to limit confounding and to facilitate an evaluation of the association of interest (intraoperative tidal volume and ALI) in the initial investigation.
Preoperative baseline characteristics, comorbidities, and clinical variables were extracted from the electronic medical record. Potential ALI risk factors were identified a priori and included: procedural factors such as surgical specialty (cardiac, aortic vascular, thoracic, abdominal, orthopedic, neurologic, urologic, otolaryngology), current and past smoking,6,17,18 alcohol abuse,6,17,19–21, body mass index (BMI),22 recent chemotherapy (within 6 months of the surgical procedure),20,23 diabetes mellitus (DM),6,24–26 chronic obstructive pulmonary disease (COPD),6,27 restrictive lung disease,23,28,29 cirrhosis, gastroesophageal reflux disease (GERD), and use of amiodarone,30,31 statins,32 angiotensin-converting enzyme inhibitors33 or angiotensin II receptor blockers34. Smoking was defined as never, former, or active. Former smoking was defined as more than one year since the last tobacco use. Alcohol abuse was defined as any one of the following: 1) more than 14 alcohol-containing drinks per week (> 2 drinks per day),12 2) a score of 1 or more on the CAGE questionaire,35,36 or 3) presence of an alcohol-related medical diagnosis such as alcohol-related cirrhosis, alcohol-related pancreatitis, or alcohol withdrawal. The CAGE questionnaire is a simple 4-question test (have you ever felt the need to cut down on your drinking?; have people annoyed you by criticizing your drinking?; have you ever felt guilty about drinking?; have you ever had a drink first thing in the morning - eye-opener - to steady your nerves or to get rid of a hangover?) that has been recognized as a useful tool for alcohol abuse screening.35,36 BMI was calculated from the most recent height and weight documented prior to the surgical procedure. If there was no documented weight in the 6 months prior to the surgical procedure, BMI was not calculated. Chemotherapy was considered present if administered at any time during the 6-month interval preceding surgery. We included both type 1 and type 2 diabetes mellitus in the definition. A history of gestational diabetes mellitus was not sufficient. COPD was defined as a history of emphysema or chronic bronchitis. Both interstitial lung diseases and extrinsic disorders such spinal deformity and morbid obesity were included in the definition of restrictive lung disease. For the later category (extrinsic disorders), formal documentation of “restrictive lung disease” in the medical record and/or confirmation with pulmonary function tests was required. Cirrhosis was defined as the documentation of “end-stage liver disease” or “cirrhosis” in the electronic medical record. GERD was defined as the documentation of “GERD,” “gastroesophageal reflux,” “esophageal reflux,” or “heartburn” in the electronic medical record. Due to a large proportion of patients being referred from outside facilities and the retrospective nature of this investigation, documentation of formal diagnostic criteria for the comorbidities of interest was not required. Rather, comorbidities (DM, COPD, restrictive lung disease, cirrhosis, GERD) were considered present if a physician documented the diagnosis in the electronic medical record prior to the surgical procedure. Medications (amiodarone, statins, angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers) were considered present if documented in the list of current medications for the patient at the time of hospital admission. The majority of these risk factors were chosen because each had been previously noted to have an association with ALI. GERD and cirrhosis were included in light of our preliminary data suggesting an association with early postoperative ALI.
Data were collected from two primary sources. The first source was the original database from which our population was identified.6 These data were collected prospectively by a single research coordinator who was blinded to the assessment of outcomes. The following variables were identified using this data set: age at surgery, sex, BMI, surgical specialty, smoking status, and alcohol history. Due to the inconsistent nature of reporting social history, a second patient-provided data source was interrogated as well. This data source arises from a questionnaire administered to all patients receiving care at Mayo Clinic in Rochester, MN. In addition to questions regarding medical history, social history is evaluated with multiple questions specifically relating to smoking and alcohol status. Review of the responses to these questionnaires was performed by a single physician investigator (DJK). When discrepancies were identified between the two data sources, the prospectively collected database determined the allocation of smoking and alcohol status. Study participants with no documentation regarding smoking (n = 85; 1.94%) and/or alcohol status (n = 38; 0.87%) were left classified as “missing” and were not included in any of the analyses evaluating the associations between smoking and alcohol with early postoperative ALI.
The second source of data was the patient’s electronic medical record. These data were collected using a web-based query tool. This tool is used to extract pertinent data with high-reliability from various source databases which contain clinical notes, laboratory tests, diagnostic findings, and related clinical information. The variables extracted using this technique included DM, COPD, restrictive lung disease, GERD, cirrhosis, recent chemotherapy, amiodarone, statins, angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers. For comorbidities, all clinical notes in the 5-year interval before surgery were evaluated for the diagnoses of interest. For medications, all clinical notes in the 3-month interval (6-months for chemotherapy) before surgery were interrogated. When medications were identified with the query tool, the medical record was manually reviewed to confirm active administration of the medication at the time of surgery. All electronic queries were performed by one of two physician investigators (DJK or AA).
To ensure the validity of the data obtained using this web-based tool, we compared the performance of the automated data extraction strategies to manual data extraction in a subset of 249 patients. Using Landis and Koch Cohen kappa statistic magnitude guidelines,37 agreement between manual and automatic electronic data collection strategies were almost perfect for 4 variables (COPD, restrictive lung disease, DM, cirrhosis), substantial for 1 (statin therapy), and moderate for 4 (GERD, chemotherapy, immunosuppressive therapy, and amiodarone). To better define the validity of the web-based queries, we also performed an independent exhaustive review of the medical record for the 249 patient subgroup. This evaluation served as a “gold standard” for comparing the initial manual data extraction with automated data collection. In this evaluation, automated data extraction strategies were noted to have greater sensitivity than manual data extraction for all variables evaluated except statin therapy. The specificities were uniformly high for all variables with both data extraction strategies. The web-based data collection strategies for angiotensin-converting enzyme inhibitors and angiotensin II receptor blockers had been previously validated in random sample of this study population (n = 1629 and 690 patients, respectively). The sensitivities for the electronic searches were noted to be 97.6% and 94.2% with specificities of 92.4% and 98.7%, respectively.
Details regarding the surgical procedure were manually extracted from the electronic medical record of all patients by two physician investigators (DJK and AA). Hypothesizing that specific procedural characteristics would influence the frequency of ALI within high-risk specialties such as cardiac, vascular, and thoracic surgery, subgroup analyses evaluating surgical details in these three populations were planned a priori. Specifically, we evaluated the frequency of ALI in patients undergoing major categories of cardiac procedures including coronary artery bypass grafting, valve replacement, valve repair, pericardial resection, ascending aortic/aortic arch repair, atrial septal defect/ventricular septal defect repair, myectomy, and other less invasive procedures such as sternal wound revision and pacemaker lead extraction. In a similar fashion, aortic vascular surgery was stratified into descending thoracic or thoracoabdominal aortic repairs and abdominal aortic repairs. Further consideration was given to an open versus endovascular approach. Thoracic surgery was stratified into video-assisted thoracoscopic surgery, fundoplication procedures, open lung biopsy, lung resection involving wedge resection or segmentectomy, lung resection involving multiple segments or lobectomy, lung resection involving multiple lobe resections, pneumonectomy, esophagectomy, lung decortication, and other miscellaneous procedures. Finally, the impact of revision surgery versus an initial/primary surgical procedure was considered as well. Procedures were then grouped into low- or high-risk categories to more accurately characterize the procedure-related risk of ALI/ARDS. Effect estimates were used to assist in establishing these categories, but statistical significance was not required due to the limited number of ALI outcomes in many of the subgroups analyzed. The classification of procedures into low- and high-risk for ALI/ARDS are presented in Table 1.
ALI/ARDS was defined according to the standard American-European consensus conference definition as the development of acute, bilateral pulmonary infiltrates and hypoxemia (PaO2/FIO2 <300 - ALI, PaO2/FIO2 <200 - ARDS) in the absence of clinical signs of left atrial hypertension as the main explanation for pulmonary edema.38 Postoperative ALI/ARDS was defined as occurring within the first 5 postoperative days. The outcome assessment was restricted to this time interval because ALI occurring after this period is unlikely to have resulted from insults encountered during the surgical course.21 In addition, we believed the inclusion of these later cases would negatively impact the performance of the predictive model. Importantly, our recent work6 and the work of Licker et al.21 both confirm that the vast majority of postoperative ALI cases occur within this early window. The outcome assessment was performed during the initial prospective investigation by an investigator who underwent a structured ALI/ARDS tutorial prior to reviewing patient’s medical records.6
Assuming an ALI incidence of 3% from our preliminary data, a sample size of 4,000 patients was calculated to allow us to determine the sensitivity, specificity, positive and negative predictive values of the model to predict ALI within 95% confidence intervals of approximately +/− 0.07. Dichotomous variables are presented as counts with percentages. Continuous data are presented as median with 25%–75% interquartile ranges. For univariate analyses, comparisons between the two groups were performed with a Pearson's χ2 test or Fisher’s exact test as appropriate for categorical variables. Continuous variables were tested with the Mann-Whitney rank-sum test.
The primary analysis consisted of determining the predictive validity of the SLIP model. Model derivation began with univariate analyses evaluating the associations between each risk factor and postoperative ALI/ARDS. Variables associated with postoperative ALI/ARDS (p ≤ 0.1) in these univariate analyses were included in a multivariate logistic regression model. Variables with biologic plausibility and multiple existing reports suggesting a strong relationship with the development of postoperative ALI/ARDS were also considered for inclusion in the initial multivariate model, irrespective of their statistical association with ALI/ARDS. Risk factors with significant associations with postoperative ALI in the initial multivariate analysis (p ≤ 0.05) were included in a second and final multivariate model and were assigned SLIP points. Vascular, cardiac, and thoracic procedures were classified as low- and high-risk based on univariate analysis as described above (Methods section, paragraph beginning, “Details regarding the surgical procedure….”) prior to inclusion of procedural characteristics into the model. SLIP points were then assigned to each predictor in the final model by multiplying the predictor’s parameter estimate by a factor of 10 and rounding to the nearest integer. Recognizing that a combination of low exposure frequency and a moderate number of ALI/ARDS outcomes can reduce the likelihood of identifying significant associations with ALI/ARDS, a second model was also created including variables with existing literature supporting an association with ALI/ARDS and moderate or large effect estimates (odds ratio ≥ 1.5) in the initial multivariate analysis (irrespective of the presence of statistical significance). These procedures (inclusion of variables with biologic plausibility and moderate to large effect estimates, despite a lack of statistically significant association) were performed to improve the external validity and replicability of the scoring system.
Model discrimination was assessed by calculating the area under the Receiver Operating Characteristic Curve (AUC). Model calibration was assessed using the Hosmer and Lemeshow Goodness-of-Fit test statistic. The threshold score which maximizes the Youden index [sensitivity + (specificity − 1)]39 was determined and the corresponding positive and negative predictive values, positive and negative likelihood ratios, and their 95% confidence intervals at this optimal cut-off were calculated. To improve the functionality of the prediction model, a sensitivity analysis was performed to determine the model performance at two additional cut off points. The first point was chosen to assist in identifying patients at risk of ALI with greater sensitivity and to improve the negative likelihood ratio. The second point was chosen to assist with the identification of patients at high risk of ALI with greater specificity and to improve the positive likelihood ratio. Using these cut points, we also developed three stratum of risk for postoperative acute lung injury: low, intermediate, and high.
Due to the absence of an external data set for model validation, a 10-fold cross-validation procedure was performed. The AUC was estimated in 10 test samples to provide an estimate of what the AUC would be if the model were used for prediction in an independent external validation sample. The AUC was estimated in place of the misclassification error rate as the anticipated future use of the model is as a prediction score rather than a dichotomous classification. The SAS statistical software (SAS® 9.1 For Windows, SAS Institute Inc., Cary, NC) Proc Logistic, was used with a backwards elimination variable selection procedure in the 10-fold cross-validation. All variables with statistically significant associations with ALI in univariate analyses were considered potential predictors, but only those that had a p-value < 0.05 were retained in the final 10 learning models. Thus, 10 learning models were developed and applied to the 10-test samples with the AUC calculated in each test sample. This was repeated 50 times giving 500 estimates of the AUC in samples that were not used to fit the model.
Between November 2005 and August 2006, 4366 patients undergoing high-risk surgery were identified for inclusion in this study. Fifty-four patients were excluded from the initial study of Fernandex-Perez et al (n=4420).6 Thirty-two patients had rescinded research authorization for the use of their medical record in the interval between the initial study and the present investigation. Twenty-two patients were excluded due to a duration of anesthesia less than 3 hours (a pre-defined exclusion criteria). One-hundred thirteen (2.6%; 95% CI = 2.2% – 3.1%) patients developed early postoperative ALI, of whom fifty-five met criteria for ARDS. Mortality was significantly higher among those who developed ALI/ARDS when compared to those without ALI/ARDS (14.2% versus 1.2%; OR 13.5; 95% CI = 7.48 – 24.7; p < 0.01) as was the median length of hospital stay [11 days (25%–75% interquartile range = 7 – 20 days) versus 5 days (25%–75% interquartile range = 4 – 8 days; p < 0.01)]. Baseline characteristics and ALI predictors are presented in Table 2.
Numerous baseline characteristics and ALI predisposing conditions differed in univariate analyses among those who did and those who did not develop early postoperative ALI/ARDS (Table 2). Specifically, patients who developed ALI/ARDS were older and more likely to undergo high-risk cardiac, vascular, or thoracic surgery. In contrast, abdominal, spine, orthopedic (hip and knee replacements), and neurologic procedures were associated with a lower incidence of early postoperative ALI/ARDS. The frequencies of ALI/ARDS by surgical procedure are listed in Figure 1. High-risk vascular surgery was associated with the greatest risk of ALI/ARDS (22%; 95% CI = 11% – 41%) while spine surgery was associated with a much lower frequency of postoperative ALI/ARDS (0.5%; 95% CI = 0.1% – 1.6%). Patient’s who developed ALI were also more likely to have diagnoses of DM, COPD, restrictive lung disease, and GERD. Smoking status, alcohol abuse, recent chemotherapy, and pre-operative amiodarone and/or statin therapy were also significant predictors of postoperative ALI/ARDS. Cirrhosis, BMI and angiotensin-converting enzyme inhibitors/angiotensin II receptor blocker therapy were not significantly associated with ALI/ARDS.
SLIP points were determined based on the coefficients from multivariate logistic regression analysis as described above (Methods section, paragraph beginning, “Risk factors with significant associations ….”). The logistic regression coefficients and odds ratios for ALI risk factors included in the initial multivariate analysis are shown in table 3 and the logistic regression coefficients and corresponding SLIP points for predictors included in the final scoring system are shown in table 4. Age, sex, low-risk vascular surgery, restrictive lung disease, tobacco use, recent chemotherapy in patients undergoing lung or esophageal resection for cancer, and preoperative amiodarone and/or statin therapy were not assigned SLIP points in the final model as there was insufficient evidence (p > 0.05) for an association with early postoperative ALI/ARDS in the initial multivariate logistic regression analysis. Age was evaluated in the multivariate model as a continuous variable and then separately as a dichotomous variable with a cut-off of 75 years of age. It was not associated with postoperative ALI in either model. Low-risk vascular surgery, restrictive lung disease, recent chemotherapy in patients undergoing lung or esophageal resection for cancer, and active smoking had moderate to large effect estimates (OR ≥ 1.5) and existing literature suggesting an association with postoperative ALI. Therefore, these variables were included in the secondary SLIP model (See Table 1, Supplemental Digital Content 1, which is a table showing the variables and associated SLIP points included in the secondary SLIP model).
SLIP scores ranged from 0 to 60 (median 7). The SLIP model discriminated patients who developed early postoperative ALI/ARDS from those who did not with an AUC (95% CI) of 0.82 (0.78 – 0.86; Figure 2). The model was well calibrated with a Hosmer and Lemeshow Goodness-of-Fit Chi-Square value of 4.95 (p = 0.55). In the internal validation procedure, the mean +/− standard deviation AUC in the 500 learning and test samples were 0.82 +/− 0.01 and 0.79 +/− 0.08, respectively. Receiver Operating Characteristic Curve analysis determined the optimal cut-off for maximizing the Youden index to be 22. At this level, the positive likelihood ratio (95% CI) was determined to be 3.81 (3.34, 4.34) with a negative likelihood ratio (95% CI) of 0.34 (0.25, 0.46). The associated sensitivity (95% CI) and specificity (95% CI) were 72% (63% to 80%) and 81% (80% to 82%), respectively. The sensitivity analysis evaluating the sensitivity, specificity, positive and negative predictive value and positive and negative likelihood ratios at the two additional SLIP cut points are shown in Table 5.
Using the two SLIP cut points identified in the sensitivity analysis, we defined three groups of patients: low risk (SLIP score < 10), moderate risk (SLIP score 10 to 26), and high risk for postoperative ALI/ARDS (SLIP score ≥ 27). This resulted in the assignment of 51.3% of patients to the low-risk group with an associated frequency of ALI of 0.54%, 37.9% of patients to the moderate-risk group with an associated frequency of ALI of 2.62%, and 10.8% of patients to the high-risk group with an associated ALI frequency of 12.2%. Figure 3 displays the frequency of ALI/ARDS by stratum of SLIP score.
In this single-center, retrospective cohort evaluation, we developed a scoring system for predicting risk of early postoperative ALI/ARDS based solely on preoperative patient characteristics and procedural factors. Using these readily available data, the SLIP score identified patients at risk of early postoperative ALI before undergoing their surgical procedure. Importantly, identification of patients who are at high risk for postoperative ALI prior to their surgical procedure may afford an opportunity for the implementation of timely interventions to prevent this complication. Moreover, it could facilitate the enrollment of participants into meaningful mechanistic, prevention, and early treatment trials.
Although a number of risk prediction models for postoperative respiratory failure have been described, all have important limitations when attempting to determine risk of ALI prior to a surgical procedure. Most are not specific to ALI/ARDS1,12,13,40,41 and those looking at ALI/ARDS are isolated to specific surgical populations such as lung resection surgery5,21,42–46 or cardiopulmonary bypass47–49 (Table 6). In addition, most have emphasized the importance of a variety of intra-operative and postoperative variables. Our prediction model is specifically restricted to factors identifiable preoperatively to identify those who are at high risk for postoperative ALI before their surgical procedure.
Existing literature supports associations between high-risk surgical procedures such as cardiac, vascular, and thoracic surgery6,15,21,43 as well as alcohol abuse6,17,19–21 with ALI/ARDS. In addition to these previously identified associations, we have also confirmed and/or identified a number of less well-described predictors of postoperative ALI. Specifically, we noted COPD, DM and GERD to be associated with development of early postoperative ALI. These associations remained significant following multivariate adjustment (Table 3). Although previous investigations have recognized COPD as a risk factor for ALI following lung resection surgery,27 its association with ALI in more diverse surgical populations is a novel finding. In contrast to much of the published literature,6,24–26 we also noted DM to be associated with risk for ALI. Importantly, the evidence suggesting a protective role for DM in patients at risk of ALI appears most significant in the setting of sepsis.20,26,50 Moreover, a suggestion of increased risk of early postoperative ALI in patients with DM has been previously described.21 To our knowledge, the association between GERD and postoperative ALI has not been well described. While we hypothesize this increased risk may be the result perioperative microaspiration, our study was not designed to address this issue and thus cannot definitively characterize this association.
In contrast, we did not confirm previous reports of associations between active smoking,6,17,18 recent chemotherapy in patients undergoing lung resection or esophagectomy for cancer,20,23 restrictive lung disease23,28,29 or amiodarone30,31 with ALI. Considering the relatively infrequent occurrence of these variables (1.5% – 13%), we believe this is largely the result of inadequate power. As an example, 567 patients were actively smoking at the time of their surgical procedure. To identify an odds ratio of 1.5 when assessing the association between active smoking and postoperative ALI (assuming a frequency of ALI of 2.6% in the non-smoking group), a sample of 1,661 active smokers and 12,783 non-smokers would be required to obtain a beta level of 0.80 (two-sided alpha of 0.05). Recognizing the potential importance of these previously identified risk factors, a second SLIP model was generated (See Table 1, Supplemental Digital Content 1, which is a table showing the variables and associated SLIP points included in the secondary SLIP model). The performance of this secondary model was similar to the more parsimonious primary model with an AUC (95% CI) of 0.82 (0.79 – 0.86) and a Hosmer and Lemeshow Goodness-of-Fit Chi-Square value of 3.63 (p = 0.73). Regarding statins, our univariate evaluation suggested risk for early postoperative ALI with statin therapy. However, this association was lost when adjusting for additional, potentially confounding variables (Table 3). Currently, the evidence addressing this potential association is conflicting.32,51
Although the modest overall performance of the SLIP score may limit its usefulness in clinical practice, its potential utility in identifying high-risk patients for participation in prospective investigations of ALI/ARDS prevention persists, particularly if the higher cuttoff value is chosen. Moreover, it may still be clinically useful for future low-cost, low-risk ALI/ARDS prevention interventions. The addition of intraoperative and early postoperative variables such as transfusion, ventilator management, and duration of cardiopulmonary bypass or aortic cross clamp placement would likely increase the accuracy of the prediction model. However, the primary aim of the proposed scoring system is to identify patients at risk of ALI before their surgical procedure for potential inclusion in prospective mechanistic, prevention, and early treatment trials. By identifying patients at higher risk, the SLIP score can greatly enhance the feasibility of such investigations. As an example, the sample size requirements for a clinical trial of postoperative ALI/ARDS prevention for an effective intervention that was shown in preclinical studies to halve the risk of ALI/ARDS development is much lower when utilizing the SLIP score at a cut off of 28 points (830 total, 415 per group) than it would be without the SLIP score (3662 total, 1831 per group).
In addition to the usual limitations of a retrospective study such as the potential for bias and confounding, this study has other important limitations as well. The limited number of ALI/ARDS cases (n = 113) and the low frequency of some potentially important risk factors may have masked important associations with ALI/ARDS. Specific examples include chemotherapy, restrictive lung disease, amiodarone, and smoking. We attempted to mitigate this possibility by performing a secondary analysis including factors with existing evidence for an association with postoperative ALI/ARDS and moderate to large effect estimates in the present study (odds ratios ≥ 1.5), despite a lack of significance in the initial multivariate model. It should be emphasized that additional predictors of ALI may still have been missed despite this secondary analysis.
An additional limitation is the heterogeneity of the study population. This heterogeneity is at least partially responsible for the suboptimal sensitivity (72%; 95% CI 63% – 80%) and specificity (81%; 80% – 82%) at the optimal cut point of the SLIP model. Improved model performance may be seen in more homogeneous surgical populations or with more specific characterization of the surgical procedures. Indeed, previous investigations have focused on specific surgical specialties such as cardiac47–49 or thoracic surgery4,5,18,21,27,44,46. However, restricting the study to a specific surgical population would necessarily preclude the screening of high-risk patients undergoing other types of surgery. As it is important to identify high-risk for ALI in these surgical populations as well, we aimed to develop a risk scoring system that would be more broadly applicable. We further recognize that the discriminative power of the SLIP score might be improved with additional variables and more complex modeling strategies. However, our primary aim was to create a model that could be used for the preoperative identification of high-risk participants for future ALI prevention and mechanistic studies. Well-designed investigations in these time-sensitive studies require very efficient risk prediction strategies. As complex scoring systems entail time-consuming calculations, we elected to focus on creating a more parsimonious risk prediction model. In addition, the utility of increasing the granularity of procedural detail beyond what has been described is unclear and would not appear to be supported with our moderate number of ALI/ARDS outcomes. We also acknowledge the selection bias which results from our exclusion of patients undergoing lower risk procedures lasting less than three hours. As a result of their exclusion, we cannot be sure that our model will generalize to these low risk populations.
A third limitation is our lack of consideration for intraoperative and postoperative variables which may be associated with risk for postoperative ALI. Factors such as infection,52 ventilator management,43,53 fluid21,54 and transfusion strategies55, and choice of volatile anesthetic,56 may potentially impact the development and/or progression of ALI in the postoperative period. While the addition of such variables would be expected to improve the overall performance of our prediction model, their inclusion would preclude the use of this model as intended (preoperative risk assessment), prohibiting the identification of patients at high-risk of ALI prior to the surgical procedure. In turn, we believe this may delay the recognition of patient risk and could lead to missed opportunities for ALI prevention strategies and adequately detailed studies of ALI mechanism. Another limitation of this study is the single-center tertiary care nature of the institution providing care to the study population. This raises concern for both referral and institution-specific bias as well as overall generalizability. The final and most important limitation of this investigation is the lack of a validation cohort. To address this limitation, we performed an internal validation of the SLIP model using the 10-fold cross-validation technique. Only slight shrinkage in model performance was noted in this validation procedure with a mean +/− standard deviation AUC of 0.79 +/− 0.08.
In conclusion, we have developed a SLIP score to predict risk of early postoperative ALI/ARDS before the operative procedure. If validated in an external data set, this score will assist clinicians in estimating surgical patient’s risk of early postoperative ALI/ARDS. Moreover, by identifying patients who are at high-risk of ALI prior to their surgical procedure, it may facilitate the performance of prospective investigations of postoperative ALI pathogenesis, prevention, and early treatment.
Summary Statement: We aimed to develop a preoperative prediction model for identifying risk of postoperative acute lung injury. To obtain this aim, we developed a surgical lung injury prediction model using readily available preoperative data.
This work was supported by Grant Number [KL2 RR024151] from the National Center for Research Resources (NCRR), a component of the National Institutes of Health (NIH), and the NIH Roadmap for Medical Research.
National Center for Research Resources, National Institutes of Health, Department of Health and Human Services, 6701 Democracy Boulevard MSC 4874, Bethesda MD, 20892-4874
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Department and Institution to which the work should be attributed: Department of Anesthesiology, Mayo Clinic, Rochester, MN.