Although the area under the receiver operating characteristic (ROC) curve (AUC) is the most popular measure of the performance of prediction models, it has limitations, especially when it is used to evaluate the added discrimination of a new risk marker in an existing risk model. Pencina et al. (2008) proposed two indices, the net reclassification improvement (NRI) and integrated discrimination improvement (IDI), to supplement the improvement in the AUC (IAUC). Their NRI and IDI are based on binary outcomes in case-control settings, which do not involve time-to-event outcome. However, many disease outcomes are time-dependent and the onset time can be censored. Measuring discrimination potential of a prognostic marker without considering time to event can lead to biased estimates. In this paper, we extended the NRI and IDI to time-to-event settings and derived the corresponding sample estimators and asymptotic tests. Simulation studies showed that the time-dependent NRI and IDI have better performance than Pencina’s NRI and IDI for measuring the improved discriminatory power of a new risk marker in prognostic survival models.
Improved discrimination; Prognostic survival models; Time-dependent NRI; Time-dependent IDI
Concerns have been raised about the use of traditional measures of model fit in evaluating risk prediction models for clinical use, and reclassification tables have been suggested as an alternative means of assessing the clinical utility of a model. Several measures based on the table have been proposed, including the reclassification calibration (RC) statistic, the net reclassification improvement (NRI), and the integrated discrimination improvement (IDI), but the performance of these in practical settings has not been fully examined. We used simulations to estimate the type I error and power for these statistics in a number of scenarios, as well as the impact of the number and type of categories, when adding a new marker to an established or reference model. The type I error was found to be reasonable in most settings, and power was highest for the IDI, which was similar to the test of association. The relative power of the RC statistic, a test of calibration, and the NRI, a test of discrimination, varied depending on the model assumptions. These tools provide unique but complementary information.
Calibration; Discrimination; Model accuracy; Prediction; Reclassification
The performance of prediction models can be assessed using a variety of different methods and metrics. Traditional measures for binary and survival outcomes include the Brier score to indicate overall model performance, the concordance (or c) statistic for discriminative ability (or area under the receiver operating characteristic (ROC) curve), and goodness-of-fit statistics for calibration.
Several new measures have recently been proposed that can be seen as refinements of discrimination measures, including variants of the c statistic for survival, reclassification tables, net reclassification improvement (NRI), and integrated discrimination improvement (IDI). Moreover, decision–analytic measures have been proposed, including decision curves to plot the net benefit achieved by making decisions based on model predictions.
We aimed to define the role of these relatively novel approaches in the evaluation of the performance of prediction models. For illustration we present a case study of predicting the presence of residual tumor versus benign tissue in patients with testicular cancer (n=544 for model development, n=273 for external validation).
We suggest that reporting discrimination and calibration will always be important for a prediction model. Decision-analytic measures should be reported if the predictive model is to be used for making clinical decisions. Other measures of performance may be warranted in specific applications, such as reclassification metrics to gain insight into the value of adding a novel predictor to an established model.
Models for risk prediction are widely used in clinical practice to risk stratify and assign treatment strategies. The contribution of new biomarkers has largely been based on the area under the receiver operating characteristic curve, but this measure can be insensitive to important changes in absolute risk. Methods based on risk stratification have recently been proposed to compare predictive models. These include the reclassification calibration statistic, the net reclassification improvement (NRI), and the integrated discrimination improvement (IDI). This work demonstrates the use of reclassification measures, and illustrates their performance for well-known cardiovascular risk predictors in a cohort of women. These measures are targeted at evaluating the potential of new models and markers to change risk strata and alter treatment decisions.
Patients with advanced peripheral artery disease (PAD) have a high prevalence of cardiovascular (CV) risk factors and shortened life expectancy. However, CV risk factors poorly predict midterm (<5 years) mortality in this population. This study was designed to test the hypothesis that baseline biochemical parameters would add clinically meaningful predictive information in patients undergoing lower extremity bypass.
This was a prospective cohort study of subjects with clinically advanced PAD undergoing lower extremity bypass surgery. The Cox proportional hazard was used to assess the main outcome of all-cause mortality. A clinical model was constructed with known cardiovascular risk factors and the incremental value of the addition of clinical chemistry, lipid, and a panel of 11 inflammatory parameters were investigated using c-statistic, the integrated discrimination improvement (IDI) index and Akaike information criterion (AIC).
225 subjects were followed for a median 893 days; IQR 539–1315 days). In this study 50 (22.22%) subjects died during the follow-up period. By life table analysis (expressed as percent surviving ± standard error), survival at 1, 2, 3, 4, and 5 years respectively was 90.5 ± 1.9%, 83.4 ± 2.5%, 77.5 ± 3.1%, 71.0 ± 3.8%, and 65.3 ± 6.5%. Compared with survivors, decedents were older, diabetic, had extant CAD, and were more likely to present with CLI as their indication for bypass surgery, P<.05. After adjustment for the above, clinical chemistry and inflammatory parameters significant for all cause mortality were albumin, HR .43 (95% CI .26–.71); P=.001, estimated glomerular filtration rate (eGFR), HR .98 (95% CI .97–.99), P=.023, high sensitivity C-reactive protein (hsCRP), HR 3.21 (95% CI 1.21–8.55), P=.019, and soluble vascular cell adhesion molecule (sVCAM), HR 1.74 (1.04–2.91), P=.034. Of all inflammatory molecules investigated, hsCRP proved most robust and representative of the integrated inflammatory response. Albumin, eGFR, and hsCRP improved the c-statistic and IDI beyond that of the clinical model and produced a final c-statistic of .82.
A risk prediction model including traditional risk factors and parameters of inflammation, renal function and nutrition had excellent discriminatory ability in predicting all cause mortality in patients with clinically advanced PAD undergoing bypass surgery.
Hospitalized heart failure (HF) patients are at high risk for death and readmission. We examined the incremental value of data obtained 1 week after HF hospital discharge in predicting mortality and readmission.
Methods and results
In the Efficacy of Vasopressin Antagonism in Heart Failure Outcome Study with tolvaptan, 1528 hospitalized patients (ejection fraction ≤40%) with a physical examination, laboratories, and health status [Kansas City Cardiomyopathy Questionnaire (KCCQ)] assessments 1 week after discharge were included. The ability to predict 1 year cardiovascular rehospitalization and mortality was assessed with Cox models, c-statistics, and the integrated discrimination improvement (IDI). Not using a beta-blocker, rales, pedal oedema, hyponatraemia, lower creatinine clearance, higher brain natriuretic peptide, and worse health status were independent risk factors for rehospitalization and death. The c-statistic for the base model (history and medications) was 0.657. The model improved with physical examination, laboratory, and KCCQ results, with IDI increases of 4.9, 7.0, and 3.2%, respectively (P < 0.001 each). The combination of all three offered the greatest incremental gain (c-statistic 0.749; IDI increase 10.8%).
Physical examination, laboratories, and KCCQ assessed 1 week after discharge offer important prognostic information, suggesting that all are critical components of outpatient evaluation after HF hospitalization.
Heart failure; Prognosis; Health status; Hospitalization; Mortality; BNP
Mortality among patients with heart failure (HF) is high. Though individual biomarkers have been investigated to determine their value in mortality risk prediction, the role of a multimarker strategy requires further evaluation.
Methods and Results
Olmsted County residents presenting with HF from July 2004 to September 2007 were recruited to undergo biomarker measurement. We investigated whether addition of C-reactive protein (CRP), B-type natriuretic peptide (BNP), and troponin T (TnT) to a model including established risk indicators improved 1-year mortality risk prediction using the c statistic, integrated discrimination improvement (IDI), and net reclassification improvement (NRI). Among 593 participants, the mean age was 76.4 years and 48% were men. After 1 year follow-up, 122 (20.6%) participants had died. Patients with CRP (<11.8mg/L), BNP (<350pg/mL), and TnT (≤0.01ng/mL) below the median had low 1-year mortality (3.3%), while those with two or three biomarkers above the median had markedly increased mortality (30.8% and 35.5%, respectively). The addition of two or more biomarkers to the model offered greater improvement in 1-year mortality risk prediction than use of a single biomarker. The combination of CRP and BNP resulted in an increase in the c statistic from 0.757 to 0.810 (p<0.001), an IDI gain of 7.1% (p<0.001), and a NRI of 22.1% (p<0.001). Use of all three biomarkers offered no incremental gain (IDI gain 0.7% vs. CRP+BNP, p=0.065).
Biomarkers improved 1-year mortality risk prediction beyond established indicators. The use of a two-biomarker combination was superior to a single biomarker in risk prediction, though addition of a third biomarker conferred no added benefit.
epidemiology; heart failure; prognosis; inflammation; community
To date, the only established model for assessing risk for nasopharyngeal carcinoma (NPC) relies on the sero-status of the Epstein-Barr virus (EBV). By contrast, the risk assessment models proposed here include environmental risk factors, family history of NPC, and information on genetic variants. The models were developed using epidemiological and genetic data from a large case-control study, which included 1,387 subjects with NPC and 1,459 controls of Cantonese origin. The predictive accuracy of the models were then assessed by calculating the area under the receiver-operating characteristic curves (AUC). To compare the discriminatory improvement of models with and without genetic information, we estimated the net reclassification improvement (NRI) and integrated discrimination index (IDI). Well-established environmental risk factors for NPC include consumption of salted fish and preserved vegetables and cigarette smoking (in pack years). The environmental model alone shows modest discriminatory ability (AUC = 0.68; 95% CI: 0.66, 0.70), which is only slightly increased by the addition of data on family history of NPC (AUC = 0.70; 95% CI: 0.68, 0.72). With the addition of data on genetic variants, however, our model’s discriminatory ability rises to 0.74 (95% CI: 0.72, 0.76). The improvements in NRI and IDI also suggest the potential usefulness of considering genetic variants when screening for NPC in endemic areas. If these findings are confirmed in larger cohort and population-based case-control studies, use of the new models to analyse data from NPC-endemic areas could well lead to earlier detection of NPC.
To determine whether ascites can improve risk discrimination beyond MELD and serum sodium (MELDNa).
Consecutive cirrhotic patients were evaluated for ascites based on an outpatient CT along with concurrent MELD and Na values. Cox models were used to determine the added value of ascites for predicting one-year mortality. Increases in the C-index, integrated discrimination improvement (IDI), and the net reclassification index (NRI) were used to assess improvements in discrimination after the addition of ascites.
1,003 patients had Na and MELD scores available within 30 days of the CT scan. Sixty deaths occurred within one year, with mortality higher in patients with ascites (21.4% versus 4.0%, HR 6.08, 95% CI 3.62–10.19, p<0.0005). In the presence of ascites, the MELD and MELDNa score underestimated mortality risk when the scores were less than 21. The addition of ascites to the MELDNa model substantially improved discrimination by the C-index (0.804 versus 0.770, increase of 3.4%, 95% CI 0.2%–9.9%), IDI (1.8%, p=0.016), and NRI (15.8%, p=0.0006).
The incorporation of radiographic ascites significantly improves upon MELDNa for predicting one-year mortality. The presence of ascites may help identify patients at increased risk for mortality not otherwise captured by MELD or MELDNa.
Ascites; MELD; MELDNa; prognostic model; cirrhosis
The discovery and development of new biomarkers continues to be an exciting and promising field. Improvement of prediction of risk of developing disease is one of the key motivations in these pursuits. Appropriate statistical measures are necessary for drawing meaningful conclusions about the clinical usefulness of these new markers. In this review, we present several novel metrics proposed to serve this purpose. We use reclassification tables constructed based on clinically meaningful disease risk categories to discuss the concepts of calibration, risk separation, risk discrimination, and risk classification accuracy. We discuss the notion that the net reclassification improvement is a simple yet informative way to summarize information contained in risk reclassification tables. In the absence of meaningful risk categories, we suggest a ‘category-less’ version of the net reclassification improvement and integrated discrimination improvement as metrics to summarize the incremental value of new biomarkers. We also suggest that predictiveness curves be preferred to receiver-operating-characteristic curves as visual descriptors of a statistical model’s ability to separate predicted probabilities of disease events. Reporting of standard metrics, including measures of relative risk and the c statistic is still recommended. These concepts are illustrated with a risk prediction example using data from the Framingham Heart Study.
reclassification; risk prediction; NRI; IDI; calibration; discrimination
Rapid detection of methicillin-resistant Staphylococcus aureus (MRSA) by PCR can be performed directly from nasal specimens with the IDI-MRSA assay. To improve the efficiency of screening, we evaluated the performance of the IDI-MRSA assay for the detection of MRSA from pooled and unpooled specimens cultured in a selective broth. Of the 287 specimens evaluated, 71 were culture and PCR positive, 203 were culture and PCR negative, 3 were culture positive and PCR negative, 8 were culture negative and PCR positive, and 2 remained inhibited. A methicillin-susceptible Staphylococcus aureus isolate was recovered from five of the eight specimens with false-positive PCR results. Compared to the results of culture, the sensitivity, specificity, and negative and positive predictive values of the IDI-MRSA assay for detection of MRSA from broth were 96%, 96%, 90%, and 98%, respectively. Following implementation of the IDI-MRSA assay, PCR-positive broths were subcultured for evaluation of assay performance. Of the 298 IDI-MRSA assay-positive broths, the results for 103 could not be confirmed by culture. A methicillin-susceptible S. aureus (MSSA) isolate was recovered from 77 of these 103 broths. Repeat testing by the IDI-MRSA assay directly with the MSSA isolates confirmed the original positive PCR result. The positive predictive value of the IDI-MRSA assay fell from 90% during the evaluation phase to 65% postimplementation. The IDI-MRSA assay performed well for the detection of MRSA from a selective broth compared to the performance of the detection of MRSA from culture. However, because of the burden associated with implementation of infection control precautions, cultures remain essential in confirming positive IDI-MRSA results.
This study compares inflammation-related biomarkers with established cardiometabolic risk factors in the prediction of incident type 2 diabetes and incident coronary events in a prospective case-cohort study within the population-based MONICA/KORA Augsburg cohort.
Methods and Findings
Analyses for type 2 diabetes are based on 436 individuals with and 1410 individuals without incident diabetes. Analyses for coronary events are based on 314 individuals with and 1659 individuals without incident coronary events. Mean follow-up times were almost 11 years. Areas under the receiver-operating characteristic curve (AUC), changes in Akaike's information criterion (ΔAIC), integrated discrimination improvement (IDI) and net reclassification index (NRI) were calculated for different models. A basic model consisting of age, sex and survey predicted type 2 diabetes with an AUC of 0.690. Addition of 13 inflammation-related biomarkers (CRP, IL-6, IL-18, MIF, MCP-1/CCL2, IL-8/CXCL8, IP-10/CXCL10, adiponectin, leptin, RANTES/CCL5, TGF-β1, sE-selectin, sICAM-1; all measured in nonfasting serum) increased the AUC to 0.801, whereas addition of cardiometabolic risk factors (BMI, systolic blood pressure, ratio total/HDL-cholesterol, smoking, alcohol, physical activity, parental diabetes) increased the AUC to 0.803 (ΔAUC [95% CI] 0.111 [0.092–0.149] and 0.113 [0.093–0.149], respectively, compared to the basic model). The combination of all inflammation-related biomarkers and cardiometabolic risk factors yielded a further increase in AUC to 0.847 (ΔAUC [95% CI] 0.044 [0.028–0.066] compared to the cardiometabolic risk model). Corresponding AUCs for incident coronary events were 0.807, 0.825 (ΔAUC [95% CI] 0.018 [0.013–0.038] compared to the basic model), 0.845 (ΔAUC [95% CI] 0.038 [0.028–0.059] compared to the basic model) and 0.851 (ΔAUC [95% CI] 0.006 [0.003–0.021] compared to the cardiometabolic risk model), respectively.
Inclusion of multiple inflammation-related biomarkers into a basic model and into a model including cardiometabolic risk factors significantly improved the prediction of type 2 diabetes and coronary events, although the improvement was less pronounced for the latter endpoint.
To determine among community patients with heart failure (HF), whether pulmonary artery systolic pressure (PASP) assessed by Doppler echocardiography was associated with death and improved risk prediction over established factors, using the integrated discrimination improvement (IDI) and net reclassification improvement (NRI).
While several studies have focused on idiopathic pulmonary arterial hypertension, less is known about pulmonary hypertension among patients with HF, particularly on its prognostic value in the community.
Olmsted County residents with HF between 2003 and 2010 prospectively underwent assessment of ejection fraction (EF), diastolic function, and PASP by Doppler echocardiography.
PASP was recorded in 1049 of 1153 patients (mean age 76±13, 51% women). Median PASP was 48 mmHg (25th-75th percentile, 37.0-58.0). There were 489 deaths after a follow-up of 2.7±1.9 years. There was a strong positive graded association between PASP and mortality. Increasing PASP was associated with an increased risk of death (HR 1.45, 95%CI 1.13-1.85 for tertile 2; HR 2.07, 95%CI 1.62-2.64 for tertile 3, versus tertile 1), independently of age, sex, comorbidities, EF and diastolic function. Adding PASP to models including these clinical characteristics resulted in an increase in the c-statistic from 0.704 to 0.742 (p=0.007), an IDI gain of 4.2% (p<0.001), and an NRI of 14.1% (p=0.002), indicating that PASP improved prediction of death over traditional prognostic factors. All results were similar for CV death.
Among community patients with HF, PASP strongly predicts death and provides incremental and clinically relevant prognostic information independently of known predictors of outcomes.
heart failure; pulmonary hypertension; mortality; community
We evaluated whether the addition of carotid intima media thickness and plaque (CIMT-P), and, a single nucleotide polymorphism on chromosome 9p21 (9p21) together improve coronary heart disease (CHD) risk prediction in the ARIC study.
Ten year CHD risk was estimated using the ARIC coronary risk score (ACRS) alone and in combination with CIMT-P and 9p21 individually and together in White participants (n=9338). Area under the receiver operating characteristic curve (AUC), model calibration, net reclassification index (NRI), integrated discrimination index (IDI) and number of individuals reclassified were estimated.
The AUC of the ACRS, ACRS+9p21, ACRS+CIMT-P and ACRS+CIMT-P+9p21 models were 0.748, 0.751, 0.763 and 0.766 respectively. The percentage of individuals reclassified, model calibration, NRI and IDI improved when CIMT-P and 9p21 were added to the ACRS only model (see manuscript).
Addition of 9p21 allele information to CIMT-P minimally improves CHD risk prediction in whites in the ARIC study.
Carotid intima media thickness; Plaque; 9p21; Risk prediction; Coronary heart disease
Current management of lung nodules is complicated by nontherapeutic resections and missed chances for cure. We hypothesized that a serum proteomic signature may add diagnostic information beyond that provided by combined clinical and radiographic data.
Cohort A included 265 and cohort B 114 patients. Using multivariable logistic regression analysis we calculated the area under the receiver operating characteristic curve (AUC) and quantified the added value of a previously described serum proteomic signature beyond clinical and radiographic risk factors for predicting lung cancer using the integration discrimination improvement (IDI) index.
The average computed tomography (CT) measured nodule size in cohorts A and B was 37.83 versus 23.15 mm among patients with lung cancer and 15.82 versus 17.18 mm among those without, respectively. In cohort A, the AUC increased from 0.68 to 0.86 after adding chest CT imaging variables to the clinical results, but the proteomic signature did not provide meaningful added value. In contrast, in cohort B, the AUC improved from 0.46 with clinical data alone to 0.61 when combined with chest CT imaging data and to 0.69 after adding the proteomic signature (IDI of 20% P = 0.0003). In addition, in a subgroup of 100 nodules between 5 and 20 mm in diameter, the proteomic signature added value with an IDI of 15% (P ≤ 0.0001).
The results show that this serum proteomic biomarker signature may add value to the clinical and chest CT evaluation of indeterminate lung nodules.
This study suggests a possible role of a blood biomarker in the evaluation of indeterminate lung nodules.
The use of an Infectious Diseases Impact Statement (IDIS) is proposed for predictive assessments of local changes in infectious diseases arising from human-engineered activities. IDIS is intended to be analogous to an Environmental Impact Statement. The drafting of an IDIS for specific activities, particularly in developing nations, would provide a formal mechanism for examining potential changes in local health conditions, including infected and susceptible populations, diseases likely to fluctuate in response to development, existing control measures, and vectors likely to be affected by human activities. The resulting survey data could provide a rational basis and direction for development, surveillance, and prevention measures. An IDIS process that balances environmental alterations, local human health, and economic growth could substantially alter the nature of international development efforts and infectious disease outbreaks.
B-type natriuretic peptide (BNP) has been associated with short- and long-term post-discharge prognosis among hospitalized heart failure (HF) patients. It is unknown if admission, discharge, or change from admission to discharge BNP measure is the most important predictor of long-term outcomes.
Methods and Results
We linked patients ≥65 years from hospitals in OPTIMIZE-HF to Medicare claims. Among patients with recorded admission and discharge BNP, we compared Cox models predicting 1-year mortality and/or rehospitalization, including clinical variables and clinical variables plus BNP. We calculated the net reclassification improvement (NRI) and integrated discrimination improvement (IDI) for the best-fit model for each outcome versus the model with clinical variables alone. Among 7039 patients in 220 hospitals, median (25th, 75th) admission and discharge BNP were 832 pg/mL (451, 1660) and 534 pg/mL (281, 1111). Observed 1-year mortality and 1-year mortality or rehospitalization rates were 35.2% and 79.4%. The discharge BNP model had the best performance and was the most important characteristic for predicting 1-year mortality (hazard ratio [HR] for log transformation 1.34; 95% CI 1.28–1.40) and 1-year death or rehospitalization (HR 1.15; 95% CI 1.12–1.18). Compared with a clinical variables only model, the discharge BNP model improved risk reclassification and discrimination in predicting each outcome (1-year mortality: NRI 5.5%, P<0.0001; IDI 0.023, P<0.0001; 1-year mortality or rehospitalization: NRI 4.2%, P<0.0001; IDI 0.010, P<0.0001).
Discharge BNP best predicts 1-year mortality and/or rehospitalization among older patients hospitalized with HF. Discharge BNP plus clinical variables modestly improves risk classification and model discrimination for long-term outcomes.
B-type natriuretic peptide; outcomes; OPTIMIZE-HF; risk stratification
An updated IDI-MRSA assay version was released to address the assay's low positive predictive value (PPV). A prospective analysis of two assay versions indicated no significant improvement in the PPV. Colonization by methicillin-resistant Staphylococcus aureus in 24% of patients would not have been detected if only nasal samples had been tested, as approved, by this molecular method.
Risk prediction procedures can be quite useful for the patient’s treatment selection, prevention strategy, or disease management in evidence-based medicine. Often, potentially important new predictors are available in addition to the conventional markers. The question is how to quantify the improvement from the new markers for prediction of the patient’s risk in order to aid cost–benefit decisions. The standard method, using the area under the receiver operating characteristic curve, to measure the added value may not be sensitive enough to capture incremental improvements from the new markers. Recently, some novel alternatives to area under the receiver operating characteristic curve, such as integrated discrimination improvement and net reclassification improvement, were proposed. In this paper, we consider a class of measures for evaluating the incremental values of new markers, which includes the preceding two as special cases. We present a unified procedure for making inferences about measures in the class with censored event time data. The large sample properties of our procedures are theoretically justified. We illustrate the new proposal with data from a cancer study to evaluate a new gene score for prediction of the patient’s survival.
area under the receiver operating characteristic curve; C-statistic; Cox’s regression; integrated discrimination improvement; net reclassification improvement; risk prediction
Isopentenyl diphosphate isomerase (IDI) catalyzes the interconversion of isopentenyl diphosphate (IPP) and dimethylallyl diphosphate (DMAPP), the basic five-carbon building blocks of isoprenoid molecules. Two structurally unrelated classes of IDI are known. Type I IPP isomerase (IDI-1) utilizes a divalent metal in a protonation-deprotonation reaction. In contrast, the type II enzyme (IDI-2) requires reduced flavin, raising the possibility that the reaction catalyzed by IDI-2 involves the net addition/abstraction of a hydrogen atom. As part of our studies of the mechanism of isomerization for IDI-2, we synthesized allene and alkyne substrate analogues for the enzyme. These molecules are predicted to be substantially less reactive toward proton addition than IPP and DMAPP, but have similar reactivities toward hydrogen atom addition. This prediction was verified by calculations of gas phase heats of reaction for addition of a proton and of a hydrogen atom to 1-butyne (3) and 1,2-butadiene (4) to form the 1-buten-2-yl carbocation and radical, respectively, and related affinities for 2-methyl-1-butene (5) and 2-methyl-2-butene (6) using G3MP2B3 and CBS-QB3 protocols. Alkyne 1-OPP and allene 2-OPP were not substrates for Thermus thermophilus IDI-2 or Escherichia coli IDI-1, but instead were competitive inhibitors. The experimental and computational results are consistent with a protonation-deprotonation mechanism for the enzyme-catalyzed isomerization of IPP and DMAPP.
Starting in June 2010 the Infectious Diseases Institute (IDI) clinic (a large urban HIV out-patient facility) switched to provider-based Electronic Medical Records (EMR) from paper EMR entered in the database by data-entry clerks. Standardized clinics forms were eliminated but providers still fill free text clinical notes in physical patients’ files. The objective of this study was to compare the rate of errors in the database before and after the introduction of the provider-based EMR.
Methods and Findings
Data in the database pre and post provider-based EMR was compared with the information in the patients’ files and classified as correct, incorrect, and missing. We calculated the proportion of incorrect, missing and total error for key variables (toxicities, opportunistic infections, reasons for treatment change and interruption). Proportions of total errors were compared using chi-square test. A survey of the users of the EMR was also conducted. We compared data from 2,382 visits (from 100 individuals) of a retrospective validation conducted in 2007 with 34,957 visits (from 10,920 individuals) of a prospective validation conducted in April–August 2011. The total proportion of errors decreased from 66.5% in 2007 to 2.1% in 2011 for opportunistic infections, from 51.9% to 3.5% for ART toxicity, from 82.8% to 12.5% for reasons for ART interruption and from 94.1% to 0.9% for reasons for ART switch (all P<0.0001). The survey showed that 83% of the providers agreed that provider-based EMR led to improvement of clinical care, 80% reported improved access to patients’ records, and 80% appreciated the automation of providers’ tasks.
The introduction of provider-based EMR improved the quality of data collected with a significant reduction in missing and incorrect information. The majority of providers and clients expressed satisfaction with the new system. We recommend the use of provider-based EMR in large HIV programs in Sub-Saharan Africa.
Isopentenyl diphosphate isomerase from M. jannaschii has been overexpressed in E. coli, purified and crystallized. Diffraction data were collected to 2.08 Å resolution.
Type 2 isopentenyl diphosphate isomerase (IDI-2) is a flavoprotein. Recently, flavin has been proposed to play a role as a general acid–base catalyst with no redox role during the enzyme reaction. To clarify the detailed enzyme reaction mechanism of IDI-2 and the unusual role of flavin, structural analysis of IDI-2 from Methanocaldococcus jannaschii (MjIDI) was performed. Recombinant MjIDI was crystallized at 293 K using calcium acetate as a precipitant. The diffraction of the crystal extended to 2.08 Å resolution at 100 K. The crystal belonged to the tetragonal space group I422, with unit-cell parameters a = 126.46, c = 120.03 Å. The presence of one monomer per asymmetric unit gives a crystal volume per protein weight (V
M) of 3.0 Å3 Da−1 and a solvent constant of 59.0% by volume.
isopentenyl diphosphate isomerase; flavoproteins; no net redox reaction
The recently identified type II isopentenyl diphosphate (IPP):dimethylallyl diphosphate (DMAPP) isomerase (IDI-2) is a flavoenzyme that requires FMN and NAD(P)H for activity. IDI-2 is an essential enzyme for the biosynthesis of isoprenoids in several pathogenic bacteria including Staphylococcus aureus, Streptococcus pneumoniae, and Enterococcus faecalis, and thus is considered as a potential new drug target to battle bacterial infections. One notable feature of the IDI-2 reaction is that there is no net change in redox state between the substrate (IPP) and product (DMAPP), indicating that the FMN cofactor must start and finish each catalytic cycle in the same redox state. Here, we report the characterization and initial mechanistic studies of the S. aureus IDI-2. The steady-state kinetic analyses under aerobic and anaerobic conditions show that FMN must be reduced to be catalytically active and the overall IDI-2 reaction is O2 sensitive. Interestingly, our results demonstrate that NADPH is needed only in catalytic amounts to activate the enzyme for multiple turnovers of IPP to DMAPP. The hydride transfer from NAD(P)H to reduce FMN is determined to be pro-S stereospecific. Photoreduction and oxidation-reduction potential studies reveal that the S. aureus IDI-2 can stabilize significant amounts of the neutral FMN semiquinone. In addition, reconstitution of apo-IDI-2 with 5-deazaFMN resulted in a dead enzyme, whereas reconstitution with 1-deazaFMN led to the full recovery of enzyme activity. Taken together, these studies of S. aureus IDI-2 support a catalytic mechanism in which the reduced flavin coenzyme mediates a single electron transfer to and from the IPP substrate during catalysis.
Terpenoids constitute a large family of natural products, attracting commercial interest for a variety of uses as flavours, fragrances, drugs and alternative fuels. Saccharomyces cerevisiae offers a versatile cell factory, as the precursors of terpenoid biosynthesis are naturally synthesized by the sterol biosynthetic pathway.
S. cerevisiae wild type yeast cells, selected for their capacity to produce high sterol levels were targeted for improvement aiming to increase production. Recyclable integration cassettes were developed which enable the unlimited sequential integration of desirable genetic elements (promoters, genes, termination sequence) at any desired locus in the yeast genome. The approach was applied on the yeast sterol biosynthetic pathway genes HMG2, ERG20 and IDI1 resulting in several-fold increase in plant monoterpene and sesquiterpene production. The improved strains were robust and could sustain high terpenoid production levels for an extended period. Simultaneous plasmid-driven co-expression of IDI1 and the HMG2 (K6R) variant, in the improved strain background, maximized monoterpene production levels. Expression of two terpene synthase enzymes from the sage species Salvia fruticosa and S. pomifera (SfCinS1, SpP330) in the modified yeast cells identified a range of terpenoids which are also present in the plant essential oils. Co-expression of the putative interacting protein HSP90 with cineole synthase 1 (SfCinS1) also improved production levels, pointing to an additional means to improve production.
Using the developed molecular tools, new yeast strains were generated with increased capacity to produce plant terpenoids. The approach taken and the durability of the strains allow successive rounds of improvement to maximize yields.
This study evaluated the relation between adiponectin and atherosclerosis in both genders, and investigated whether adiponectin provides useful additional information for assessing the risk of atherosclerosis.
We measured serum adiponectin levels and other cardiovascular risk factors in 1033 subjects (454 men, 579 women) from the Korean Genomic Rural Cohort study. Carotid intima–media-thickness (CIMT) was used as measure of atherosclerosis. Odds ratios (ORs) with 95% confidence intervals (95% CI) were calculated using multiple logistic regression, and receiver operating characteristic curves (ROC), the category-free net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were calculated.
After adjustment for conventional cardiovascular risk factors, such as age, waist circumference, smoking history, low-density and high-density lipoprotein cholesterol, triglycerides, systolic blood pressure and insulin resistance, the ORs (95%CI) of the third tertile adiponectin group were 0.42 (0.25–0.72) in men and 0.47 (0.29–0.75) in women. The area under the curve (AUC) on the ROC analysis increased significantly by 0.025 in men and 0.022 in women when adiponectin was added to the logistic model of conventional cardiovascular risk factors (AUC in men: 0.655 to 0.680, p = 0.038; AUC in women: 0.654 to 0.676, p = 0.041). The NRI was 0.32 (95%CI: 0.13–0.50, p<0.001), and the IDI was 0.03 (95%CI: 0.01–0.04, p<0.001) for men. For women, the category-free NRI was 0.18 (95%CI: 0.02–0.34, p = 0.031) and the IDI was 0.003 (95%CI: −0.002–0.008, p = 0.189).
Adiponectin and atherosclerosis were significantly related in both genders, and these relationships were independent of conventional cardiovascular risk factors. Furthermore, adiponectin provided additional information to conventional cardiovascular risk factors regarding the risk of atherosclerosis.