elderly; clopidogrel; glycoprotein IIb/IIIa blockers
To investigate whether the beneficial and harmful effects of platelet glycoprotein IIb/IIIa receptor blockers in non‐ST elevation acute coronary syndromes (NSTE‐ACS) depend on age.
A meta‐analysis of six trials of platelet glycoprotein IIb/IIIa receptor blockers in patients with NSTE‐ACS (PRISM, PRISM‐PLUS, PARAGON‐A, PURSUIT, PARAGON‐B, GUSTO IV‐ACS; n = 31 402) was performed. We applied multivariable logistic regression analyses to evaluate the drug effects on death or non‐fatal myocardial infarction at 30 days, and on major bleeding, by age subgroups (<60, 60–69, 70–79, ⩾80 years). We quantified the reduction of death or myocardial infarction as the number needed to treat (NNT), and the increase of major bleeding as the number needed to harm (NNH).
Subgroups had 11 155 (35%), 9727 (31%), 8468 (27%) and 2049 (7%) patients, respectively. The relative benefit of platelet glycoprotein IIb/IIIa receptor blockers did not differ significantly (p = 0.5) between age subgroups (OR (95% CI) for death or myocardial infarction: 0.86 (0.74 to 0.99), 0.90 (0.80 to 1.02), 0.97 (0.86 to 1.10), 0.90 (0.73 to 1.16); overall 0.91 (0.86 to 0.99). ORs for major bleeding were 1.9 (1.3 to 2.8), 1.9 (1.4 to 2.7), 1.6 (1.2 to 2.1) and 2.5 (1.5–4.1). Overall NNT was 105, and overall NNH was 90. The oldest patients had larger absolute increases in major bleeding, but also had the largest absolute reductions of death or myocardial infarction. Patients ⩾80 years had half of the NNT and a third of the NNH of patients <60 years.
In patients with NSTE‐ACS, the relative reduction of death or non‐fatal myocardial infarction with platelet glycoprotein IIb/IIIa receptor blockers was independent of patient age. Larger absolute outcome reductions were seen in older patients, but with a higher risk of major bleeding. Close monitoring of these patients is warranted.
The discriminative ability of a risk model is often measured by Harrell’s concordance-index (c-index). The c-index estimates for two randomly chosen subjects the probability that the model predicts a higher risk for the subject with poorer outcome (concordance probability). When data are clustered, as in multicenter data, two types of concordance are distinguished: concordance in subjects from the same cluster (within-cluster concordance probability) and concordance in subjects from different clusters (between-cluster concordance probability). We argue that the within-cluster concordance probability is most relevant when a risk model supports decisions within clusters (e.g. who should be treated in a particular center). We aimed to explore different approaches to estimate the within-cluster concordance probability in clustered data.
We used data of the CRASH trial (2,081 patients clustered in 35 centers) to develop a risk model for mortality after traumatic brain injury. To assess the discriminative ability of the risk model within centers we first calculated cluster-specific c-indexes. We then pooled the cluster-specific c-indexes into a summary estimate with different meta-analytical techniques. We considered fixed effect meta-analysis with different weights (equal; inverse variance; number of subjects, events or pairs) and random effects meta-analysis. We reflected on pooling the estimates on the log-odds scale rather than the probability scale.
The cluster-specific c-index varied substantially across centers (IQR = 0.70-0.81; I
= 0.76 with 95% confidence interval 0.66 to 0.82). Summary estimates resulting from fixed effect meta-analysis ranged from 0.75 (equal weights) to 0.84 (inverse variance weights). With random effects meta-analysis – accounting for the observed heterogeneity in c-indexes across clusters – we estimated a mean of 0.77, a between-cluster variance of 0.0072 and a 95% prediction interval of 0.60 to 0.95. The normality assumptions for derivation of a prediction interval were better met on the probability than on the log-odds scale.
When assessing the discriminative ability of risk models used to support decisions at cluster level we recommend meta-analysis of cluster-specific c-indexes. Particularly, random effects meta-analysis should be considered.
Clustered data; Concordance; Discrimination; Meta-analysis; Prediction; Risk model
It is known that interprofessional collaboration is crucial for integrated care delivery, yet we are still unclear about the underlying mechanisms explaining effectiveness of integrated care delivery to older patients. In addition, we lack research comparing integrated care delivery between hospitals. Therefore, this study aims to (i) provide insight into the underlying components ‘relational coordination’ and ‘situational awareness’ of integrated care delivery and the role of team and organizational context in integrated care delivery; and (ii) compare situational awareness, relational coordination, and integrated care delivery of different hospitals in the Netherlands.
This cross-sectional study took place in 2012 among professionals from three different hospitals involved in the delivery of care to older patients. A total of 215 professionals filled in the questionnaire (42% response rate).Descriptive statistics and paired-sample t-tests were used to investigate the level of situational awareness, relational coordination, and integrated care delivery in the three different hospitals. Correlation and multilevel analyses were used to investigate the relationship between background characteristics, team context, organizational context, situational awareness, relational coordination and integrated care delivery.
No differences in background characteristics, team context, organizational context, situational awareness, relational coordination and integrated care delivery were found among the three hospitals. Correlational analysis revealed that situational awareness (r = 0.30; p < 0.01), relational coordination (r = 0.17; p < 0.05), team climate (r = 0.29; p < 0.01), formal internal communication (r = 0.46; p < 0.01), and informal internal communication (r = 0.36; p < 0.01) were positively associated with integrated care delivery. Stepwise multilevel analyses showed that formal internal communication (p < 0.001) and situational awareness (p < 0.01) were associated with integrated care delivery. Team climate was not significantly associated with integrated care delivery when situational awareness and relational coordination were included in the equation. Thus situational awareness acted as mediator between team climate and integrated care delivery among professionals delivering care to older hospitalized patients.
The results of this study show the importance of formal internal communication and situational awareness for quality of care delivery to hospitalized older patients.
Patients with multiple colorectal adenomas may carry germline mutations in the APC or MUTYH genes.
To determine the prevalence of pathogenic APC and MUTYH mutations in patients who had undergone genetic testing and compare the prevalence and clinical characteristics of APC and MUTYH mutation carriers.
Design, Setting and Participants
This cross-sectional study consisted of 8676 unrelated individuals who had undergone full gene sequencing and large rearrangement analysis of the APC gene and targeted sequence analysis for the two most common MUTYH mutations (Y179C and G396D) between 2004 and 2011. Individuals with either mutation underwent full MUTYH gene sequencing. We evaluated APC and MUTYH mutation prevalence by polyp burden and the clinical characteristics associated with a pathogenic mutation using logistic regression analyses.
Main Outcome Measure
Deleterious mutations in APC and MUTYH genes.
Colorectal adenomas were reported in 7225 individuals; 1457 with classic polyposis (≥ 100 adenomas) and 3253 with attenuated polyposis (20-99 adenomas). The prevalence of APC and biallelic MUTYH mutations was 95/119 (80%, 95%CI 71-87%) and 2/119 (2%, 95%CI 0.2-6%) among individuals with ≥ 1000 adenomas, 756/1338 (56%, 95%CI 54-59%) and 94/1338 (7%, 95%CI 6-8%) among individuals with 100-999 adenomas, 326/3253 (10%, 95%CI (9-11%) and 233/3253 (7%, 95%CI 6-8%) among individuals with 20-99 adenomas, and 50/970 (5%, 95%CI 4-7%) and 37/970 (4%, 95%CI 3-5%) among those with 10-19 adenomas.
Among patients with multiple colorectal adenomas, APC and MUTYH mutation prevalence varied considerably by adenoma count including within those with a classic polyposis phenotype. APC mutations predominate in patients with classic polyposis, whereas prevalence of APC and MYH mutations is similar in attenuated polyposis. These findings require external validation.
Individual participant data (IPD) meta-analyses often analyze their IPD as if coming from a single study. We compare this approach with analyses that rather account for clustering of patients within studies.
Study Design and Setting
Comparison of effect estimates from logistic regression models in real and simulated examples.
The estimated prognostic effect of age in patients with traumatic brain injury is similar, regardless of whether clustering is accounted for. However, a family history of thrombophilia is found to be a diagnostic marker of deep vein thrombosis [odds ratio, 1.30; 95% confidence interval (CI): 1.00, 1.70; P = 0.05] when clustering is accounted for but not when it is ignored (odds ratio, 1.06; 95% CI: 0.83, 1.37; P = 0.64). Similarly, the treatment effect of nicotine gum on smoking cessation is severely attenuated when clustering is ignored (odds ratio, 1.40; 95% CI: 1.02, 1.92) rather than accounted for (odds ratio, 1.80; 95% CI: 1.29, 2.52). Simulations show models accounting for clustering perform consistently well, but downwardly biased effect estimates and low coverage can occur when ignoring clustering.
Researchers must routinely account for clustering in IPD meta-analyses; otherwise, misleading effect estimates and conclusions may arise.
Individual participant data meta-analysis; Individual patient data; Evidence synthesis; Cluster; Simulation; Binary outcome; Pooled analysis
To examine the prognostic value of different comorbidity coding schemes for predicting survival of newly diagnosed elderly cancer patients.
Materials and Methods
We analyzed data from 8,867 patients aged 65 years of age or older, newly diagnosed with cancer. Comorbidities present at the time of diagnosis were collected using the Adult Comorbidity Evaluation-27 index (ACE-27). We examined multiple scoring schemes based on the individual comorbidity ailments, and their severity rating. Harrell’s c index and Akaike Information Criterion (AIC) were used to evaluate the performance of the different comorbidity models.
Comorbidity led to an increase in c index from 0.771 for the base model to 0.782 for a model that included indicator variables for every ailment. The prognostic value was however much higher for prostate and breast cancer patients. A simple model which considered linear scores from 0 to 3 per ailment, controlling for cancer type, was optimal according to AIC.
The presence of comorbidity impacts on the survival of elderly cancer patients, especially for less lethal cancers, such as prostate and breast cancers. Different ailments have different impacts on survival, necessitating the use of different weights per ailment in a simple summary score of the ACE-27.
Comorbidity; comorbid ailment; elderly; cancer patients; prognostic; survival
For health care performance indicators (PIs) to be reliable, data underlying the PIs are required to be complete, accurate, consistent and reproducible. Given the lack of regulation of the data-systems used in the Netherlands, and the self-report based indicator scores, one would expect heterogeneity with respect to the data collection and the ways indicators are computed. This might affect the reliability and plausibility of the nationally reported scores.
We aimed to investigate the extent to which local hospital data collection and indicator computation strategies differ and how this affects the plausibility of self-reported indicator scores, using survey results of 42 hospitals and data of the Dutch national quality database.
The data collection and indicator computation strategies of the hospitals were substantially heterogenic. Moreover, the Hip and Knee replacement PI scores can be regarded as largely implausible, which was, to a great extent, related to a limited (computerized) data registry. In contrast, Breast Cancer PI scores were more plausible, despite the incomplete data registry and limited data access. This might be explained by the role of the regional cancer centers that collect most of the indicator data for the national cancer registry, in a standardized manner. Hospitals can use cancer registry indicator scores to report to the government, instead of their own locally collected indicator scores.
Indicator developers, users and the scientific field need to focus more on the underlying (heterogenic) ways of data collection and conditional data infrastructures. Countries that have a liberal software market and are aiming to implement a self-report based performance indicator system to obtain health care transparency, should secure the accuracy and precision of the heath care data from which the PIs are calculated. Moreover, ongoing research and development of PIs and profound insight in the clinical practice of data registration is warranted.
Performance indicators; Health care quality; Reliability; Hospital information system
Prognostic models for outcome prediction in patients with traumatic brain injury (TBI) are important instruments in both clinical practice and research. To remain current a continuous process of model validation is necessary. We aimed to investigate the performance of the International Mission on Prognosis and Analysis of Clinical Trials in TBI (IMPACT) prognostic models in predicting mortality in a contemporary New York State TBI registry developed and maintained by the Brain Trauma Foundation. The Brain Trauma Foundation (BTF) TBI-trac® database contains data on 3125 patients who sustained severe TBI (Glasgow Coma Scale [GCS] score ≤8) in New York State between 2000 and 2009. The outcome measure was 14-day mortality. To predict 14-day mortality with admission data, we adapted the IMPACT Core and Extended models. Performance of the models was assessed by determining calibration (agreement between observed and predicted outcomes), and discrimination (separation of those patients who die from those who survive). Calibration was explored graphically with calibration plots. Discrimination was expressed by the area under the receiver operating characteristic (ROC) curve (AUC). A total of 2513 out of 3125 patients in the BTF database met the inclusion criteria. The 14-day mortality rate was 23%. The models showed excellent calibration. Mean predicted probabilities were 20% for the Core model and 24% for the Extended model. Both models showed good discrimination with AUCs of 0.79 (Core) and 0.83 (Extended). We conclude that the IMPACT models validly predict 14-day mortality in the BTF database, confirming generalizability of these models for outcome prediction in TBI patients.
external validation; outcome; prediction models; traumatic brain injury
We aimed to determine to what extent covariate adjustment could affect power in a randomized controlled trial (RCT) of a heterogeneous population with traumatic brain injury (TBI).
Study Design and Setting
We analyzed 14-day mortality in 9497 participants in the Corticosteroid Randomisation After Significant Head Injury (CRASH) RCT of corticosteroid vs. placebo. Adjustment was made using logistic regression for baseline covariates of two validated risk models derived from external data (IMPACT) and from the CRASH data. The relative sample size (RESS) measure, defined as the ratio of the sample size required by an adjusted analysis to attain the same power as the unadjusted reference analysis, was used to assess the impact of adjustment.
Corticosteroid was associated with higher mortality compared to placebo (OR=1.25, 95% CI: 1.13, 1.39). RESS of 0.79 and 0.73 were obtained by adjustment using the IMPACT and CRASH models, respectively, which for example implies an increase from 80% to 88% and 91% power, respectively.
Moderate gains in power may be obtained using covariate adjustment from logistic regression in heterogeneous conditions such as TBI. Although analyses of RCTs might consider covariate adjustment to improve power, we caution against this approach in the planning of RCTs.
covariate adjustment; prognostic targeting; strict selection; relative sample size; power in clinical trials; traumatic brain injury
New markers may improve prediction of diagnostic and prognostic outcomes. We review various measures to quantify the incremental value of markers over standard, readily available characteristics. Widely used traditional measures include the improvement in model fit or in the area under the receiver operating characteristic (ROC) curve (AUC). New measures include the net reclassification index (NRI) and decision–analytic measures, such as the fraction of true positive classifications penalized for false positive classifications (‘net benefit’, NB).
For illustration we discuss a case study on the presence of residual tumor versus benign tissue in 544 patients with testicular cancer. We assessed 3 tumor markers (AFP, HCG, and LDH) for their incremental value over currently standard clinical predictors. AUC and R2 values suggested adding continuous LDH and AFP whereas NB only favored HCG as a potentially promising marker at a clinically defendable decision threshold of 20% risk. Results based on the NRI fell in the middle, suggesting reclassification potential of all three markers.
We conclude that improvement in standard discrimination measures, which focus on finding variables that might be promising across all decision thresholds, may not detect the most informative markers at a specific threshold of particular clinical relevance. When a marker is intended to support decision making, calculation of the improvement in a decision–analytic measure, such as NB, is preferable over an overall judgment as obtained from the AUC in ROC analysis.
prediction; logistic regression model; performance measures; incremental value
The performance of prediction models can be assessed using a variety of different methods and metrics. Traditional measures for binary and survival outcomes include the Brier score to indicate overall model performance, the concordance (or c) statistic for discriminative ability (or area under the receiver operating characteristic (ROC) curve), and goodness-of-fit statistics for calibration.
Several new measures have recently been proposed that can be seen as refinements of discrimination measures, including variants of the c statistic for survival, reclassification tables, net reclassification improvement (NRI), and integrated discrimination improvement (IDI). Moreover, decision–analytic measures have been proposed, including decision curves to plot the net benefit achieved by making decisions based on model predictions.
We aimed to define the role of these relatively novel approaches in the evaluation of the performance of prediction models. For illustration we present a case study of predicting the presence of residual tumor versus benign tissue in patients with testicular cancer (n=544 for model development, n=273 for external validation).
We suggest that reporting discrimination and calibration will always be important for a prediction model. Decision-analytic measures should be reported if the predictive model is to be used for making clinical decisions. Other measures of performance may be warranted in specific applications, such as reclassification metrics to gain insight into the value of adding a novel predictor to an established model.
Peritoneal dialysis is an effective treatment for end-stage renal disease. Key to successful peritoneal dialysis is a well-functioning catheter. The different insertion techniques may be of great importance. Mostly, the standard operative approach is the open technique; however, laparoscopic insertion is increasingly popular. Catheter malfunction is reported up to 35% for the open technique and up to 13% for the laparoscopic technique. However, evidence is lacking to definitely conclude that the laparoscopic approach is to be preferred. This review and meta-analysis was carried out to investigate if one of the techniques is superior to the other.
Comprehensive searches were conducted in MEDLINE, Embase and CENTRAL (the Cochrane Library 2012, issue 10). Reference lists were searched manually. The methodology was in accordance with the Cochrane Handbook for interventional systematic reviews, and written based on the PRISMA-statement.
Three randomized controlled trials and eight cohort studies were identified. Nine postoperative outcome measures were meta-analyzed; of these, seven were not different between operation techniques. Based on the meta-analysis, the proportion of migrating catheters was lower (odds ratio (OR) 0.21, confidence interval (CI) 0.07 to 0.63; P = 0.006), and the one-year catheter survival was higher in the laparoscopic group (OR 3.93, CI 1.80 to 8.57; P = 0.0006).
Based on these results there is some evidence in favour of the laparoscopic insertion technique for having a higher one-year catheter survival and less migration, which would be clinically relevant.
In this article, the third in the PROGRESS series on prognostic factor research, Sara Schroter and colleagues review how prognostic models are developed and validated, and then address how prognostic models are assessed for their impact on practice and patient outcomes, illustrating these ideas with examples.
In the second article in the PROGRESS series on prognostic factor research, Sara Schroter and colleagues discuss the role of prognostic factors in current clinical practice, randomised trials, and developing new interventions, and explain why and how prognostic factor research should be improved.
Lynch syndrome is caused by germline mismatch repair (MMR) gene mutations. The PREMM1,2,6 model predicts the likelihood of a MMR gene mutation based on personal and family cancer history.
To compare strategies using PREMM1,2,6 and tumour testing (microsatellite instability (MSI) and/or immunohistochemistry (IHC) staining) to identify mutation carriers.
Data from population-based or clinic-based patients with colorectal cancers enrolled through the Colon Cancer Family Registry were analysed. Evaluation included MSI, IHC and germline mutation analysis for MLH1, MSH2, MSH6 and PMS2. Personal and family cancer histories were used to calculate PREMM1,2,6 predictions. Discriminative ability to identify carriers from non-carriers using the area under the receiver operating characteristic curve (AUC) was assessed. Predictions were based on logistic regression models for (1) cancer assessment using PREMM1,2,6, (2) MSI, (3) IHC for loss of any MMR protein expression, (4) MSI + IHC, (5) PREMM1,2,6 + MSI, (6) PREMM1,2,6 + IHC, (7) PREMM1,2,6 + IHC + MSI.
Among 1651 subjects, 239 (14%) had mutations (90 MLH1, 125 MSH2, 24 MSH6). PREMM1,2,6 discriminated well with AUC 0.90 (95% CI 0.88 to 0.92). MSI alone, IHC alone, or MSI + IHC each had lower AUCs: 0.77, 0.82 and 0.82, respectively. The added value of IHC + PREMM1,2,6 was slightly greater than PREMM1,2,6 + MSI (AUC 0.94 vs 0.93). Adding MSI to PREMM1,2,6 + IHC did not improve discrimination.
PREMM1,2,6 and IHC showed excellent performance in distinguishing mutation carriers from non-carriers and performed best when combined. MSI may have a greater role in distinguishing Lynch syndrome from other familial colorectal cancer subtypes among cases with high PREMM1,2,6 scores where genetic evaluation does not disclose a MMR mutation.
The Prevention and Reactivation Care Program (PReCaP) entails an innovative multidisciplinary, integrated and goal oriented approach aimed at reducing hospital related functional decline among elderly patients. Despite calls for process evaluation as an essential component of clinical trials in the geriatric care field, studies assessing fidelity lag behind the number of effect studies. The threefold purpose of this study was (1) to systematically assess intervention fidelity of the hospital phase of the PReCaP in the first year of the intervention delivery; (2) to improve our understanding of the moderating factors and modifications affecting intervention fidelity; and (3) to explore the feasibility of the PReCaP fidelity assessment in view of the modifications.
Based on the PReCaP description we developed a fidelity instrument incorporating nineteen (n=19) intervention components. A combination of data collection methods was utilized, i.e. data collection from patient records and individual Goal Attainment Scaling care plans, in-depth interviews with stakeholders, and non-participant observations. Descriptive analysis was performed to obtain levels of fidelity of each of the nineteen PReCaP components. Moderating factors were identified by using the Conceptual Framework for Implementation Fidelity.
Ten of the nineteen intervention components were always or often delivered to the group of twenty elderly patients. Moderating factors, such as facilitating strategies and context were useful in explaining the non- or low-adherence of particular intervention components.
Fidelity assessment was carried out to evaluate the adherence to the PReCaP in the Vlietland Ziekenhuis in the Netherlands. Given that the fidelity was assessed in the first year of PReCaP implementation it was commendable that ten of the nineteen intervention components were performed always or often. The adequate delivery of the intervention components strongly depended on various moderating factors. Since the intervention is still developing and undergoing continuous modifications, it has been concluded that the fidelity criteria should evolve with the modified intervention. Furthermore, repeated intervention fidelity assessments will be necessary to ensure a valid and reliable fidelity assessment of the PReCaP.
The Netherlands National Trial Register: NTR2317
Geriatric care intervention; Intervention fidelity; Moderating factors
The Glasgow Outcome Scale (GOS) is firmly established as the primary outcome measure for use in Phase III trials of interventions in traumatic brain injury (TBI). However, the GOS has been criticized for its lack of sensitivity to detect small but clinically relevant changes in outcome. The Glasgow Outcome Scale-Extended (GOSE) potentially addresses this criticism, and in this study we estimate the efficiency gain associated with using the GOSE in place of the GOS in ordinal analysis of 6-month outcome. The study uses both simulation and the reanalysis of existing data from two completed TBI studies, one an observational cohort study and the other a randomized controlled trial. As expected, the results show that using an ordinal technique to analyze the GOS gives a substantial gain in efficiency relative to the conventional analysis, which collapses the GOS onto a binary scale (favorable versus unfavorable outcome). We also found that using the GOSE gave a modest but consistent increase in efficiency relative to the GOS in both studies, corresponding to a reduction in the required sample size of the order of 3–5%. We recommend that the GOSE be used in place of the GOS as the primary outcome measure in trials of TBI, with an appropriate ordinal approach being taken to the statistical analysis.
clinical trial; Glasgow Outcome Scale-Extended; Glasgow Outcome Scale; ordinal analysis; outcome; traumatic brain injury
We developed a Monte Carlo Markov model designed to investigate the effects of modifying cardiovascular disease (CVD) risk factors on the burden of CVD. Internal, predictive, and external validity of the model have not yet been established.
The Rotterdam Ischemic Heart Disease and Stroke Computer Simulation (RISC) model was developed using data covering 5 years of follow-up from the Rotterdam Study. To prove 1) internal and 2) predictive validity, the incidences of coronary heart disease (CHD), stroke, CVD death, and non-CVD death simulated by the model over a 13-year period were compared with those recorded for 3,478 participants in the Rotterdam Study with at least 13 years of follow-up. 3) External validity was verified using 10 years of follow-up data from the European Prospective Investigation of Cancer (EPIC)-Norfolk study of 25,492 participants, for whom CVD and non-CVD mortality was compared.
At year 5, the observed incidences (with simulated incidences in brackets) of CHD, stroke, and CVD and non-CVD mortality for the 3,478 Rotterdam Study participants were 5.30% (4.68%), 3.60% (3.23%), 4.70% (4.80%), and 7.50% (7.96%), respectively. At year 13, these percentages were 10.60% (10.91%), 9.90% (9.13%), 14.20% (15.12%), and 24.30% (23.42%). After recalibrating the model for the EPIC-Norfolk population, the 10-year observed (simulated) incidences of CVD and non-CVD mortality were 3.70% (4.95%) and 6.50% (6.29%). All observed incidences fell well within the 95% credibility intervals of the simulated incidences.
We have confirmed the internal, predictive, and external validity of the RISC model. These findings provide a basis for analyzing the effects of modifying cardiovascular disease risk factors on the burden of CVD with the RISC model.
Cardiovascular disease prevention; Simulation modeling; Model validation
In well-child care it is difficult to determine whether preschool children with asthma symptoms actually have or will develop asthma at school age. The PIAMA (Prevention and Incidence of Asthma and Mite Allergy) Risk Score has been proposed as an instrument that predicts asthma at school age, using eight easy obtainable parameters, assessed at the time of first asthma symptoms at preschool age. The aim of this study is to present the rationale and design of a study 1) to externally validate and update the PIAMA Risk Score, 2) to develop an Asthma Risk Appraisal Tool to predict asthma at school age in (specific subgroups of) preschool children with asthma symptoms and 3) to test implementation of the Asthma Risk Appraisal Tool in well-child care.
Methods and design
The study will be performed within the framework of Generation R, a prospective multi-ethnic cohort study. In total, consent for postnatal follow-up was obtained from 7893 children, born between 2002 and 2006. At preschool age the PIAMA Risk Score will be assessed and used to predict asthma at school age. Discrimination (C-index) and calibration will be assessed for the external validation. We will study whether the predictive ability of the PIAMA Risk Score can be improved by removing or adding predictors (e.g. preterm birth). The (updated) PIAMA Risk Score will be converted to the Asthma Risk Appraisal Tool- to predict asthma at school age in preschool children with asthma symptoms. Additionally, we will conduct a pilot study to test implementation of the Asthma Risk Appraisal Tool in well-child care.
Application of the Asthma Risk Appraisal Tool in well-child care will help to distinguish preschool children at high- and low-risk of developing asthma at school age when asthma symptoms appear.
This study will increase knowledge about the validity of the PIAMA risk score and might improve risk assessment of developing asthma at school age in (specific subgroups of) preschool children, who present with asthma symptoms at well-child care.
Asthma Risk Appraisal Tool; Children; External validation; Prediction; Well-child care
In biomedical research, the logistic regression model is the most commonly used method for predicting the probability of a binary outcome. While many clinical researchers have expressed an enthusiasm for regression trees, this method may have limited accuracy for predicting health outcomes. We aimed to evaluate the improvement that is achieved by using ensemble-based methods, including bootstrap aggregation (bagging) of regression trees, random forests, and boosted regression trees. We analyzed 30-day mortality in two large cohorts of patients hospitalized with either acute myocardial infarction (N = 16,230) or congestive heart failure (N = 15,848) in two distinct eras (1999–2001 and 2004–2005). We found that both the in-sample and out-of-sample prediction of ensemble methods offered substantial improvement in predicting cardiovascular mortality compared to conventional regression trees. However, conventional logistic regression models that incorporated restricted cubic smoothing splines had even better performance. We conclude that ensemble methods from the data mining and machine learning literature increase the predictive performance of regression trees, but may not lead to clear advantages over conventional logistic regression models for predicting short-term mortality in population-based samples of subjects with cardiovascular disease.
PMID: 22777999 CAMSID: cams2404
Acute myocardial infarction; Bagging; Boosting; Data mining; Heart failure
Individuals with a family history of colorectal cancer (CRC) are at increased risk for CRC. Current screening recommendations for these individuals are based on expert opinion. We investigated optimal screening strategies for individuals with a varying degree of family history of CRC based on a cost-effectiveness analysis.
We used the MISCAN-Colon micro-simulation model to estimate costs and effects of CRC screening strategies, varying by age to start and stop screening and screening interval. We defined four risk groups, characterized by the number of affected first degree relatives (FDR) and their age at CRC diagnosis. For all risk groups, the optimal screening strategy had an incremental cost effectiveness ratio (ICER) of approximately $50,000 per life-year gained.
The optimal screening strategy for individuals with 1 FDR diagnosed after age 50 was 6 colonoscopies every 5 years starting at age 50, compared to 4 colonoscopies every 7 years starting at age 50 for average risk individuals. The optimal strategy had 10 colonoscopies every 4 years for individuals with 1 FDR diagnosed before age 50, 13 colonoscopies every 3 years for individuals with 2 or more FDRs diagnosed after age 50, and 15 colonoscopies every 3 years for individuals with two or more FDRs of which at least 1 is diagnosed before age 50.
The optimal screening strategy varies considerably with the number of affected FDRs and their age of diagnosis. Shorter screening intervals than the currently recommended 5 years may be appropriate for the highest risk individuals.
Early Detection of Cancer; Colonoscopy; Colorectal Neoplasm; Familial Risk; Cost-Effectiveness Analysis
Diagnostic and prognostic literature is overwhelmed with studies reporting univariable predictor-outcome associations. Currently, methods to incorporate such information in the construction of a prediction model are underdeveloped and unfamiliar to many researchers.
This article aims to improve upon an adaptation method originally proposed by Greenland (1987) and Steyerberg (2000) to incorporate previously published univariable associations in the construction of a novel prediction model. The proposed method improves upon the variance estimation component by reconfiguring the adaptation process in established theory and making it more robust. Different variants of the proposed method were tested in a simulation study, where performance was measured by comparing estimated associations with their predefined values according to the Mean Squared Error and coverage of the 90% confidence intervals.
Results demonstrate that performance of estimated multivariable associations considerably improves for small datasets where external evidence is included. Although the error of estimated associations decreases with increasing amount of individual participant data, it does not disappear completely, even in very large datasets.
The proposed method to aggregate previously published univariable associations with individual participant data in the construction of a novel prediction models outperforms established approaches and is especially worthwhile when relatively limited individual participant data are available.
When outcomes are binary, the c-statistic (equivalent to the area under the Receiver Operating Characteristic curve) is a standard measure of the predictive accuracy of a logistic regression model.
An analytical expression was derived under the assumption that a continuous explanatory variable follows a normal distribution in those with and without the condition. We then conducted an extensive set of Monte Carlo simulations to examine whether the expressions derived under the assumption of binormality allowed for accurate prediction of the empirical c-statistic when the explanatory variable followed a normal distribution in the combined sample of those with and without the condition. We also examine the accuracy of the predicted c-statistic when the explanatory variable followed a gamma, log-normal or uniform distribution in combined sample of those with and without the condition.
Under the assumption of binormality with equality of variances, the c-statistic follows a standard normal cumulative distribution function with dependence on the product of the standard deviation of the normal components (reflecting more heterogeneity) and the log-odds ratio (reflecting larger effects). Under the assumption of binormality with unequal variances, the c-statistic follows a standard normal cumulative distribution function with dependence on the standardized difference of the explanatory variable in those with and without the condition. In our Monte Carlo simulations, we found that these expressions allowed for reasonably accurate prediction of the empirical c-statistic when the distribution of the explanatory variable was normal, gamma, log-normal, and uniform in the entire sample of those with and without the condition.
The discriminative ability of a continuous explanatory variable cannot be judged by its odds ratio alone, but always needs to be considered in relation to the heterogeneity of the population.
Logistic regression; c-statistic; Area under the receiver operating characteristic curve; ROC curve; Discrimination; Regression model; Prediction; Predictive model; Predictive accuracy
The purpose of this article is to report on the quality of the existing evidence base regarding the effectiveness of clinical pathway (CPW) research in the hospital setting. The analysis is based on a recently published Cochrane review of the effectiveness of CPWs.
An integral component of the review process was a rigorous appraisal of the methodological quality of published CPW evaluations. This allowed the identification of strengths and limitations of the evidence base for CPW effectiveness. We followed the validated Cochrane Effective Practice and Organisation of Care Group (EPOC) criteria for randomized and non-randomized clinical pathway evaluations. In addition, we tested the hypotheses that simple pre-post studies tend to overestimate CPW effects reported.
Out of the 260 primary studies meeting CPW content criteria, only 27 studies met the EPOC study design criteria, with the majority of CPW studies (more than 70 %) excluded from the review on the basis that they were simple pre-post evaluations, mostly comparing two or more annual patient cohorts. Methodologically poor study designs are often used to evaluate CPWs and this compromises the quality of the existing evidence base.
Cochrane EPOC methodological criteria, including the selection of rigorous study designs along with detailed descriptions of CPW development and implementation processes, are recommended for quantitative evaluations to improve the evidence base for the use of CPWs in hospitals.