Covariate adjustment in randomized clinical trials has the potential benefit of precision gain. It also has the potential pitfall of reduced objectivity as it opens the possibility of selecting “favorable” model that yields strong treatment benefit estimate. Although there is a large volume of statistical literature targeting on the first aspect, realistic solutions to enforce objective inference and improve precision are rare. As a typical randomized trial needs to accommodate many implementation issues beyond statistical considerations, maintaining the objectivity is at least as important as precision gain if not more, particularly from the perspective of the regulatory agencies. In this article, we propose a two-stage estimation procedure based on inverse probability weighting to achieve better precision without compromising objectivity. The procedure is designed in a way such that the covariate adjustment is performed before seeing the outcome, effectively reducing the possibility of selecting a “favorable” model that yields a strong intervention effect. Both theoretical and numerical properties of the estimation procedure are presented. Application of the proposed method to a real data example is presented.
clinical trials; covariate adjustment; efficiency; inverse probability weighting; objectivity
It is well recognized that the conventional summary of treatment effect by averaging across individual patients has its limitation in ignoring the heterogeneous responses to the treatment in the target population. However, there are few alternative metrics in the literature that are designed to capture such heterogeneity. We propose the concept of treatment benefit rate (TBR) and treatment harm rate (THR) that characterize both the overall treatment effect and the magnitude of heterogeneity. We discuss a method to estimate TBR and THR that easily incorporates a sensitivity analysis scheme, and illustrate the idea through analysis of a randomized trial that evaluates the Implantable Cardioverter-Defibrillator (ICD) in reducing mortality. A simulation study is presented to assess the performance of the proposed method.
Causal inference; Heterogeneity in treatment effect; Potential outcomes; Sub-group analysis
Prophylactic mastectomy (PM) offers 90% or greater reduction in
risk of breast cancer to women at increased hereditary risk. Nonetheless,
acceptance in North America has been low (0–36%). Most women
report reduced cancer worry post-operatively, but up to 25–50%
of women electing surgery also report psychological distress and/or difficulty
adapting following PM. Psychological consultation to aid decision-making and
improve post-surgical coping isn’t routinely offered. This
retrospective, cross-sectional study explored, quantitatively and qualitatively,
interest in and acceptability of psychological consultation for issues related
to PM among 108 women who had undergone or were considering surgery. Of the 71
women who had undergone PM, more than half felt pre-surgical psychological
consultation was advisable and nearly 2/3 felt post-surgical psychological
consultation would be helpful. All 37 women (100%) currently considering
PM believed psychological consultation would aid decision-making and preparation
Narratives from the interviews illustrate the nature and intensity of the
need for psychological support and describe preferences for the role of the
psychologist. Suggestions are offered for the integration of psychological
services for women deciding about or adapting to PM.
Drug–drug interaction (DDI) alerting is an important form of clinical decision support, yet physicians often fail to attend to critical DDI warnings due to alert fatigue. We previously described a model for highlighting patients at high risk of a DDI by enhancing alerts with relevant laboratory data. We sought to evaluate the effect of this model on alert adherence in high-risk patients.
A 6-month randomized controlled trial involving 1029 outpatient physicians was performed. The target interactions were all DDIs known to cause hyperkalemia. Alerts in the intervention group were enhanced with the patient's most recent potassium and creatinine levels. The control group received unmodified alerts. High -risk patients were those with baseline potassium >5.0 mEq/l and/or creatinine ≥1.5 mg/dl (132 μmol/l).
We found no significant difference in alert adherence in high-risk patients between the intervention group (15.3%) and the control group (16.8%) (p=0.71). Adherence in normal risk patients was significantly lower in the intervention group (14.6%) than in the control group (18.6%) (p<0.01). In neither group did physicians increase adherence in patients at high risk.
Physicians adhere poorly to hyperkalemia-associated DDI alerts even in patients with risk factors for a clinically significant interaction, and the display of relevant laboratory data in these alerts did not improve adherence levels in the outpatient setting. Further research is necessary to determine optimal strategies for conveying patient-specific DDI risk.
Decision Support Systems, Clinical; Drug Interactions; Drug Therapy, Computer Assisted; Medical Order Entry Systems; Reminder Systems
We review the class of inverse probability weighting (IPW) approaches for the analysis of missing data under various missing data patterns and mechanisms. The IPW methods rely on the intuitive idea of creating a pseudo-population of weighted copies of the complete cases to remove selection bias introduced by the missing data. However, different weighting approaches are required depending on the missing data pattern and mechanism. We begin with a uniform missing data pattern (i.e., a scalar missing indicator indicating whether or not the full data is observed) to motivate the approach. We then generalize to more complex settings. Our goal is to provide a conceptual overview of existing IPW approaches and illustrate the connections and differences among these approaches.
missing data; inverse probability weighting; missing at random; missing not at random; monotone missing; non-monotone missing
Rapamycin acts synergistically with platinum agents to induce apoptosis and inhibit proliferation in breast cancer cell lines. Combination of everolimus also known as RAD001 (oral mammalian target of rapamycin (mTOR) inhibitor) and carboplatin may have activity in metastatic triple-negative breast cancer (TNBC).
The primary objective of this study was to determine clinical benefit rate (CBR), that is (complete remission (CR) + partial remission (PR) + stable disease (SD) lasting ≥6 months) and the toxicity of everolimus/carboplatin in women with metastatic TNBC. Prior carboplatin was allowed. Treatment consisted of intravenous carboplatin area under the curve (AUC) 6 (later decreased to AUC 5 and subsequently to AUC 4) every 3 weeks with daily 5 mg everolimus.
We enrolled 25 patients in this study. Median age was 58 years. There were one CR, six PRs, seven SDs and eight PDs (progression of disease). CBR was 36% (95% confidence interval (CI) 21.1 to 57.4%). One SD was achieved in a patient progressing on single agent carboplatin. The median progression free survival (PFS) was 3 months (95% CI 1.6 to 4.6 months) and overall survival (OS) was 16.6 months (95% CI 7.3 months to not reached). There were seven patients (28%) with ≥ grade 3 thrombocytopenia; three (12%) with grade 3 neutropenia (no bleeding/febrile neutropenia) and one (4%) with grade 3 anemia. Greater hematological toxicity was seen in the first seven patients treated with carboplatin AUC5/6. After the amendment for starting dose of carboplatin to AUC 4, the regimen was well tolerated with only one out of 18 patients with grade 3 neutropenia and two patients with grade 3 thrombocytopenia. There was only one case of mucositis.
Everolimus-carboplatin was efficacious in metastatic TNBC. Dose limiting hematological toxicity was observed when AUC5/6 of carboplatin was combined with everolimus. However, carboplatin AUC 4 was well tolerated in combination with everolimus with continuing responses.
Evaluation of impact of potential uncontrolled confounding is an important component for causal inference based on observational studies. In this article, we introduce a general framework of sensitivity analysis that is based on inverse probability weighting. We propose a general methodology that allows both non-parametric and parametric analyses, which are driven by two parameters that govern the magnitude of the variation of the multiplicative errors of the propensity score and their correlations with the potential outcomes. We also introduce a specific parametric model that offers a mechanistic view on how the uncontrolled confounding may bias the inference through these parameters. Our method can be readily applied to both binary and continuous outcomes and depends on the covariates only through the propensity score that can be estimated by any parametric or non-parametric method. We illustrate our method with two medical data sets.
Causal inference; Inverse probability weighting; Propensity score; Sensitivity analysis; Uncontrolled confounding
As CT screening is integrated into non-small cell lung cancer (NSCLC) care, additional parameters are needed to help distinguish cancers from benign nodules. Osteopontin (OPN), a secreted phosphoprotein, has elevated plasma levels in NSCLC. We hypothesize that changes in plasma OPN over time (i.e., OPN velocity [OPNV]) can differentiate NSCLC patients from those without cancer in a CT screening population.
A nested case-control study was conducted within a NSCLC CT screening trial. Incident cancers with serial plasma were matched to controls. OPN was measured by ELISA. Demographic, OPN, and OPNV were compared between cancers and controls using Wilcoxon Signed Rank test.
Ten incident cancers were identified. The pack years distributions were similar, but cancers were older (68.0 vs. 63.3, p=0.002) and their surveillance intervals were shorter (25.5 vs. 27.0 months, p=0. 03) than matched controls. Baseline OPN was similar (median difference: −5.15 ng/ml, p=0.50), but OPNV in the cancers was significantly greater than that of matched controls, (median difference: 1.06 ng/ml/month, p = 0.01). Accuracy rate for prediction of disease status by OPNV (adjusted for age and surveillance) was 83%.
These are early evidence for utility of monitoring plasma OPN during CT screening to assist in identification of NSCLCs.
non-small cell lung cancer; osteopontin; biomarker; early detection; CT screening
Exact unconditional tests have been widely applied to test the difference between two probabilities for 2×2 matched-pairs binary data with small sample size. In this context, Lloyd (2008, Biometrics 64, 716–723) proposed an E + M p-value, that showed better performance than the existing M p-value and C p-value. However, the analytical calculation of the E + M p-value requires that the Barnard convexity condition be satisfied; this can be challenging to prove theoretically. In this paper, by a simple reformulation, we show that a weaker condition, conditional monotonicity, is sufficient to calculate all three p-values (M, C and E + M) and their corresponding exact sizes. Moreover, this conditional monotonicity condition is applicable to non-inferiority tests.
Barnard convexity; conditional monotonicity; exact unconditional test; matched-pairs; non-inferiority test
The objective of the current study was to comprehensively compare the genomic profiles in the breast of parous and nulliparous postmenopausal women to identify genes that permanently change their expression following pregnancy.
The study was designed as a two-phase approach. In the discovery phase, we compared breast genomic profiles of 37 parous with 18 nulliparous postmenopausal women. In the validation phase, confirmation of the genomic patterns observed in the discovery phase was sought in an independent set of 30 parous and 22 nulliparous postmenopausal women. RNA was hybridized to Affymetrix HG_U133 Plus 2.0 oligonucleotide arrays containing probes to 54,675 transcripts; scanned and the images analyzed using Affymetrix GCOS software. Surrogate variable analysis, logistic regression and significance analysis for microarrays were used to identify statistically significant differences in expression of genes. The False Discovery Rate (FDR) approach was used to control for multiple comparisons. We found that 208 genes (305 probe sets) were differentially expressed between parous and nulliparous women in both discovery and validation phases of the study at a FDR of 10% and with at least a 1.25-fold change. These genes are involved in regulation of transcription, centrosome organization, RNA splicing, cell cycle control, adhesion and differentiation. The results provide persuasive evidence that full-term pregnancy induces long-term genomic changes in the breast. The genomic signature of pregnancy could be used as an intermediate marker to assess potential chemopreventive interventions with hormones mimicking the effects of pregnancy for prevention of breast cancer.
Pregnancy; breast; genes; expression; postmenopausal
Drug-drug interactions (DDIs) are a common cause of adverse drug events. In this paper, we combined a literature discovery approach with analysis of a large electronic medical record database method to predict and evaluate novel DDIs. We predicted an initial set of 13197 potential DDIs based on substrates and inhibitors of cytochrome P450 (CYP) metabolism enzymes identified from published in vitro pharmacology experiments. Using a clinical repository of over 800,000 patients, we narrowed this theoretical set of DDIs to 3670 drug pairs actually taken by patients. Finally, we sought to identify novel combinations that synergistically increased the risk of myopathy. Five pairs were identified with their p-values less than 1E-06: loratadine and simvastatin (relative risk or RR = 1.69); loratadine and alprazolam (RR = 1.86); loratadine and duloxetine (RR = 1.94); loratadine and ropinirole (RR = 3.21); and promethazine and tegaserod (RR = 3.00). When taken together, each drug pair showed a significantly increased risk of myopathy when compared to the expected additive myopathy risk from taking either of the drugs alone. Based on additional literature data on in vitro drug metabolism and inhibition potency, loratadine and simvastatin and tegaserod and promethazine were predicted to have a strong DDI through the CYP3A4 and CYP2D6 enzymes, respectively. This new translational biomedical informatics approach supports not only detection of new clinically significant DDI signals, but also evaluation of their potential molecular mechanisms.
Drug-drug interactions are a common cause of adverse drug events. In this paper, we developed an automated search algorithm which can predict new drug interactions based on published literature. Using a large electronic medical record database, we then analyzed the correlation between concurrent use of these potentially interacting drugs and the incidence of myopathy as an adverse drug event. Myopathy comprises a range of musculoskeletal conditions including muscle pain, weakness, and tissue breakdown (rhabdomyolysis). Our statistical analysis identified 5 drug interaction pairs: (loratadine, simvastatin), (loratadine, alprazolam), (loratadine, duloxetine), (loratadine, ropinirole), and (promethazine, tegaserod). When taken together, each drug pair showed a significantly increased risk of myopathy when compared to the expected additive myopathy risk from taking either of the drugs alone. Further investigation suggests that two major drug metabolism proteins, CYP2D6 and CYP3A4, are involved with these five drug pairs' interactions. Overall, our method is robust in that it can incorporate all published literature, all FDA approved drugs, and very large clinical datasets to generate predictions of clinically significant interactions. The interactions can then be further validated in future cell-based experiments and/or clinical studies.
The authors developed a sensitivity analysis method to address the issue of uncontrolled confounding in observational studies. In this method, the authors use a 1-dimensional function of the propensity score, which they refer to as the sensitivity function (SF), to quantify the hidden bias due to unmeasured confounders. The propensity score is defined as the conditional probability of being treated given the measured covariates. Then the authors construct SF-corrected inverse-probability-weighted estimators to draw inference on the causal treatment effect. This approach allows analysts to conduct a comprehensive sensitivity analysis in a straightforward manner by varying sensitivity assumptions on both the functional form and the coefficients in the 1-dimensional SF. Furthermore, 1-dimensional continuous functions can be well approximated by low-order polynomial structures (e.g., linear, quadratic). Therefore, even if the imposed SF is practically certain to be incorrect, one can still hope to obtain valuable information on treatment effects by conducting a comprehensive sensitivity analysis using polynomial SFs with varying orders and coefficients. The authors demonstrate the new method by implementing it in an asthma study which evaluates the effect of clinician prescription patterns regarding inhaled corticosteroids for children with persistent asthma on selected clinical outcomes.
confounding factors (epidemiology); inverse probability weighting; propensity score; sensitivity analysis; sensitivity function; uncontrolled confounding
Little evidence exists on effective interventions to integrate HIV-care guidelines into practices within developing countries. This study tested the hypothesis that clinical summaries with computer-generated reminders could improve clinicians' compliance with CD4 testing guidelines in the resource-limited setting of sub-Saharan Africa.
A prospective comparative study of two randomly selected outpatient adult HIV clinics in western Kenya. Printed summaries with reminders for overdue CD4 tests were made available to clinicians in the intervention clinic but not in the control clinic.
Changes in order rates for overdue CD4 tests were compared between and within the two clinics.
The computerized reminder system identified 717 encounters (21%) with overdue CD4 tests. Analysis by study assignment (regardless of summaries being printed or not) revealed that with computer-generated reminders, CD4 order rates were significantly higher in the intervention clinic compared to the control clinic (53% vs 38%, OR=1.80, CI 1.34 to 2.42, p<0.0001). When comparison was restricted to encounters where summaries with reminders were printed, order rates in intervention clinic were even higher (63%). The intervention clinic increased CD4 ordering from 42% before reminders to 63% with reminders (50% increase, OR=2.32, CI 1.67 to 3.22, p<0.0001), compared to control clinic with only 8% increase from prestudy baseline (CI 0.83 to 1.46, p=0.51).
Evaluation was conducted at two clinics in a single institution.
Clinical summaries with computer-generated reminders significantly improved clinician compliance with CD4 testing guidelines in the resource-limited setting of sub-Saharan Africa. This technology can have broad applicability to improve quality of HIV care in these settings.
Standardized means, commonly used in observational studies in epidemiology to adjust for potential confounders, are equal to inverse probability weighted means with inverse weights equal to the empirical propensity scores. More refined standardization corresponds with empirical propensity scores computed under more flexible models. Unnecessary standardization induces efficiency loss. However, according to the theory of inverse probability weighted estimation, propensity scores estimated under more flexible models induce improvement in the precision of inverse probability weighted means. This apparent contradiction is clarified by explicitly stating the assumptions under which the improvement in precision is attained.
Causal inference; Propensity score; Standardized mean
Patients on multiple medications are at increased risk for adverse drug events. While physicians can reduce this risk by regularly reviewing the side-effect profiles of their patients’ medications, this process can be time-consuming. We created a decision support system designed to expedite reviewing potential adverse reactions through information visualization. The system includes a database containing 16,340 unique drug and side-effect pairs, representing 250 common medications. A numeric score is assigned to each pair reflecting the strength of association between drug and effect. Based on these scores, the system generates graphical adverse reaction maps for any user-selected combination of drugs. A study comparing speed and accuracy of retrieving side-effect data using this tool versus UpToDate® demonstrated a 60% reduction in time to complete a query (61 seconds vs. 155 seconds, p<0.0001) with no decrease in accuracy. These findings suggest that information visualization can significantly expedite review of potential adverse drug events.
adverse drug events; side-effects; information visualization; polypharmacy
To evaluate risk of suicide ideation (SI) after childhood cancer, prevalence of SI in a cohort of adult survivors of pediatric cancers was compared with prevalence in a sibling comparison group. The relationship of SI to cancer treatment and current health was examined, and the hypothesis that poor physical health is significantly associated with suicidality, after adjusting for depression, was specifically tested.
Nine thousand one hundred twenty-six adult survivors of childhood cancer and 2,968 siblings enrolled onto the Childhood Cancer Survivor Study completed a survey describing their demographics and medical and psychological functioning, including SI in the prior week.
Of survivors, 7.8% reported SI compared with 4.6% of controls (odds ratio = 1.79; 95% CI, 1.4 to 2.4). Suicidality was unrelated to age, age at diagnosis, sex, cancer therapy, recurrence, time since diagnosis, or second malignancy. SI was associated with primary CNS cancer diagnosis, depression, and poor health outcomes including chronic conditions, pain, and poor global health rating. A logistic regression analysis showed that poor current physical health was significantly associated with SI even after adjusting for cancer diagnosis and depression.
Adult survivors of childhood cancers are at increased risk for SI. Risk of SI is related to cancer diagnosis and post-treatment mental and physical health, even many years after completion of therapy. The association of suicidal symptoms with physical health problems is important because these may be treatable conditions for which survivors seek follow-up care and underscores the need for a multidisciplinary approach to survivor care.
Poor communication of tests whose results are pending at hospital discharge can lead to medical errors.
To determine the adequacy with which hospital discharge summaries document tests with pending results and the appropriate follow-up providers.
Retrospective study of a randomly selected sample
Six hundred ninety-six patients discharged from two large academic medical centers, who had test results identified as pending at discharge through queries of electronic medical records.
INTERVENTION AND MEASUREMENTS
Each patient’s discharge summary was reviewed to identify whether information about pending tests and follow-up providers was mentioned. Factors associated with documentation were explored using clustered multivariable regression models.
Discharge summaries were available for 99.2% of 668 patients whose data were analyzed. These summaries mentioned only 16% of tests with pending results (482 of 2,927). Even though all study patients had tests with pending results, only 25% of discharge summaries mentioned any pending tests, with 13% documenting all pending tests. The documentation rate for pending tests was not associated with level of experience of the provider preparing the summary, patient’s age or race, length of hospitalization, or duration it took for results to return. Follow-up providers’ information was documented in 67% of summaries.
Discharge summaries are grossly inadequate at documenting both tests with pending results and the appropriate follow-up providers.
tests with pending results; continuity of care; patient safety; discharge summary; medical errors
We evaluate the performance of a Natural Language Processing (NLP) application designed to extract follow-up provider information from free-text discharge summaries at two hospitals.
We compare performance by the NLP application, called the Regenstrief EXtracion tool (REX), to performance by three physician reviewers at extracting follow-up provider names, phone/fax numbers and location information. Precision, recall, and F-measures are reported, with 95% CI for pairwise comparisons.
Of 556 summaries with follow-up information, REX performed as follows in precision, recall, F-measure respectively: Provider Name 0.96, 0.92, 0.94; Phone/Fax 0.99, 0.92, 0.96; Location 0.83, 0.82, 0.82. REX was as good as all physician-reviewers in identifying follow-up provider names and phone/fax numbers, and slightly inferior to two physicians at identifying location information. REX took about four seconds (vs. 3–5 minutes for physician-reviewers) to extract follow-up information.
A NLP program had physician-like performance at extracting provider follow-up information from discharge summaries.