Summary

Receiver operating characteristic (ROC) curves can be used to assess the accuracy of tests measured on ordinal or continuous scales. The most commonly used measure for the overall diagnostic accuracy of diagnostic tests is the area under the ROC curve (AUC). A gold standard test on the true disease status is required to estimate the AUC. However, a gold standard test may sometimes be too expensive or infeasible. Therefore, in many medical research studies, the true disease status of the subjects may remain unknown. Under the normality assumption on test results from each disease group of subjects, using the expectation-maximization (EM) algorithm in conjunction with a bootstrap method, we propose a maximum likelihood based procedure for construction of confidence intervals for the difference in paired areas under ROC curves in the absence of a gold standard test. Simulation results show that the proposed interval estimation procedure yields satisfactory coverage probabilities and interval lengths. The proposed method is illustrated with two examples.

SUMMARY

To estimate the multivariate regression model from multiple individual studies, it would be challenging to obtain results if the input from individual studies only provide univariate or incomplete multivariate regression information. Samsa et al. (J. Biomed. Biotechnol. 2005; 2:113–123) proposed a simple method to combine coefficients from univariate linear regression models into a multivariate linear regression model, a method known as synthesis analysis. However, the validity of this method relies on the normality assumption of the data, and it does not provide variance estimates. In this paper we propose a new synthesis method that improves on the existing synthesis method by eliminating the normality assumption, reducing bias, and allowing for the variance estimation of the estimated parameters.

Summary

Covariate-specific ROC curves are often used to evaluate the classification accuracy of a medical diagnostic test or a biomarker, when the accuracy of the test is associated with certain covariates. In many large-scale screening tests, the gold standard is subject to missingness due to high cost or harmfulness to the patient. In this paper, we propose a semiparametric estimation of the covariate-specific ROC curves with a partial missing gold standard. A location-scale model is constructed for the test result to model the covariates’ effect, but the residual distributions are left unspecified. Thus the baseline and link functions of the ROC curve both have flexible shapes. With the gold standard missing at random (MAR) assumption, we consider weighted estimating equations for the location-scale parameters, and weighted kernel estimating equations for the residual distributions. Three ROC curve estimators are proposed and compared, namely, imputation-based, inverse probability weighted and doubly robust estimators. We derive the asymptotic normality of the estimated ROC curve, as well as the analytical form the standard error estimator. The proposed method is motivated and applied to the data in an Alzheimer's disease research.

Our research is motivated by 2 methodological problems in assessing diagnostic accuracy of traditional Chinese medicine (TCM) doctors in detecting a particular symptom whose true status has an ordinal scale and is unknown—imperfect gold standard bias and ordinal scale symptom status. In this paper, we proposed a nonparametric maximum likelihood method for estimating and comparing the accuracy of different doctors in detecting a particular symptom without a gold standard when the true symptom status had an ordered multiple class. In addition, we extended the concept of the area under the receiver operating characteristic curve to a hyper-dimensional overall accuracy for diagnostic accuracy and alternative graphs for displaying a visual result. The simulation studies showed that the proposed method had good performance in terms of bias and mean squared error. Finally, we applied our method to our motivating example on assessing the diagnostic abilities of 5 TCM doctors in detecting symptoms related to Chills disease.

In this paper, we derive sequential conditional probability ratio tests to compare diagnostic tests without distributional assumptions on test results. The test statistics in our method are nonparametric weighted areas under the receiver-operating characteristic curves. By using the new method, the decision of stopping the diagnostic trial early is unlikely to be reversed should the trials continue to the planned end. The conservatism reflected in this approach to have more conservative stopping boundaries during the course of the trial is especially appealing for diagnostic trials since the end point is not death. In addition, the maximum sample size of our method is not greater than a fixed sample test with similar power functions. Simulation studies are performed to evaluate the properties of the proposed sequential procedure. We illustrate the method using data from a thoracic aorta imaging study.

In this paper, we proposed a semi-parametric single-index two-part regression model to weaken assumptions in parametric regression methods that were frequently used in the analysis of skewed data with additional zero values. The estimation procedure for the parameters of interest in the model was easily implemented. The proposed estimators were shown to be consistent and asymptotically normal. Through a simulation study, we showed that the proposed estimators have reasonable finite-sample performance. We illustrated the application of the proposed method in one real study on the analysis of health care costs.

Existing research shows differences in medication use for Alzheimer's disease (AD) based on demographics such as race, ethnicity, and geographical location. To determine individual and community characteristics associated with differences in acetylcholinesterase inhibitor (AChEI) and memantine use in AD, 3,049 AD subjects were drawn from 30 centers and evaluated using the Uniform data set (UDS). Cases were evaluated at the individual level within the context of 31 communities (one center encompassed two separate geographical regions). Multivariate analysis was used to determine the significance of individual variables on medication use. Compared to non-Hispanic Whites, Blacks were less likely to use AChEI and memantine with odds ratios (OR) of 0.59 (95% CI 0.46-0.76) and 0.43 (95% CI 0.32-0.57), respectively. Compared to non-Hispanic Whites, non-Black Hispanics were less likely to use memantine (OR = 0.69 (95% CI 0.49-0.98)). No association was found between the proportion of Blacks or non-Black Hispanics versus non-Hispanic Whites at an Alzheimer Disease Center and individual use of AChEI or memantine. Other significant variables include gender, age, marital status, dementia severity, source of referral, AChEI use, and education. Education and age somewhat mitigated disparity. Significant racial and ethnic differences in AChEI and memantine use exist at the individual level regardless of the racial and ethnic composition of the individual's community. Research and initiatives at the societal level may be an important consideration toward addressing these differences.

Background

Although correlations of cervical cytology to human papillomavirus (HPV) load and histopathology are recognized, it is largely undetermined whether viral load-related risks of cervical intraepithelial neoplasia III (CIN3) differ by cytology.

Methods

Study subjects were 821 women enrolled in the ASCUS-LSIL Triage Study who were positive for HPV16 at entry. Women were followed semi-annually over 2 years. Baseline HPV16 load was measured by real-time PCR; expressed as log10 [HPV16 copies per-nanogram of cellular DNA].

Results

CIN3 was confirmed in 34.8% of 821 women during 2-year follow-up. The adjusted odds ratio (OR) associating 2-year cumulative risk of CIN3 with per log10–unit increase in HPV16 load was 1.46 (95%CI, 1.29-1.64). The ORs varied from 1.66 (95%CI, 1.16-2.37) for women with normal cytology at enrollment to 0.86 (95%CI, 0.61-1.20) for those with HSIL. Among women with normal cytology at enrollment, the area under the receiver operating characteristic curve for detecting CIN3 by viral load was 0.70 (95%CI, 0.61-0.78).

Conclusion

HPV16 DNA load was associated with CIN3 risk but the associations varied with cytology detected at the time when the viral load was measured. Clinical utility of testing for HPV16 load for CIN3 detection was minimal even in women with normal cytology.

Summary

In this paper, we propose a new semi-parametric maximum likelihood (ML) estimate of an ROC curve that satisfies the property of invariance of the ROC curve and is easy to compute. We show that our new estimator is n-consistent and has an asymptotically normal distribution. Our extensive simulation studies show the proposed method is efficient, robust, and simple to compute. Finally, we illustrate the application of the proposed estimator in a real data set.

Objective

Translation of evidence-based guidelines into clinical practice has been inconsistent. We performed a randomized, controlled trial of guideline-based care suggestions delivered to physicians when writing orders on computer workstations.

Study Setting

Inner-city academic general internal medicine practice.

Study Design

Randomized, controlled trial of 246 physicians (25 percent faculty general internists, 75 percent internal medicine residents) and 20 outpatient pharmacists. We enrolled 706 of their primary care patients with asthma or chronic obstructive pulmonary disease. Care suggestions concerning drugs and monitoring were delivered to a random half of the physicians and pharmacists when writing orders or filling prescriptions using computer workstations. A 2 × 2 factorial randomization of practice sessions and pharmacists resulted in four groups of patients: physician intervention, pharmacist intervention, both interventions, and controls.

Data Extraction/Collection Methods

Adherence to the guidelines and clinical activity was assessed using patients' electronic medical records. Health-related quality of life, medication adherence, and satisfaction with care were assessed using telephone questionnaires.

Principal Findings

During their year in the study, patients made an average of five scheduled primary care visits. There were no differences between groups in adherence to the care suggestions, generic or condition-specific quality of life, satisfaction with physicians or pharmacists, medication compliance, emergency department visits, or hospitalizations. Physicians receiving the intervention had significantly higher total health care costs. Physician attitudes toward guidelines were mixed.

Conclusions

Care suggestions shown to physicians and pharmacists on computer workstations had no effect on the delivery or outcomes of care for patients with reactive airways disease.

Background

Plasma total homocysteine (tHcy) is commonly elevated in persons with diabetes. This may be due to effects of insulin and/or glucose and/or metabolic control on the metabolism or plasma levels of tHcy. This study examined the effects of fasting plasma glucose status on fasting tHcy levels among adults without diabetes, and diabetes per se among adults with a self-report history of diabetes.

Methods

Analysis of data on adults (≥ 20y) who had fasted at least 8 hours, from the National Health and Nutrition Examination Survey (1999–2000 and 2001–2002). Subjects with no self-report history of diabetes were grouped according to fasting plasma glucose status as normal (< 100 mg/dL = NFG, n = 2,244), impaired (≥ 100 < 126 mg/dL = IFG, n = 1,108), or a provisional diagnosis of diabetes (≥ 126 mg/dL = DFG, n = 133). Subjects with a self-report history of diabetes (n = 275) were examined separately.

Results

Fasting tHcy was higher (Ps < 0.01) among non-diabetic subjects with DFG and IFG, compared to NFG (median [95% confidence interval] = 8.6 [8.0–9.2], 8.3 [8.1–8.5], and 7.4 [7.3–7.5] μmol/L, respectively). Diabetic subjects had levels similar to non-diabetic subjects with DFG and IFG (8.3 [7.9–8.6] μmol/L). Age and estimated creatinine clearance were strong correlates of fasting tHcy among non-diabetic subjects (r = 0.38 to 0.44 and r = -0.35 to -0.46, respectively) and diabetic subjects (r = 0.41 and r = -0.46, respectively) (Ps < 0.001), while fasting glucose and glycohemoglobin (HbA1c) were weaker (but still significant) correlates of tHcy in non-diabetic and diabetic subjects. Fasting glucose status was not a significant independent predictor of fasting tHcy levels in non-diabetic subjects, and HbA1c was not a significant independent predictor of tHcy in diabetic subjects (Ps > 0.05).

Conclusion

Fasting tHcy levels are elevated among non-diabetic adults with elevated fasting glucose levels, compared to persons with normal fasting glucose levels, and among diabetic adults. However, elevations in fasting tHcy appear to be mediated primarily by age and kidney function, and not by measures of glucose metabolism.

BACKGROUND

Electronic information systems have been proposed as one means to reduce medical errors of commission (doing the wrong thing) and omission (not providing indicated care).

OBJECTIVE

To assess the effects of computer-based cardiac care suggestions.

DESIGN

A randomized, controlled trial targeting primary care physicians and pharmacists.

SUBJECTS

A total of 706 outpatients with heart failure and/or ischemic heart disease.

INTERVENTIONS

Evidence-based cardiac care suggestions, approved by a panel of local cardiologists and general internists, were displayed to physicians and pharmacists as they cared for enrolled patients.

MEASUREMENTS

Adherence with the care suggestions, generic and condition-specific quality of life, acute exacerbations of their cardiac disease, medication compliance, health care costs, satisfaction with care, and physicians' attitudes toward guidelines.

RESULTS

Subjects were followed for 1 year during which they made 3,419 primary care visits and were eligible for 2,609 separate cardiac care suggestions. The intervention had no effect on physicians' adherence to the care suggestions (23% for intervention patients vs 22% for controls). There were no intervention-control differences in quality of life, medication compliance, health care utilization, costs, or satisfaction with care. Physicians viewed guidelines as providing helpful information but constraining their practice and not helpful in making decisions for individual patients.

CONCLUSIONS

Care suggestions generated by a sophisticated electronic medical record system failed to improve adherence to accepted practice guidelines or outcomes for patients with heart disease. Future studies must weigh the benefits and costs of different (and perhaps more Draconian) methods of affecting clinician behavior.

BACKGROUND

Discussions of end-of-life care should be held prior to acute, disabling events. Many barriers to having such discussions during primary care exist. These barriers include time constraints, communication difficulties, and perhaps physicians' anxiety that patients might react negatively to such discussions.

OBJECTIVE

To assess the impact of discussions of advance directives on patients' satisfaction with their primary care physicians and outpatient visits.

DESIGN

Prospective cohort study of patients enrolled in a randomized, controlled trial of the use of computers to remind primary care physicians to discuss advance directives with their elderly, chronically ill patients.

SETTING

Academic primary care general internal medicine practice affiliated with an urban teaching hospital.

PARTICIPANTS

Six hundred eighty-six patients who were at least 75 years old, or at least 50 years old with serious underlying disease, and their 87 primary care physicians (57 residents, 30 faculty general internists) participated in the study.

MEASUREMENTS AND MAIN RESULTS

We assessed patients' satisfaction with their primary care physicians and visits via interviews held in the waiting room after completed visits. Controlling for satisfaction at enrollment and physician, patient, and visit factors, discussing advance directives was associated with greater satisfaction with the physician (P = .052). At follow-up, the strongest predictor of satisfaction with the primary care visit was having previously discussed advance directives with that physician (P = .004), with a trend towards greater visit satisfaction when discussions were held during that visit (P = .069). The percentage of patients scoring a visit as “excellent” increased from 34% for visits without prior advance directive discussions to 51% for visits with such discussions (P = .003).

CONCLUSIONS

Elderly patients with chronic illnesses were more satisfied with their primary care physicians and outpatient visits when advanced directives were discussed. The improvement in visit satisfaction was substantial and persistent. This should encourage physicians to initiate such discussions to overcome communication barriers might result in reduced patient satisfaction levels.

Abstract Objective: To measure the effect of computer-based outpatient prescription writing by internal medicine physicians on pharmacist work patterns.

Design: Work sampling at a hospital-based outpatient pharmacy. Data were collected from pharmacists wearing silent, random-signal generators before and after the implementation of computer-based prescribing.

Measurements: The type of work performed by pharmacists (activity), the reason for their work (function), and the people they contacted (contact) were measured.

Results: Total staff hours and prescriptions handled were similar before and after computer-based prescribing. Pharmacists recorded 4,687 observations before and 4,735 observations after implementation of computer-based outpatient prescription writing. After implementation, pharmacists spent 12.9 percent more time correcting prescription problems, had 3.9 percent less idle time, and spent 2.2 percent less time in discussions with others. Pharmacists also spent 34.0 percent less time filling prescriptions, 45.8 percent more time in problem-solving activities involving prescriptions, and 3.4 percent less time providing advice. Over 80 percent of pharmacist time was spent working alone both before and after computer-based outpatient prescription writing.

Conclusion: Computer-based prescribing results in major changes in the type of work done by hospital-based outpatient pharmacists and in the reason for their work and small changes in the people contacted during their work.

OBJECTIVE

To determine the incidence of major hemorrhage among outpatients started on warfarin therapy after the recommendation in 1986 for reduced-intensity anticoagulation therapy was made, and to identify baseline patient characteristics that predict those patients who will have a major hemorrhage.

DESIGN

Retrospective cohort study.

SETTING

A university-affiliated Veterans Affairs Medical Center.

PATIENTS

Five hundred seventy-nine patients who were discharged from the hospital after being started on warfarin therapy.

MEASUREMENTS AND MAIN RESULTS

The primary outcome variable was major hemorrhage. In our cohort of 579 patients, there were 40 first-time major hemorrhages with only one fatal bleed. The cumulative incidence was 7% at 1 year. The average monthly incidence of major hemorrhage was 0.82% during the first 3 months of treatment and decreased to 0.36% thereafter. Three independent predictors of major hemorrhage were identified: a history of alcohol abuse, chronic renal insufficiency, and a previous gastrointestinal bleed. Age, comorbidities, medications known to influence prothrombin levels, and baseline laboratory values were not associated with major hemorrhage.

CONCLUSIONS

The incidence of major hemorrhage in this population of outpatients treated with warfarin was lower than previous estimates of major hemorrhage measured before the recommendation for reduced-intensity anticoagulation therapy was made, but still higher than estimates reported from clinical trials. Alcohol abuse, chronic renal insufficiency, and a previous gastrointestinal bleed were associated with increased risk of major hemorrhage.

Abstract Objective: Errors of omission are a common cause of systems failures. Physicians often fail to order tests or treatments needed to monitor/ameliorate the effects of other tests or treatments. The authors hypothesized that automated, guideline-based reminders to physicians, provided as they wrote orders, could reduce these omissions.

Design: The study was performed on the inpatient general medicine ward of a public teaching hospital. Faculty and housestaff from the Indiana University School of Medicine, who used computer workstations to write orders, were randomized to intervention and control groups. As intervention physicians wrote orders for 1 of 87 selected tests or treatments, the computer suggested corollary orders needed to detect or ameliorate adverse reactions to the trigger orders. The physicians could accept or reject these suggestions.

Results: During the 6-month trial, reminders about corollary orders were presented to 48 intervention physicians and withheld from 41 control physicians. Intervention physicians ordered the suggested corollary orders in 46.3% of instances when they received a reminder, compared with 21.9% compliance by control physicians (p < 0.0001). Physicians discriminated in their acceptance of suggested orders, readily accepting some while rejecting others. There were one third fewer interventions initiated by pharmacists with physicians in the intervention than control groups.

Conclusion: This study demonstrates that physician workstations, linked to a comprehensive electronic medical record, can be an efficient means for decreasing errors of omissions and improving adherence to practice guidelines.

Abstract

Objective: To use routine data from a comprehensive electronic medical record system to predict death among patients with reactive airways disease.

Design: Retrospective cohort study conducted in an academic primary care internal medicine practice. Subjects were 1,536 adults with reactive airways disease: 542 with asthma and 994 with chronic obstructive pulmonary disease (COPD).

Measurements: The dependent variable was death from any cause within 3 years following patients' first primary care appointment in 1992. Multivariable logistic regression was used to identify independent predictors of 3-year mortality, with half of the patients used to derive the predictive model and the other half used to assess its predictability.

Results: Of the 1,536 study patients, 191 (12%) died in the 3-year follow-up period. From information available on or before patients' first primary care visit in 1992, multivariable predictors of 3-year mortality were coincidental heart failure, male sex, presence of COPD, lower weight, low serum albumin concentration level, and a prior arterial PO2 of less than 60 mmHg; use of an inhaled corticosteroid was protective. The c-statistic (ROC curve area) in the validation cohort was 0.76, indicating good discrimination, and goodness of fit was excellent by Hosmer-Lemeshow chi-square (P > 0.5). Only 24% of the patients in the validation cohort were designated at high risk (estimated ≥15% 3-year mortality), but this group contained more than half of the deaths within 3 years for the entire cohort.

Conclusions: Data generated during routine care and stored in a comprehensive electronic medical record can accurately predict mortality among patients with reactive airways disease. Such technology can be used by practices to control for severity of illness when assessing clinical practice and to identify high-risk patients for interventions to improve prognosis.

OBJECTIVE

Both physicians and patients view advance directives as important, yet discussions occur infrequently. We assessed differences and correlations between physicians’ and their patients’ desires for end-of-life care for themselves.

MEASUREMENTS AND MAIN RESULTS

Study physicians (n = 78) were residents and faculty practicing in an inner-city, academic primary care general internal medicine practice. Patients (n = 831) received primary care from these physicians and were either at least 75 or between 50 and 74 years of age, with selected morbid conditions. Physicians and patients completed identical questionnaires that included an assessment of their preferences for six specific treatments if they were terminally ill. There were significant differences between physicians' and patients' preferences for all six treatments (p < .0001), with physicians wanting less treatment than their patients for five of them. Patients desiring more care (p < .01) were more often male (odds ratio [OR] 1.7), African-American (OR 1.6), and older (OR 1.02 per year). There were no such correlates with physicians' preferences. A treatment preference score was calculated from respondents' desires to receive or refuse the six treatments. Physicians' scores were highly correlated with those of their enrolled primary care patients (r = .51, p < .0001).

CONCLUSIONS

Although patients and physicians as groups differ substantially in their preferences for end-of-life care, there was significant correlation between individual academic physicians' preferences and those of their primary care patients. Reasons for this correlation are unknown.

Background

The clinical relevance of human papillomavirus type 16 (HPV16) DNA methylation has not been well documented, although its role in modulation of viral transcription is recognized.

Methods

Study subjects were 211 women attending Planned Parenthood clinics in Western Washington for routine Papanicolaou screening who were HPV16 positive at the screening and/or subsequent colposcopy visit. Methylation of 11 CpG dinucleotides in the 3′ end of the long control region of the HPV16 genome was examined by sequencing the cloned polymerase chain reaction products. The association between risk of CIN2/3 and degree of CpG methylation was estimated using a logistic regression model.

Results

CIN2/3 was histologically confirmed in 94 (44.5%) of 211 HPV16 positive women. The likelihood of being diagnosed as CIN2/3 increased significantly with decreasing numbers of methylated CpGs (meCpGs) in the 3′ end of the long control region (Pfor trend = 0.003). After adjusting for HPV16 variants, number of HPV16-positive visits, current smoking status and lifetime number of male sex partners, the odds ratio for the association of CIN2/3 with ≥4 meCpGs was 0.31 (95% confidence interval, 0.12–0.79). The proportion of ≥4 meCpGs decreased appreciably as the severity of the cervical lesion increased (Pfor trend = 0.001). The inverse association remained similar when CIN3 was used as the clinical endpoint. Although not statistically significant, the ≥4 meCpGs-related risk reduction was more substantial among current, as compared to noncurrent, smokers.

Conclusion

Results suggest that degree of the viral genome methylation is related to the outcome of an HPV16 cervical infection.