|Home | About | Journals | Submit | Contact Us | Français|
With the analysis of complex, messy data sets, the statistics community has recently focused attention on “reproducible research,” namely research that can be readily replicated by others. One standard that has been proposed is the availability of data sets and computer code. However, in some situations, raw data cannot be disseminated for reasons of confidentiality or because the data are so messy as to make dissemination impractical. For one such situation, we propose 2 steps for reproducible research: (i) presentation of a table of data and (ii) presentation of a formula to estimate key quantities from the table of data. We illustrate this strategy in the analysis of data from the Prostate Cancer Prevention Trial, which investigated the effect of the drug finasteride versus placebo on the period prevalence of prostate cancer. With such an important result at stake, a transparent analysis was important.
The Prostate Cancer Prevention Trial (PCPT) investigated the effect of finasteride versus placebo on the period prevalence of prostate cancer determined by biopsy (Thompson and others, 2003). The PCPT randomized 18 882 men to receive either placebo or finasteride over 7 years. At annual intervals, men in both arms were scheduled to receive tests for prostate-specific antigen (PSA) and also a digital rectal examination (DRE). If PSA was elevated or DRE indicated an abnormal finding, a biopsy was recommended. A biopsy was also offered at the end of the study to all living participants who had not been previously diagnosed with prostate cancer. A nonrandom sample of participants with prostate cancer on biopsy received surgery. The original endpoint was prostate cancer on biopsy. However, due to problems with interpreting this endpoint, it was later decided that a more relevant endpoint was prostate cancer determined on surgery and classified as either low grade (less severe) or high grade (more severe) based on a Gleason score (GS). The analysis is complicated because endpoint data were missing in 2 stages, biopsy and surgery following biopsy, both of which depended on whether or not biopsy was recommended and the grade on biopsy.
Three recent papers proposed different ways to estimate the effect of finasteride on high-grade prostate cancer ascertained by surgery in the presence of missing data (Pinsky and others, 2008; Redman and others, 2008; Shepherd and others, 2008). However, none of these papers was conducive to reproducibility. A reproducible and transparent analysis is important because the data are very complex and the results could affect a large number of men who currently receive finasteride for benign conditions.
One standard for reproducibility is the availability of data sets and computer code (Peng, 2009). However, due to confidentiality considerations and the messiness of the data, it was not feasible to disseminate the raw data. Instead, we chose another approach to reproducible research, namely presenting tables of counts and reporting simple closed-form maximum likelihood estimates. Please also see the supplementary material available at Biostatistics online for additional clinical background and important documentation for reproducible research on definitions, classification of subjects, derivations, and calculations.
The data involve 3 types of counts:
Because persons who had surgery must have had prostate cancer on biopsy and hence had to be diagnosed with prostate cancer at surgery, kxayd is only defined for y ≠ 0 and d ≠ 0. Investigators were interested in 2 possible definitions for high-grade prostate cancer, either a GS greater than or equal to 7 or a GS greater than or equal to 8 (Tables 1 and and22).
Data from PCPT using GS greater than or equal to 7 as the definition of high-grade prostate cancer
Data from PCPT using GS greater than or equal to 8 as the definition of high-grade prostate cancer
We analyzed the data in the tables using a transparent approach involving closed-form maximum likelihood estimates. The parameter of interest is the probability of “definitive” prostate cancer outcome d conditional on randomization group x,
Other parameters are the probability of cancer outcome y on biopsy conditional on randomization group x and definitive prostate cancer outcome d,
and the probability of biopsy recommendation a conditional on randomization group x, biopsy outcome y, and definitive prostate cancer outcome d,
We make 2 reasonable assumptions.
The probability of missing a biopsy outcome depends on randomization group x and biopsy recommendation a, but not biopsy outcome y, namely πxa = pr(MY = 0|x,a) = pr(MY = 0|x,a,y).
The probability of missing a surgery outcome following a biopsy depends on randomization group x, biopsy recommendation a, and outcome of biopsy y, but not the outcome of the surgery d, namely γxay = pr(MD = 0|x,a,y,MY = 1) = pr(MD = 0|x,a,y,d,MY = 1).
Because the model is saturated, we computed maximum likelihood estimates by setting observed cell counts equal to their expected values. Let “+” in a subscript denote summation. The maximum likelihood estimate of the relative risk (RR) for finasteride versus placebo group of high-grade prostate cancer (d = 2) is
The estimated variance of RR was computed using the multinomial–Poisson transformation (Baker, 1994). We also computed estimates after adjusting for baseline variables using propensity-to-be-missing scores (Baker and others, 2006).
Based on the results in Table 3, we draw 2 clinically important conclusions. First, finasteride likely lowers the risk of high-grade prostate cancer defined as a GS of 7 or greater, with a point estimate indicating a lower risk under finasteride and an upper bound of the 95% confidence interval indicating a very slight increase in risk (hence borderline statistical significance). Second, finasteride possibly increases the risk of high-grade prostate cancer defined as a GS of 8 or greater, with a point estimate indicating higher risk under finasteride but a lower bound of the 95% confidence interval indicating a substantial decrease in risk (hence statistically not significant). The main caveats, shared by all analyses of the PCPT data, are unmeasured confounders, misclassification of data, and lack of a definitive prostate cancer mortality endpoint. Although the results are qualitatively similar to previous results, it does not diminish the clinical importance of this analysis. Because our basic analysis is transparent and reproducible, it should carry extra weight with the clinical community.
Estimated RRs for high-grade prostate cancer for finasteride versus placebo (95% confidence intervals)
National Cancer Institute, Division of Cancer Prevention (5U10CA037429-25); Southwest Oncology Group CCOP Research Base, PCPT/PCPT LTFU, and SELECT Programs; Division of Cancer Prevention in the National Cancer Institute and the National Institutes of Health.
We thank Ian Thompson, the principal investigator for PCPT, for facilitating our use of the data, Cathy Tangen and Phyllis Goodman for assistance in processing the data, and Mary Redman for helpful comments. Conflict of Interest: None declared.