Home | About | Journals | Submit | Contact Us | Français |

**|**HHS Author Manuscripts**|**PMC5736152

Formats

Article sections

- Abstract
- 1. Introduction
- 2. Simulation Studies: Validation of the beta-binomial model
- 3. Comparisons of the MLE to the MME
- 4. Discussion
- Supplementary Material
- References

Authors

Related links

Commun Stat Simul Comput. Author manuscript; available in PMC 2018 January 1.

Published in final edited form as:

Commun Stat Simul Comput. 2017; 46(2): 807–814.

Published online 2016 November 11. doi: 10.1080/03610918.2014.960091PMCID: PMC5736152

NIHMSID: NIHMS900341

The beta-binomial model has been widely used as an analytically tractable
alternative that captures the overdispersion of an intra-correlated, binomial
random variable, *X*. However, the model validation for
*X* has been rarely investigated. As a beta-binomial mass
function takes on a few different shapes, the model validation is examined for
each of the classified shapes in this paper. Further, the mean square error
(MSE) is illustrated for each shape by the maximum likelihood estimator (MLE)
based on a beta-binomial model approach and the method of moments estimator
(MME) in order to gauge when and how much the MLE is biased.

The beta-binomial model has been widely used as an analytically tractable
alternative that captures the overdispersion of a binomial random variable,
*X*, which is a sum of Bernoulli random variables with success
probability *p* and intraclass correlation
*ρ*, since Skellam
(1948). When *p* is assumed to follow a beta distribution,
*p* ~ Beta (*α*,
*β*) and then unconditional on *p*, the
*X* follows a beta-binomial distribution (BBD). Here, we denote
this as *Z*. Similar to a beta distribution, a probability mass
function of *Z* takes on a few different shapes: for example,
Bell-shape, J-shape, inverse J-shape, and U-shape/Bimodal. Although the success
probability and intraclass correlation of *Z* are identical to those
of *X*, there may exist significant discrepancy in distribution
between them as the bimodal shape is more likely to reflect the intraclass
correlation between Bernoulli random variables within *X*. In such
cases, inference based on the beta-binomial model approach is biased.

A few methods have been developed for the goodness-of-fit (GOF) test of a
beta-binomial model (Brooks *et. al*,
1997 and Garren *et. al*,
2000). However, they have been rarely used to investigate the adequacy of
the beta-binomial model for *X*, which may or may not follow a BBD.
In this paper the model validation for *X* is examined. Specifically,
we investigate the model validation for each of the classified shapes through the
simulation studies in Section 2, along with a real application for screening
mammography data. The mean square error (MSE) is illustrated for each shape by the
maximum likelihood estimator (MLE) based on a beta-binomial model approach and the
method of moments estimator (MME) in order to gauge when and how much the MLE is
biased in Section 3. The paper concludes with final comments in Section 4.

Consider a discrete random variable *X _{i}*
{0, 1, …,

Suppose that conditional on *P* =
*p*, a discrete random variable
*Z _{i}* {0, 1, …,

$${f}_{z}({z}_{i}|{n}_{i},\alpha ,\beta )=(\begin{array}{c}{n}_{i}\\ {z}_{i}\end{array})\frac{B(\alpha +{z}_{i},{n}_{i}-{z}_{i}+\beta )}{B(\alpha ,\beta )},$$

where *B*(*α*,
*β*) is a beta function. The mean and variance of
*Z _{i}* are

$$\mathrm{E}({Z}_{i})=\frac{{n}_{i}\alpha}{\alpha +\beta}={n}_{i}\mu \phantom{\rule{0.2em}{0ex}}\text{and}\phantom{\rule{0.2em}{0ex}}\text{Var}({Z}_{i})={n}_{i}\mu (1-\mu )\{1+\varphi ({n}_{i}-1)\},$$

where *μ* =
*α*/(*α* +
*β*) and *ϕ* =
1/(*α* + *β* +
1). Letting *μ* = *p* and
*ϕ* = *ρ*, a
beta-binomial model has been used as an analytically tractable alternative to
the binomial that captures the overdispersion of
*X _{i}*.

Depending on *α* and *β*,
the probability mass function of *Z _{i}* can be largely
classified into Bell-shape, J-shape, inverse J-shape, and U-shape/Bimodal as
depicted in Figure 1. In simulation
studies, 40 sets of parameters (

For simulation studies, the Oman and
Zucker method (2001) is employed to generate intra-correlated
binomial data, *X _{i}*, because their method
doesn't require any assumption on a distribution for

Table 1 shows the descriptive
statistics of p-values by the Pearson's GOF test using a Monte Carlo run
of 100,000 simulations for *ρ* = 0.01 ~
0.3, *m* = *n* = 50. The results
of *ρ* = 0.5 are similar to that of
*ρ* = 0.3 and are omitted in Table 1. Q1, Q3, and “PP” indicate
25* ^{th}* and 75

Descriptive statistics of p-values by the Pearson's GOF test using a
Monte Carlo run of 100,000 simulations when *m* =
*n* = 50.

With *n* = 50, four sets of parameters are
selected for visual evaluation, depending on shape of the BBD; (A) Bell-shape
with small correlation (*μ* = 0.5,
*ϕ* = 0.01), (B) Bell-shape with moderate
correlation (*μ* = 0.5,
*ϕ* = 0.1), (C) J-shape
(*μ* = 0.75, *ϕ*
= 1/3), (D) U-shape (*μ* = 0.5,
*ϕ* = 0.4). Figure 2 depicts the comparisons of the distribution of
*X _{i}* to the binomial and the beta-binomial
mass functions. The beta-binomial model fits well for the small value of
correlation (Figure 2 (A)), while for the
other three cases (Figure 2 (B) –
(D)) substantial discrepancy is observed. The shapes of

The model validation for the data of a screening mammogram study (Beam *et al.*, 2003) is
considered as a real example. In their study, 64 out of 148 films were used for
estimating sensitivity while 84 were used for specificity. One hundred eight
radiologists each read the 148 mammography films (i.e.,
*m*=108, *n*=64 for sensitivity
and *n*=84 for specificity). The distributions of the
total number of positive mammography readings out of confirmed cancer cases and
negative readings out of non-cases are depicted and compared to the binomial and
the beta-binomial mass functions in Figure 3 (A)
and (B), respectively. The parameters of the BBD are estimated by the
MLE (denoted as and
) developed by Kim and Lee (2013). It is shown that the data are
overdispersed, and the beta-binomial model provides a superior fit to the data,
compared to the binomial model (the p-values by the Pearson's GOF test
are < 0.001). The estimates of sensitivity and intraclass correlation,
and ,
are 0.874 and 0.054, respectively, and the model fits well (p-value by the GOF
test = 0.659). For specificity, and
are 0.786 and 0.086, and no significant
difference between the distribution of negatives to non-cases and a BBD is found
(p-value = 0.394) but it appears that the shape of the specificity data
is slightly bimodal. Thus, these results support the simulation study results
presented in Table 1 and Figure 2.

Based on the first and second moments of *X _{i}*, a
method of moments estimator (MME; denoted as and
) for

$$\stackrel{\sim}{p}=\frac{1}{m}\sum _{i=1}^{m}{X}_{i}/n,\phantom{\rule{0.2em}{0ex}}\stackrel{\sim}{\rho}=\frac{1}{n-1}[\frac{n{S}_{m-1}^{2}}{\stackrel{\sim}{p}(1-\stackrel{\sim}{p})}-1],\phantom{\rule{0.2em}{0ex}}{S}_{m-1}^{2}=\frac{1}{m-1}\sum _{i=1}^{m}{({X}_{i}/n-\stackrel{\sim}{p})}^{2},$$

(1)

can be considered as a distribution-free point estimator as no particular
distribution for *X _{i}* is assumed. In simulation studies,
the MLE is used to estimate

The comparisons between the MLE and the MME are implemented through a Monte
Carlo run of 20,000 simulations with *m* = 25, 50, 100, and
*n* = 25, 50, 100. Table
2 illustrates the mean and the mean squared error (MSE) for each
estimator when *m* = *n* = 25, 50, and
100. The results of other *n* and *m* are similar to
Table 2 (results are omitted). The MSE
decreases with the number of samples, *m*, and the number of
Bernoulli variables in each sample, *n*. As Kleinman (1973) pointed out, the MLE is more efficient
than the MME when the MLE is unbiased or very close to the true parameter value. The
simulation study results show that (i) the distribution-free MME are more reliable
and robust across all regions in Figure 1, (ii)
the MLE for success probability, , is slightly
biased for (*p*, *ρ*) = (0.7, 0.5) and
(0.9, 0.5), (iii) the MLE for intraclass correlation,
, is significantly biased except when
*ρ* is very small. Specifically,
is underestimated for parameters which fall
into or get close to regions II (J-/Inverse J-shape) or III (Bell-/U-shape) in Figure 1. A 15% or more biased estimate,
compared to the true parameter value (i.e., (estimate - true value)/ (true value)
> 0.15) is highlighted with boldface in Table 2. Supplemental
Figures B1 and B2 illustrate the histogram of the MLE,
, for *ρ* =
0.05, 0.1, 0.3, and 0.5 using a Monte Carlo run of 20,000 when *m*
= *n* = 100 and *p* = 0.5,
0.7, and 0.9.

In this paper, the beta-binomial model validation is examined using the
random samples *X _{i}* which are generated by the Oman and
Zucker method. The discrepancy between the distribution of

In summary, we would recommend that both the Pearson's GOF test and
the distribution-free estimate be conducted prior to making parametric inference
under the beta-binomial model. Specifically, if the distribution-free estimate for
*ρ*, , is greater than
0.1, the GOF test should be considered to investigate if there is a significant
evidence that the beta-binomial model poorly fits the data, which could result in
biased inference. When the beta-binomial model is not valid for a given data, one
may consider an alternative model including the negative binomial model. However,
finding a valid model should be differentiated for the given data along with solid
model validation process.

Supplemental Figure A. Histograms of p-values by the
Pearson's GOF test using a Monte Carlo run of 100,000 simulations
when *m* = *n* = 50,
*p* =0.5 (top) ~ 0.9 (bottom 3 panels),
and *ρ* =0.01 (left), 0.05 (middle), and
0.075 (right 5 panels)

Supplemental Figure B1. Histogram of the MLE,
, for *ρ*
= 0.05 (left) and 0.1 (right panels) using a Monte Carlo run of
20,000 simulation when *m* = *n*
= 100 and *p* =0.5 (top), 0.7 (middle), and
0.9 (bottom panels).

Supplemental Figure B2. Histogram of the MLE,
, for *ρ*
= 0.3 (left) and 0.5 (right panels) using a Monte Carlo run of
20,000 simulation when *m* = *n*
= 100 and *p* =0.5 (top), 0.7 (middle), and
0.9 (bottom panels).

Click here to view.^{(57K, pdf)}

The authors expresses many thanks to Dr. Craig Beam for allowing the use of the screening mammogram data. This work was partially supported by NIH/NCI grant 5P30CA076292-16.

- Beam C, Conant E, Sickles E. Association of volume and volume-independent factors with accuracy in screening mammogram interpretation. Journal of the National Cancer Institute. 2003;95:282–290. [PubMed]
- Brooks S, Morgan B, Ridout M, Pack S. Finite mixture models for proportions. Biometrics. 1997;53:1097–1115. [PubMed]
- Chatfield C, Goodhardt G. The Beta-Binomial Model for Consumer Purchasing Behavior. Applied Statistics. 1970;19:240–250.
- Garren S, Simth R, Piegorsh W. On a likelihood-baed goodness-of-fit test of the beta-binomial model. Biometrics. 2000;56:947–950. [PubMed]
- Kim J, Lee JH. Simultaneous confidence intervals for a success probability and intraclass correlation, with an application to screening mammography. Biometrical Journal. 2013;55(6):944–954. [PubMed]
- Kleinman JC. Proportions with extraneous variance: single and independent sample. Journal of the American Statistical Association. 1973;68:46–54.
- Oman SD, Zucker DM. Modelling and generating correlated binary variables. Biometrika. 2001;88:287–290.
- Skellam JG. A probability distribution derived from the Binomial distribution by regarding the probability of success as variable between the sets of trials. Journal of the Royal Statistical Society, B. 1948;10:257–261.

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's national science library in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |