Home | About | Journals | Submit | Contact Us | Français |

**|**HHS Author Manuscripts**|**PMC2890259

Formats

Article sections

- Summary
- 1. Introduction
- 2. Random Effects Model for Mixed Responses
- 3. Models for Bivariate Mixed Responses with Clustering
- 4. Example: EG Data
- 5. Discussion
- Supplementary Material
- References

Authors

Related links

Biometrics. Author manuscript; available in PMC 2010 June 23.

Published in final edited form as:

Published online 2009 May 7. doi: 10.1111/j.1541-0420.2008.01232.x

PMCID: PMC2890259

NIHMSID: NIHMS208180

The publisher's final edited version of this article is available at Biometrics

See other articles in PMC that cite the published article.

We consider analysis of clustered data with mixed bivariate responses, i.e., where each member of the cluster has a binary and a continuous outcome. We propose a new bivariate random effects model that induces associations among the binary outcomes within a cluster, among the continuous outcomes within a cluster, between a binary outcome and a continuous outcome from different subjects within a cluster, as well as the direct association between the binary and continuous outcomes within the same subject. For the ease of interpretations of the regression effects, the marginal model of the binary response probability integrated over the random effects preserves the logistic form and the marginal expectation of the continuous response preserves the linear form. We implement maximum likelihood estimation of our model parameters using standard software such as
`PROC NLMIXED of SAS`. Our simulation study demonstrates the robustness of our method with respect to the misspecification of the regression model as well as the random effects model. We illustrate our methodology by analyzing a developmental toxicity study of ethylene glycol in mice.

In many biomedical studies, the basic sampling unit is a cluster of subjects, such as siblings in a family or rats in a litter. Further, each member of the cluster may have multiple outcomes, including both discrete and continuous endpoints. When the bivariate responses are both continuous and approximately Gaussian, the traditional random effects linear model framework can be used for inference. However, the presence of both categorical/ordinal as well as continuous outcomes, called mixed responses, complicates the situation due to the lack of a discrete multivariate analog of the multivariate Gaussian distribution (Fitzmaurice and Laird, 1995). For example, the developmental toxicity study of ethylene glycol (EG), a high-volume industrial chemical (Price et al., 1985), involves measurements of both binary and continuous responses, namely, (a) fetal malformation and (b) fetal weight of 1028 fetuses from 94 litters. In this study, one of the four levels of EG was administered to 94 pregnant mice over the period of major organogenesis, beginning just after implantation. In particular, 25 pregnant mice received 0 g/kg/day, 24 received 0.75 g/kg/day, 22 received 1.5 g/kg/day, and 23 received 3 g/kg/day. We expect a within cluster (“litter”) association, along with the direct association between the fetal malformation (binary) and fetal weight (continuous) of same fetus. We are primarily interested in modeling the effects of dose on the binary and continuous outcomes, as interpretable regression functions, as well as modeling the within subject and within litter associations between the binary and continuous outcomes.

A number of joint models have been proposed for developmental toxicity studies involving both discrete and continuous responses from each member of a cluster (Geys, Molenberghs, and Ryan, 1999; Regan and Catalano, 1999; Dunson, Chen, and Harry, 2003; Gueorguieva and Agresti, 2001). Catalano and Ryan (1992) suggested a model assuming that the discrete outcome is a discretization of an unobserved continuous latent variable and this latent variable and the continuous response within same subject have a bivariate normal distribution. Joint distribution of the two responses is then formed by a product of marginal distribution of the continuous response and conditional distribution of the binary response conditioned on the continuous response. The other type of models consider conditioning on the binary response. Fitzmaurice and Laird (1995) assumed a logit link function for marginal probability of binary response and a normal distribution for the continuous response given the binary response. Cox and Wermuth (1992) compared different joint distribution models for analyzing data with quantitative and qualitative responses. For more discussions related to conditional models, see Little and Schluchter (1985), Cox and Wermuth (1994), Krzanowski (1988), and Schafer (1997).

The model proposed by Fitzmaurice and Laird (1995), henceforth FL, focuses on estimating and interpreting the marginal regression functions of two responses, while treating the association between the discrete and continuous responses as a nuisance. FL models have the attractive feature that the marginal regression parameters of both responses have good physical interpretations. However, this model only considers the cluster effect on the continuous response given the binary responses from all cluster members. In this model, the specification of the regression model conditional on the cluster-specific random effects is completely unknown. The magnitude of the marginal regression parameters, particularly for the binary response, gets attenuated from that of the conditional regression parameters due to the heterogeneity of clusters. This degree of attenuation is important for future study design as well as understanding the expected effect of covariates on other populations and on individual subjects.

In this article, we avoid the above shortcomings of FL modeling. We propose a model with practical physical interpretations of the marginal as well as conditional (given the cluster-specific random effects) regression functions associated with both discrete and continuous responses. We specify a bivariate distribution of two cluster-specific random effects—presented as separate random intercepts for two types of responses. We use a separate parameter to account for the direct within-subject association between the two responses. Hence, association between the discrete and continuous responses from different subjects within the same cluster is induced by the two correlated random effects. Association between the binary and continuous outcomes on the same subject is induced by correlations between the two within-cluster random effects, as well as the direct dependence of the continuous response on the binary response from the same subject. We use a logit link for the conditional regression function of the binary response given cluster-specific random effect, and an identity link for the regression function of the continuous response conditioned on the binary response and the cluster random effects. The joint distribution of the discrete and continuous responses given the random effects is expressed by the product of the marginal distribution of the binary outcome and the conditional distribution of the continuous response given the binary response. In the FL model, they considered maximum likelihood (henceforth ML) estimation as quite complicated and intractable, and hence they proposed the generalized estimating equations (GEE) method for computational convenience. Contrary to that, our likelihood is much simpler to evaluate and amenable to maximization through routine nonadaptive Gaussian quadrature techniques, because conditional on the random effects, all outcomes are independent.

One drawback of the random effects models for clustered mixed responses is that the marginal regression model of the binary response (after integrating out the random effects) usually is not of logistic form when the subject-specific model conditional on the random effect is of logistic form. In our model, the bridge distribution (Wang and Louis, 2003) of the random effect for the binary response allows the marginal probability of the binary response, integrated over the random intercept, to have a logistic structure with an odds ratio interpretation of the marginal regression effect. The regression parameters in the marginal logistic regression model are proportional to the corresponding regression parameters in the subject-specific conditional logistic model. This is a major advantage over other existing models including the FL model.

The rest of the article is organized as follows. In Section 2, we begin with our random effects model of clustered data with bivariate mixed responses within each subject. We illustrate how different levels of the association between two random effects induce different Kendall's *τ* values for association between binary and continuous responses from same cluster. In Section 3, we extend this conditional model of the clustered mixed responses to accommodate a direct within subject association between two responses (even after adjusting for the within cluster association). We also give the likelihood representation of our model. In Section 4, we demonstrate the ML estimation of the regression parameters of our models via analyzing the data from a developmental toxicity experiment of EG on mice. Finally, Section 5 discusses the need and advantages of our model. We also study the robustness of our method to model and random effect distribution misspecification through simulation studies.

For subject *j* = 1,*…*, *M _{i}* from cluster

For the time being, we assume that given the cluster-specific random effects (*B _{i}*,

$$\text{logit}(E[{Y}_{1\mathit{\text{ij}}}|{B}_{i},{W}_{i},{X}_{\mathit{\text{ij}}}])=\text{logit}\phantom{\rule{0.1em}{0ex}}({p}_{\mathit{\text{ij}}})={X}_{\mathit{\text{ij}}}^{\prime}\phantom{\rule{0.1em}{0ex}}\beta +{B}_{i},$$

(1)

where the marginal distribution of the random intercept *B _{i}* of binary response is a bridge distribution (Wang and Louis, 2003) with unknown parameter

$$f({B}_{i}|\phi )=\frac{1}{2\pi}\frac{\text{sin}\left(\phi \pi \right)}{\text{cosh}\left(\phi {B}_{i}\right)+\text{cos}\left(\phi \pi \right)}\phantom{\rule{2em}{0ex}}\left(-\infty <{B}_{i}<\infty \right).$$

The bridge density is symmetric (mean of *B _{i}* is 0), and the variance of

$$\text{logit}\phantom{\rule{0.1em}{0ex}}(E[{Y}_{1\mathit{\text{ij}}}|{X}_{\mathit{\text{ij}}}])=\text{logit}\phantom{\rule{0.1em}{0ex}}({\pi}_{1\mathit{\text{ij}}})={X}_{\mathit{\text{ij}}}^{\prime}(\phi \beta ).$$

(2)

Small *ϕ* indicates the important facts: high correlation of the binary responses within a cluster; high heterogeneity among clusters related to the binary response; and a corresponding high degree of attenuation of ** β** in the population due to cluster heterogeneity.

The continuous response *Y*_{2}* _{ij}* given the cluster random effect (

$$E\phantom{\rule{0.1em}{0ex}}[{Y}_{2\mathit{\text{ij}}}|{B}_{i},{W}_{i},{X}_{\mathit{\text{ij}}}]={X}_{\mathit{\text{ij}}}^{\prime}\alpha +{W}_{i},$$

(3)

and variance
${\mathit{\sigma}}_{0}^{2}$. The marginal distribution of the random intercept *W _{i}* of the continuous response is normal with mean 0 and variance
${\mathit{\sigma}}_{1}^{2}$. The marginal distribution of

$$E[{Y}_{2\mathit{\text{ij}}}|{X}_{\mathit{\text{ij}}}]={X}_{\mathit{\text{ij}}}^{\prime}\alpha .$$

(4)

In this model for the clustered mixed outcomes, both the conditional (on the cluster-specific random effects) and marginal regression functions in equations (1)–(4) have useful physical interpretations. As we describe below, the association between a binary and a continuous outcome within same cluster is induced by formulating a joint bivariate distribution for the cluster random effects (*B _{i}*,

To formulate the bivariate distribution for (*B _{i}*,

$${B}_{i}={F}_{b}^{-1}\left(\Phi \left({Z}_{1i}\right)\right)$$

(5)

where Φ(*·*) is the cumulative distribution function (c.d.f.) of standard normal, and
${F}_{b}^{-1}$ (*·*) is the inverse cumulative distribution

$${F}_{b}^{-1}\left(u\right)=\frac{1}{\phi}\text{log}\left[\frac{\text{sin}(\phi \pi u)}{\text{sin}\left\{\phi \pi (1-u\right\}}\right],$$

(6)

of the bridge density for 0 < *u* < 1. This assures that the marginal density of *B _{i}* will be the bridge distribution with c.d.f.

$${F}_{b}({B}_{i})=1-\frac{1}{\pi \phi}\left[\frac{\pi}{2}-\text{arctan}\left\{\frac{\text{exp}\left(\phi {B}_{i}\right)+\text{cos}\left(\phi \pi \right)}{\text{sin}\left(\phi \pi \right)}\right\}\right].$$

Because both the bridge cumulative and inverse cumulative distributions have closed-form expressions, they can easily be implemented in standard statistical software and simulations are easy to perform. In order to obtain
${W}_{i}\sim N(0,{\sigma}_{1}^{2})$, we let *W _{i}* =

In an extensive simulation study, we find the correlation between the two random intercepts *B _{i}* and

In Section 2, models (1) and (3) induce an exchangeable association structure among binary and continuous responses in the same cluster because given cluster-specific (*B _{i}*,

$$E[{Y}_{2\mathit{\text{ij}}}|{B}_{i,}{W}_{i,}{Y}_{1\mathit{\text{ij}},}{X}_{\mathit{\text{ij}}}]={X}_{\mathit{\text{ij}}}^{\prime}\alpha +\gamma ({Y}_{1\mathit{\text{ij}}}-{\pi}_{1\mathit{\text{ij}}})+{W}_{i,}$$

(7)

where
${\pi}_{1\mathit{\text{ij}}}=E[{Y}_{1\mathit{\text{ij}}}|{X}_{\mathit{\text{ij}}}]=\text{exp}\left\{{X}_{\mathit{\text{ij}}}^{\text{'}}(\phi \beta )\right\}/[1+\text{exp}\left\{{X}_{\mathit{\text{ij}}}^{\prime}(\phi \beta )\right\}]$, while keeping the regression model for *Y*_{1}* _{ij}* identical to that of equation (1). Hence, the association between

The conditional expectation of *Y*_{2}* _{ij}* integrated over

$$\begin{array}{cc}E[{Y}_{2\mathit{\text{ij}}}|{Y}_{1\mathit{\text{ij}},}{X}_{\mathit{\text{ij}}}]& =EE[{Y}_{2\mathit{\text{ij}}}|{W}_{i,}{Y}_{1\mathit{\text{ij}},}{X}_{\mathit{\text{ij}}}]\\ & ={X}_{\mathit{\text{ij}}}^{\prime}\alpha +\gamma ({Y}_{1\mathit{\text{ij}}}-{\pi}_{1\mathit{\text{ij}}})\end{array}$$

because *E*[*W _{i}* |

From equation (7), the covariance between responses *Y*_{1}* _{ij}* and

$$\text{Cov}\phantom{\rule{0.1em}{0ex}}[{Y}_{1\mathit{\text{ij}}},{Y}_{2\mathit{\text{ij}}}]=\gamma \text{Var}\phantom{\rule{0.1em}{0ex}}({Y}_{1\mathit{\text{ij}}})+\text{Cov}\phantom{\rule{0.1em}{0ex}}[{Y}_{1\mathit{\text{ij}}},{W}_{i}].$$

(8)

The covariance of two responses from different subjects *j* and *j′* within the same cluster *i* is

$$\text{Cov}\left[{Y}_{1\mathit{\text{ij}},}{Y}_{2\mathit{\text{ij}}\prime}\right]=\gamma \text{Cov}\left[{Y}_{1\mathit{\text{ij}},}{Y}_{1\mathit{\text{ij}}\prime}\right]+\text{Cov}\left[{Y}_{1\mathit{\text{ij}},}{W}_{i}\right].$$

(9)

where Var(*Y*_{1}* _{ij}*) =

$$\text{Cov}\phantom{\rule{0.1em}{0ex}}[{Y}_{1\mathit{\text{ij}},}{Y}_{1\mathit{\text{ij}}\prime}]={E}_{B}\left[\frac{\text{exp}({X}_{\mathit{\text{ij}}}^{\prime}\beta +{X}_{\mathit{\text{ij}}\prime}^{\prime}\beta +2{B}_{i})}{\{1+\text{exp}({X}_{\mathit{\text{ij}}}^{\prime}\beta +{B}_{i})\}\{1+\text{exp}({X}_{\mathit{\text{ij}}\prime}^{\prime}\beta +{B}_{i})\}}\right]-{\pi}_{1\mathit{\text{ij}}}{\pi}_{1\mathit{\text{ij}}\prime},$$

(10)

$$\text{Cov}\phantom{\rule{0.1em}{0ex}}\left[{Y}_{1\mathit{\text{ij}},}{W}_{i}\right]={E}_{BW}\phantom{\rule{0.1em}{0ex}}\left[\frac{\text{exp}({X}_{\mathit{\text{ij}}\prime}\beta +{B}_{i})}{1+\text{exp}({X}_{\mathit{\text{ij}}\prime}\beta +{B}_{i})}{W}_{i}\right],$$

(11)

where *E _{b}* and

Now, the likelihood contribution from cluster *i* is proportional to

$${E}_{BW}\phantom{\rule{0.2em}{0ex}}\left[\prod _{j=1}^{{M}_{i}}{f}_{{Y}_{1\mathit{\text{ij}}}|{B}_{i},{W}_{i},{X}_{\mathit{\text{ij}}}}({y}_{1\mathit{\text{ij}}}|{b}_{i},{w}_{i},{x}_{\mathit{\text{ij}}}){f}_{{Y}_{2\mathit{\text{ij}}}|{B}_{i},{W}_{i},{Y}_{1\mathit{\text{ij}}},{X}_{\mathit{\text{ij}}}}({y}_{2\mathit{\text{ij}}}|{b}_{i},{w}_{i},{y}_{1\mathit{\text{ij}}},{x}_{\mathit{\text{ij}}})\right],$$

where *f*_{Y1ij | Bi, Wi, Xij} (*y*_{1}* _{ij}* |

Unlike the FL model, our likelihood is tractable and maximization is straightforward. In order to use the ML method for estimation, the likelihood is obtained via taking the expectation with respect to the joint density of (*B _{i}*,

We apply the method described above to analyze data from the development toxicity study using 94 pregnant mice randomized to receive one of the four different doses of EG, 0, 0.75, 1.5, or 3 g/kg/day. Fetal malformation and fetal weight of 1028 fetuses from the 94 litters/clusters, with cluster size ranging from 1 to 16, were recorded. A summary of the fetal malformations and fetal weights is given in Table 1. We clearly observe that the proportion of malformation increases from 0.34% to 57.08% and sample mean of weight decreases from 0.972 to 0.704 as dose level increases from 0 to 3. For more details on the specifics of this data set, see Catalano and Ryan (1992), Fitzmaurice and Laird (1995), and also the monograph by Aerts et al. (2002).

Treating dose *X _{i}*(=

Table 2 shows the regression parameter estimates for the two responses from model 1. The positive estimate for the dose effect on fetal malformation and the negative estimate on fetal weight are consistent with what we observe from Table 1: the malformation rate increases and weight decreases as dose level increases. All of the marginal parameter estimates have slightly different point estimates than the FL model (Table 2 in Fitzmaurice and Laird, 1995), but the estimated standard errors of the parameter estimates are smaller than that of the FL model, especially for the binary response (approximately 20%), leading to tighter 95% confidence intervals.

The estimate of the attenuation parameter *ϕ* = 0.6618 confirms that heterogeneity of the clusters causes moderate attenuation of the estimated marginal dose effects. The intracluster correlation *ρ*_{Y1} between the binary responses is estimated by 1 − *ϕ* = 0.3382 (Wang and Louis, 2003). The intracluster correlation *ρ*_{Y2} between the continuous responses are estimated by the between-cluster variability divided by the sum of the within- and between-cluster variabilities
${\sigma}_{1}^{2}/\left({\sigma}_{0}^{2}+{\sigma}_{1}^{2}\right)=0.6262$. These estimates are somewhat greater than the estimates of the intracluster correlations obtained from the FL model. The negative value −0.6433 for estimate of association parameter *ρ* between the two cluster-specific random effects *B _{i}* and

In smoothed residual plots of model 1 obtained via the
`LOESS` procedure in
`SAS` (Cleveland, 1979), we observe some quadratic trends of dose for both the malformation and weight responses. We reanalyze the data by adding quadratic terms of dose effect for both responses (model 2). Estimates of the parameters of model 2 are in Table 2, where *ϕβ*_{2} and *α*_{2} are the marginal parameters of squared dose effect for the two responses. No further trends are detected from corresponding residual plots. The estimates of model 2 also suggest that the quadratic dose effect terms are necessary.

Our method allows fractional polynomial regression equations (Royston and Altman, 1994; Geys et al., 1999) for *E*(*Y*_{1}* _{ij}| X_{i}*) and

To see the impact of association between fetal malformation *Y*_{1}* _{ij}* and fetal weight

Using estimates from model 2 and expressions (8)–(11), we obtain from Monte Carlo simulation that, for dose level 0, the estimated correlation between two responses from the same fetus is $\widehat{\text{Corr}}\left[{Y}_{1\mathit{\text{ij}}},{Y}_{2\mathit{\text{ij}}}\right]=-0.1481$, and the estimated correlation between two responses from different fetuses within the same cluster is $\widehat{\text{Corr}}\left[{Y}_{1\mathit{\text{ij}}},{Y}_{2\mathit{\text{ij}}\prime}\right]=-0.1245$. This shows that the former correlation is approximately 19% higher than the latter correlation.

In this article, we have developed a regression model for bivariate discrete and continuous outcomes motivated by an application in developmental toxicology where the experiment measures the fetal malformation and fetal weight for different dose levels administered to different clusters. Our model is attractive in that it provides both conditional and marginal interpretation of the dose-response modeling, which happens to be the cornerstone of quantitative risk assessment. Similar to a generalized linear mixed model setup, the binary responses are related to the covariates through a logit link whereas the conditional distribution of the continuous responses given the binary responses are related to the covariates by an identity link. Our model can efficiently handle the effects of clustering through introduction of cluster-specific random effects in a regression setup, as well as the association between mixed outcomes on the same (different) fetus.

Our method of estimation is ML as opposed to the GEE methods adapted in the FL approach. With a correctly specified model, provided estimation techniques are computationally feasible, the ML method reigns supreme over any other estimation methods for its fidelity to the likelihood principle as well as for its asymptotic properties. There are clearly several advantages of our ML method over the GEE method used in the FL approach. Firstly, although ML estimation appears to be complicated in a clustered mixed responses setting, our model is easily amenable to ML estimation through routine optimization techniques readily available in standard software like
`SAS (v9.1)`. Secondly, ML estimates are asymptotically efficient (Lehmann and Casella, 1998). Thus, using our method we achieve smaller standard errors of the parameter estimates and consequently tighter confidence intervals over the estimates obtained by the GEE method. Thirdly, GEE methods provide only marginal models, thus we are unable to measure the attenuation of the dose effect to the population due to the heterogeneity induced by clustering (litter effect). Our model has both conditional and marginal interpretation in this context.

In theory, the only advantage GEE has over our full-likelihood approach is when the marginal regression model is correctly specified, but the full likelihood is misspecified; in this case, the GEE estimate of the marginal regression parameters will be asymptotically unbiased, but those from our full likelihood could be biased. To investigate the robustness of our ML approach, we performed small-scale simulation studies under misspecification of regression function as well as misspecification of the random effect distribution for binary response. Our current model (model 1) assumes that the parameter (*γ*) measuring the direct association between the two responses within the same subject does not depend on dose (or other covariates). We generate 50 data sets from a modification of this model with *γ* replaced by *γ*_{1} + *γ*_{2}*X _{i}* assuming

The authors are grateful for the support provided by grants ROI-CA69222 and ROI-AI60373 from the National Institutes of Health.

Supplementary Materials: Web Appendix derivations of equations (10) and (11) for *X _{ij}* = 0, as well as the EG data analyzed in Section 4 and an

- Aerts M, Geys H, Molenberghs G, Ryan LM. Topics in Modelling of Clustered Data. Boca Raton, Florida: Chapman and Hall; 2002.
- Catalano P, Ryan LM. Bivariate latent variable models for clustered discrete and continuous outcomes. Journal of the American Statistical Association. 1992;87:651–658.
- Cleveland WS. Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association. 1979;74:829–836.
- Cox DR, Wermuth N. Response models for mixed binary and quantitative variables. Biometrika. 1992;79:441–461.
- Cox DR, Wermuth N. Multivariate Dependencies: Models, Analysis and Interpretation. London: Chapman and Hall; 1994.
- Dunson DB, Chen Z, Harry J. A Bayesian approach for joint modeling of cluster size and subunit-specific outcomes. Biometrics. 2003;59:521–530. [PubMed]
- Fitzmaurice GM, Laird NM. Regression models for a bivariate discrete and continuous outcome with clustering. Journal of the American Statistical Association. 1995;90:845–852.
- Geys H, Molenberghs G, Ryan LM. Pseudolikelihood modeling of multivariate outcomes in developmental toxicology. Journal of the American Statistical Association. 1999;94:734–745.
- Gueorguieva R, Agresti A. A correlated probit model for joint modeling of clustered binary and continuous responses. Journal of the American Statistical Association. 2001;96:1102–1112.
- Hougaard P. Analysis of Multivariate Survival Data. New York: Springer; 2000.
- Krzanowski WJ. Principles of Multivariate Analysis. Oxford: Clarendon Press; 1988.
- Lehmann EL, Casella G. Theory of Point Estimation. 2nd edition. New York: Springer; 1998.
- Little RJA, Schluchter MD. Maximum likelihood estimation for mixed continuous and categorical data with missing values. Biometrika. 1985;72:497–512.
- Price CJ, Kimmel CA, Tyl RW, Marr MC. The developmental toxicity of ethylene glycol in rats and mice. Toxicological Applications in Pharmacology. 1985;81:113–127. [PubMed]
- Regan MM, Catalano PJ. Likelihood models for clustered binary and continuous outcomes: Applications to developmental toxicology. Biometrics. 1999;55:760–768. [PubMed]
- Royston P, Altman DG. Regression using fractional polynomials of continuous covariates: Parsimonious parametric modelling. Applied Statistics. 1994;43:429–467.
- Schafer JL. Analysis of Incomplete Multivariate Data. London: Chapman and Hall; 1997.
- Wang Z, Louis TA. Matching conditional and marginal shapes in binary mixed-effects models using a bridge distribution function. Biometrika. 2003;90:765–775.

PubMed Central Canada is a service of the Canadian Institutes of Health Research (CIHR) working in partnership with the National Research Council's national science library in cooperation with the National Center for Biotechnology Information at the U.S. National Library of Medicine(NCBI/NLM). It includes content provided to the PubMed Central International archive by participating publishers. |