Given the large discrepancies in the parameter estimates from the two studies, it is clear that the conclusions reached by at least one of the studies are incorrect. We examine the two possibilities in greater detail below.
If the estimates from the much larger Green et al. [2
] dataset are accurate, then the discrepancies in B and C either arose by chance (i.e., due to the uncertainty in the estimates from the much smaller Noonan et al. dataset) or due to some unknown bias or problem with the Noonan et al data. Under this scenario, either the modern European–Neanderthal split time is very recent (i.e., ≤60 Kya) or the Neanderthal admixture proportion is extremely high (>>50%). Paleoanthropological evidence suggests that Neanderthals formed a distinct group of fossils at least 250 Kya [6
], so the more recent modern European–Neanderthal split time is highly unlikely. In addition, most of the available evidence from paleoanthropology suggests that the Neanderthal contribution to the modern gene pool is limited [7
], while previous Neanderthal mtDNA studies concluded that the Neanderthal contribution could be no more than 25% [8
]. Furthermore, preliminary analyses of additional nuclear Neanderthal sequences suggest a much older human–Neanderthal sequence divergence time than was found by Green et al. [10
]. So, based on studies from the same [9
] and results from other laboratories [6
], it seems extremely improbable that the Green et al. estimates are accurate.
In contrast, if the Noonan et al. [1
] estimates were correct, then no additional assumptions would be needed to understand the Neanderthal nuclear DNA sequence data in the context of previous human evolutionary studies. This leads to consideration of three possible issues that may have compromised data quality in the Green et al. [2
] study: contamination with modern human DNA, widespread difficulties in aligning Neanderthal DNA fragments, and abnormally high DNA sequencing error rates.
Although Green et al. [2
] found little evidence of modern human mtDNA contamination, it is not clear whether this observation generalizes to the autosomal data under study. To examine this in greater detail, we divided the Green et al. sequence data into three groups: short (≥30 bp and ≤50 bp) fragments, medium-sized (>50 bp and ≤100 bp) fragments, and large (>100 bp) fragments (see Materials and Methods
). We then estimated the human–Neanderthal sequence divergence time for each of these groups. The likelihood of the data as a function of the divergence time is shown in . While the short fragments have an estimated divergence time similar to what was found in the Noonan et al. [1
] study, the large fragments are much more similar on average to modern human DNA. In fact, the large fragments have an estimated human–Neanderthal sequence divergence time that is less
than the estimated divergence time between two Hausa (West African) sequences (see Materials and Methods
). If true, this would indicate greater similarity between human and Neanderthal than between two extant members of the Hausa population. This pattern thus raises the concern that some of the longer sequence fragments are actually modern human contaminants. Modern human contamination would be expected to be size biased, since actual Neanderthal DNA would tend to be degraded into short fragments [1
]. We note that the observation of a length dependence of the results makes alignment issues alone [10
] unlikely to be a sufficient explanation, since we would expect that longer fragments would be easier to align and thus the data from longer fragments should be more accurate. We further note that we did not find a similar signal of potential contamination in the Noonan et al. data (unpublished data).
Relative Likelihood Curve for the Human–Neanderthal Divergence Time
We also tabulated the percentage of HapMap SNPs for which the Neanderthal sequence contains the derived allele for each of the three groups of Green et al. data (see ), and refer to this percentage as Nd
. For comparison, we also estimated (from simulations) the expected value of Nd
as a function of the European–Neanderthal population split time and the Neanderthal admixture proportion (see ). The expected value of Nd
increases as the European–Neanderthal population split time decreases and/or the Neanderthal admixture proportion increases, though for reasonable parameter values (i.e., a population split time ≥150 Kya and a Neanderthal admixture proportion <25%) Nd
is always <25%. In contrast, Nd
= 32.9 for the Green et al. data, which is inconsistent with the true value of Nd
being ≤25% (p
). Moreover, Nd
shows a clear trend of higher values for longer fragments, consistent with the hypothesis of widespread contamination with larger-sized modern human DNA fragments. The expected value of Nd
for modern human contaminants is 37.0, so the Green et al. data look in some ways more like modern human DNA than they do like Neanderthal DNA. If we use Nd
to estimate very roughly the proportion of the Green et al. [2
] data that are actually modern human contaminants, this leads to a contamination rate estimate of 73% (95% CI: 51%–97%; see Materials and Methods
). Alternatively, a likelihood-based estimate of the proportion of modern human contaminants yields an estimate of 78% (95% CI: 70%–88%; see Materials and Methods
). These estimates are very approximate, and given the uncertainty in the actual Nd
values for Neanderthal and modern human DNA, the data are also consistent with lower (but still substantial) levels of contamination.
Table 1 Nd Values for the Noonan et al.  and Green et al.  Datasets
Estimates of Nd as a Function of the Neanderthal Admixture Proportion and the Modern European–Neanderthal Population Split Time
On the other hand, if contamination were prevalent in the Green et al. [2
] data, then due to the large number of apparent mutations that appear in Neanderthal DNA due to post-mortem DNA damage [1
] the Neanderthal-specific sequence divergence should be smaller for the Green et al. data than it is for the Noonan et al. data. Instead, we find that the Neanderthal-specific divergences are almost the same for the two studies. It is not clear how to interpret this observation. It is clear that the two studies are inconsistent with each other, and given the weight of evidence from many previous studies, the more parsimonious explanation is still that something is wrong with the Green et al. [2
] data. Although we have highlighted strong circumstantial evidence of modern human DNA contamination in the Green et al. data, there are likely other important problems or biases affecting data quality in one or both studies. One possibility is that due to subtle differences in laboratory protocols, there is a higher sequencing error rate in the Green et al. data than in the Noonan et al. data. (Sequencing errors would look the same as Neanderthal-specific mutations in our analyses.) There is some indirect evidence of this—post-mortem DNA damage often causes the deamination of cytosine to uracil [12
], resulting in apparent C→T or G→A mutations. These particular mutations make up a significantly larger fraction of the Neanderthal-specific mutations from the Noonan et al. data than they do for the Green et al. data (χ2
, 1 degree of freedom; X2
= 21.2, p
), suggesting that some other process (besides post-mortem damage to Neanderthal DNA) is leading to the “Neanderthal-specific mutations” in the Green et al. data.
In conclusion, the sequencing of Neanderthal nuclear DNA is truly a remarkable technical achievement. However, because contamination with modern human DNA and sequencing error rates are continuing concerns, it will be important to carefully evaluate published and future data before arriving at any firm conclusions about human evolution.