I've been chewing on the modern human range expansion into Neandertal territory paper for a few days now. But I haven't been able to bring myself to say much. There are two reasons. First, it's a simulation paper, and I don't exactly know what I can say besides being skeptical of the plausibility of some of their results and their assumptions, unless I bother to replicate their simulations. There's something of a "black-box" aspect from the outside operationally in the case of these sorts of research. Second, Ed Yong has boiled down the paper to its essence rather well, while John Hawks and Dienekes have offered their critiques. Dienekes and John get at one of my gnawing worries about all these sorts of models about deep history. Here's John:
Here's how I currently see those problems. Europeans today are not the Europeans of the past. They have undergone massive population movements and replacements since the initial Upper Paleolithic people encountered Neandertals. That's not only the result of archaeology, it's also clear from the paleogenetics. If we recognize this subsequent history, then we will find it easy to explain why the rest of the population outside Africa has basically the same small amount of Neandertal ancestry: they received a massive influx of genes from some West Asian population with Neandertal mixture. Europe also got these genes, mostly long after the initial Upper Paleolithic.
This is an issue for Dienekes as well. An author on the paper responded, citing their own 2005 work and repeating that they "believe that current Europeans are mainly descending from Paleolithic people." Since 2005 I think the needle has moved far more toward the idea of non-continuity between Paleolithic and post-Neolithic Europeans, though the debate is roiling with no hard and fast resolution (I go back and forth, balancing contradictory data). As I observe above, the simulation is what it is. There's no verbal slight of hand here, the assumptions and the inferences are there for all to see. But not to bore you to death with this: but you can find simulation papers which argue that there is robust evidence for total replacement of "archaic" humans from just a few years back. Obviously there needed to be some fine-tuning...or so we assume now that our expectations have been shifted by sequencing of subfossil remains. For me one aspect that jumps out is the wave of advance and diffusion characteristic of the spread of human populations. This makes some sense intuitively, but the historical record is also awash with "pulses" and sweeps. The reasons for these bursts may be endogenous (e.g., social cycles driven by demographic waves) or exogenous (e.g., innovation) to demographic systems, but they exist nonetheless. Migration to the New World of Europeans was actually rather marginal in numbers until ~1750, when the Scots arrived from the Borders and Ireland (though fertility in New England was very high, resulting in natural increase without much migration). Later, a "baby boom" in New England resulted in a massive expansion out across the Great Lakes and Upper Midwest in the early 19th century (this is coincident with the Second Great Awakening). These periods of demographic tumult were framed by relative quiescence. Though ancient people were very different, were they that different? For example, did the Greek colonization of the Mediterranean littoral after 800 BC exhibit diffusion? No. Rather, it was a burst and leapfrog. In other words, a static situation was quickly flipped to migratory dynamism, and that migration was not just diffusive, but manifested in point-to-point "leaps" toward favorable locations. It may be that in the aggregate this sort of dynamic "averages" out in these explicit simulations of admixture between two populations. But I'm not sure so. Especially if there are ecological biases in migration which differentiate hominin lineages which are naturally not randomly distributed across the spatial landscape. More generally there is the problem that we know many "probable" things about the human past which don't cohere well into one systematic whole. I say probable without quotes because one person's probably is another person's possible, and one person's plausible is another person's crankery. There are a whole set of propositions which I can see someone assenting to at a 0.75 probability...but the joint likelihood is 0.00 because of contradictions. Some of easiest to handle issues have to do with chronology. Dienekes observes:
Second, the observation of Fst values between European and West Asian populations are approximately 1/3 of those between West and East Eurasians. If the separation between Europeans and West Asians was effected close to the time of the Upper Paleolithic colonization of Europe (40ky), and would thus have diverged genetically only slightly less than West Eurasians and East Asians have.
There is the confound that gene flow between Europe and West Asia could prevent excessive divergence, but the broader issue is that the times of settlement across Eurasia and Oceania for anatomically modern humans has a very narrow window. Australia was settled 40-50 years ago, Europe 30-40 thousand years ago, and the rest of Eurasia presumably a little earlier. These are very deep time depths. Is it plausible that later demographic expansions have not obscured previous relationships and patterns which were extant when archaic and modern humans interacted? At this point I'd say no. Which makes explicit axiom based computational models mostly good in terms of being open and amenable to skepticism because of their clarity. I just don't think we have the assumptions nailed down nearly as well as is necessary for these models. Otherwise there'd be no real reason for genomic analysis of ancient DNA besides filling in gaps on the margins.