The standard argument for why there is aversion to incest among humans as matter of innate disposition is the Westermarck effect, which is a model where aversion to mating emerges if you are raised with an individual of the opposite sex. Some basic illustrations are sketched out in The Blank Slate: The Modern Denial of Human Nature. But some comments below make me wonder if there is are alternative explanations. Robin Fox has made the claim, repeated in many places, that cousin marriage was ubiquitous in the human past:
As an anthropologist I am forced to face the fact that for the vast majority of our existence as a species close cousin marriage must have been the norm, if for no other reason than that most of the time there was no one but cousins to marry. Indeed I have spent much of my professional life analyzing the complexities of systems of marriage that not only allowed but insisted on cousin marriage by rule. Not only was it not forbidden, it was prescribed, often with a particular degree of detail. You were enjoined, for example, to marry your mother's brother's daughter but not your father's sister's daughter, or required to marry a mother's father's sister's daughter's daughter, and forbidden to marry a father's father's sister's daughter's daughter. The details don't matter. What matters is that in small-scale societies with low mobility, spouses were drawn from a pool of close relatives. Marriage relationships once set up were perpetuated over the generations by the rules of cousin marriage. Even in nomadic societies like the ancestors of the Semitic people, marriage with a close cousin was prescribed. The ideal marriage was between the children of two brothers, and this remains both the norm and the practice in Arab and Muslim societies today.
I'm moderately skeptical because of the problems with pedigree collapse. Additionally, I don't recall the South African Bushman genome had a much lower mutational load than the other samples, which would be the case of recessive alleles were always being exposed in "small-scale societies." In fact, cousin marriage seems to increase in some societies, such as Saudi Arabia and Pakistani, with modernity (larger families due to modern health results in more marriageable cousins). Let's grant Fox's anthropological observation. I think there are some genetic problems with this. How we do resolve the two? The comments below got me thinking in functional terms:
perhaps groups which promoted consanginuous practices successfully to leverage short-term gains of cohesion tended to go extinct in the long-term because of a mutational meltdown event?
This can explain Fox's observation of the ubiquity of endogamy, and yet still avoid the problems which repeated generations of cousin marriage tend to produce genetically. Human history may have been a balance between the cultural benefits of establishing tighter kin relations through intra-familial/clan marriage, and the genetic benefits of outbreeding. This is analogous to the macroevolutionary patterns with sexual vs. asexual lineages. The latter tend to be found near the "tips" of phylogenies as derived lineages; a strong clue that they are ephemeral, and tend toward extinction. Yet at any given time asexuality can seem common because it is an effective short-term strategy. Addendum: Just to be clear, I am implying here there may be multiple reasons for incest aversion. A functional model regarding the balance between genetic and anthropological factors would operate at the level of the group and meta-populations. But within the group there might also be individual level tensions, responsible for the Westermarck effect.
