On the impermanence of species: The collapse of genetic incompatibilities in hybridizing populations

Species pairs often become genetically incompatible during divergence, which is an important source of reproductive isolation. An idealized picture is often painted where incompatibility alleles accumulate and fix between diverging species. However, recent studies have shown both that incompatibilities can collapse with ongoing hybridization, and that incompatibility loci can be polymorphic within species. This paper suggests some general rules for the behavior of incompatibilities under hybridization. In particular, we argue that redundancy of genetic pathways can strongly affect the dynamics of intrinsic incompatibilities. Since fitness in genetically redundant systems is unaffected by introducing a few foreign alleles, higher redundancy decreases the stability of incompatibilities during hybridization, but also increases tolerance of incompatibility polymorphism within species. We use simulations and theories to show that this principle leads to two types of collapse: in redundant systems, exemplified by classical Dobzhansky-Muller incompatibilities, collapse is continuous and approaches a quasi-neutral polymorphism between broadly sympatric species, often as a result of isolation-by-distance. In nonredundant systems, exemplified by co-evolution among genetic elements, incompatibilities are often stable, but can collapse abruptly with spatial traveling waves. As both types are common, the proposed principle may be useful in understanding the abundance of genetic incompatibilities in natural populations.