Incompatibilities are persistent barriers when speciation happens with gene flow in Formica ants

A current goal of speciation research is identifying loci underlying reproductive barriers between species. Locating barrier loci in population genomic data is difficult due to the often complex demographic history of diverged taxa and heterogeneity in evolutionary forces across the genome. We take advantage of natural hybridization between two wood ant species (Formica aquilonia and F. polyctena) to identify regions of reduced long-term gene flow using demographically explicit scans of non-admixed genomes. In addition we identify candidate Bateson-Dobzhansky-Muller incompatibilities (BDMIs) through an imbalanced recombinant haplotype frequency analysis using a large sample of natural F. aquilonia × F. polyctena hybrid genomes. These approaches find barriers and BDMIs scattered across the genome. Furthermore, BDMIs significantly overlap with long-term barriers, indicating that some BDMIs have persisted despite divergence with gene flow. Intriguingly, the number of pairwise interactions a BDMI has correlates with its long-term barrier strength: hub-like BDMIs with many interactions reduce gene flow more effectively. Finally, we find that long-term barriers are depleted for both coding sequences (CDS) and transposable elements, while candidate BDMIs are associated with snRNAs and LTR transposons, specifically Ty1-copia. In contrast, regions where long-term barriers and BDMIs co-locate are significantly associated with introns but not CDS or TEs, implying a potential role of alternative splicing or gene regulation in long-term incompatibilities. Our results highlight the underappreciated impact of BDMI connectivity on the persistence of reproductive barriers.