Genetic analyses provide information on the numbers and kinds of genetic changes underlying reproductive barriers, as well as on the evolutionary forces responsible for their origin. Studies of pollinator isolation have shown, for example, that major quantitative trait loci (QTLs) sometimes underlie shifts in the animals that pollinate plants (pollination syndrome) (table S1) and changes in pollinator preferences in the field (). In contrast, two studies of mating system isolation detected many smaller genetic changes (table S1). These different architectures might be explained by the fact that many intermediate pollinator syndromes are maladaptive (e.g., red flowers lacking a nectar reward are unattractive to both birds and bees) and favor larger genetic steps, whereas small increases in selfing rates may be favored if inbreeding depression costs are not prohibitive (14
). Analyses of the direction of QTL effects imply that most traits contributing to prepollination isolation diverged through directional selection; as predicted for adaptive phenotypes, QTL effects for these traits are mostly in the same direction as the parental differences (15
). QTL effects are predicted to vary in direction (i.e., have opposing effects) for traits not under consistent directional selection (16
Recent genetic analyses of prezygotic and extrinsic postzygotic barriers associated with discrete habitat differences are particularly informative because many of the studies have been performed in the field. This makes it possible to estimate the strength of selection on traits and QTLs that contribute to habitat isolation. Studies have shown, for example, that the strength of selection on QTLs that contribute to habitat isolation is sufficient to permit speciation in the presence of gene flow, that hybrid inviability may arise as a by-product of habitat selection, and that interspecific hybridization can facilitate the exchange of adaptive alleles between species (table S1).
Genetic studies of postpollination, prezygotic isolation have focused on the relationship between self-incompatibility (SI) mechanisms, which enforce outcrossing in many hermaphroditic plants, and interspecific incompatibility. This interest stems from early observations that self-compatible species are more compatible in interspecific crosses than are SI species, implying that SI may contribute to both intra- and interspecific incompatibilities. This was confirmed by detection of a QTL for interspecific incompatibility that colocalizes with the SI locus, as well as observations that crosses between self-compatible species fail after transformation with a SI gene from a self-incompatible species (table S1). Diversification of genes that contribute to SI appears to result from frequency-dependent selection (17
). Interestingly, other plant reproductive proteins appear to be under positive selection as well, including candidates for species-specific recognition between pollen and stigma (table S1).
Intrinsic postzygotic barriers offer special challenges to genetic analyses because the phenotypes of interest (hybrid sterility and inviability) impede genetic study and lack obvious candidate genes for functional analyses (see below, however). Intrinsic postzygotic isolation may be caused by chromosomal rearrangements and/or changes in genes (). Population genetic theory minimizes the importance of strongly underdominant chromosomal rearrangements (those that reduce the fitness of heterozygotes) because their negative effects on fitness should prevent them from becoming established, except in small, inbred populations. Weakly underdominant rearrangements are more easily established but contribute little to reproductive isolation. In contrast, the Bateson-Dobzhansky-Muller (BDM) model accounts for the accumulation of interspecific incompatibilities in genes without loss of fitness (). Briefly, as a lineage diverges, geographically isolated or neighboring allopatric populations may accumulate independent mutations. These mutations are compatible with the ancestral genotype but are incompatible when combined. BDM incompatibilities generally involve two or more loci, although it is theoretically possible for BDM incompatibilities to result from the allopatric accumulation of independent mutations at a single locus ().
Fig. 1 Genetics of hybrid incompatibilities. (A) Example of a typical chromosomal rearrangement in plants, showing loss of fertility in heterozygotes because 50% of gametes are unbalanced genetically and inviable. (B) Classic two-locus BDM incompatibility in (more ...)
Despite theoretical doubts about their importance in speciation, chromosomal rearrangements often contribute to the sterility of hybrid plants (18
). Unlike Drosophila
(in which hybrid sterility is mostly due to BDM incompatibilities), sterile plant hybrids often recover fertility after chromosomal doubling (18
). This is expected if chromosomal rearrangements are the cause of sterility, because chromosomal doubling furnishes an exact homolog for each chromosome, whereas doubling should not affect BDM incompatibilities. Microchromosomal rearrangements such as the gain and loss of duplicate genes are more frequent than previously suspected and may lead to hybrid incompatibilities with no loss of fitness in the diverging lineages (20
). Finally, hybrid sterility in plants frequently maps to chromosomal rearrangements (21
), although whether the cause is chromosomal underdominance or BDM loci that have accumulated within the rearrangements is often unclear. The reduced recombination associated with chromosomal rearrangements can facilitate the accumulation of hybrid incompatibilities in these regions (19
) or expedite the establishment of rearrangements in the first place (23
BDM hybrid sterility in plants may be under simple or complex genetic control. However, fewer loci contribute to hybrid sterility in plants than in Drosophila
, and there appears to be no difference in the numbers of pollen (male) versus seed (female) incompatibilities, perhaps because plants largely lack differentiated sex chromosomes (24
). In addition, cytoplasmic male sterility examples characterized at the molecular level (26
). CMS phenotypes are rescued by nuclear-encoded, mitochondrial-targeted genes that restore fertility (Rf
genes). With the exception of Rf2
from maize, all cloned Rf
genes are members of the pentatricopeptide repeat gene family (PPR), an unusually large gene family in plants (441 genes in Arabidopsis
) that controls organelle gene expression. Although the molecular evolution of Rf
genes is unknown, they are likely to be involved in coevolutionary chases with CMS as a result of genetic conflict between cytoplasmic and nuclear genes. These evolutionary dynamics may reduce the long-term effectiveness of CMS as a species barrier, because the same evolutionary forces that cause the spread of CMS within species could facilitate the introgression of CMS and restorers across species boundaries.
BDM factors also can cause hybrid weakness or inviability. Hybrid weakness is often manifested as necrosis in developing seedlings or adult plant tissue, similar to the phenotype of pathogen attacks (27
). These observations imply that hybrid weakness may result from changes in pathogen resistance genes (), which diverge in response to selection pressure exerted by pathogens. More studies are needed to determine the frequency of this mechanism for hybrid (CMS), which results from an incompatibility between the plant’s nuclear genome and its cytoplasm, is frequently reported in intra- and interspecific plant hybrids, but not in animal hybrids (25
). CMS is under frequency-dependent selection in hermaphrodite-biased populations, which predominate in plants, but under strong negative selection if there are separate male and female sex morphs. CMS is caused by aberrant, frequently chimeric, mitochondrial genes in all weakness in interspecific crosses and to elucidate other mechanisms of hybrid inviability.
A final emerging difference between plants and animals (or at least Drosophila
) is that most BDM incompatibilities characterized in plants are polymorphic within species (27
. This is consistent with an origin of BDM incompatibilities through frequency-dependent selection, local adaptation, or drift. However, it also implies that BDM incompatibilities are rarely the cause of speciation in plants, because they correlate poorly with species boundaries and typically make small contributions to total isolation.