EFTA02702876

Journal of lhearrtical Biology 399(2016) 103-116 Contents lists available at ScienceDirect 1O4 aRt` SN Journal of Theoretical Biology ELSEVIER journal homepage: Evolution of worker policing Conshlark Jason W. Olejarz a, Benjamin Allen l'"', Carl yeller • Raghavendra Gadagkar -1. Martin A. Nowak a•d•g.s Program for twoltalonary Dynamks. Hanard University, Cambridge, MA 02(38. USA °Department of Mathematics. Emmanuel College, Bosron. MA 02115, USA `Center for Mathematical Sciences and Applicanons. Harvard Unhersky. Cambridge, MA 02138. USA Depaytmcnr of Organismic and Emlutionary Biology. Hamad University. Cambridge. MA 02178. USA • Centre for Ecological Sciences and Centre for Contemporary Studies. Indian Institute of Science. Bangalore 560 072. India 'Indian National Science Academy. New Delhi 110 002. India ▪ Deponent of Mathematics. Harvard University. Cambridge. MA OM& USA ARTICLE INFO ABSTRACT Ankle history. Workers in insect societies are sometimes observed to kill male eggs of other workers, a phenomenon Received 2 February 2015 known as worker policing. We perform a mathematical analysis of the evolutionary dynamics of policing. Received in revised form We investigate the selective forces behind policing for both dominant and recessive mutations for dif- 23 January 2016 ferent numbers of matings of the queen. The traditional, relatedness-based argument suggests that Accepted 2 March 2016 Available online 11 March 2016 policing evolves if the queen mates with more than two males, but does not evolve if the queen mates with a single male. We derive precise conditions for the invasion and stability of policing alleles. We And Keywords: that the relatedness-based argument is not robust with respect to small changes in colony efficiency Sociobiology caused by policing. We also calculate evolutionarily singular strategies and determine when they are Natural selection evolutionarily stable. We use a population genetics approach that applies to dominant or recessive Evolutionary dynamics Modelsisimulations mutations of any effect size. 2016 Elsevier Ltd. All rights reserved. 1. Introduction (Hughes et al.. 2008; Cornwallis et al.. 2010; Queller and Strassmann. 1998; Foster et al.. 2006; Rootnsma, 2007, 2009). In contrast, it is In populations with haplodiploid genetics, unfertilized female believed that polygamy—not monogamy—is important for the evo- workers are capable of laying male eggs. Thus, in a haplodiploid lution of police workers. colony, male eggs can in principle originate from the queen or Several papers have studied the evolution of policing. Starr from the workers. Worker policing is a phenomenon where female (1984) explores various topics in the reproductive biology and workers kill the male eggs of unmated female workers (Ratnieks. sociobiology of eusocial Hymenoptera. He defines promiscuity as 1988; Ratnieks and Visscher. 1989; Ratnieks et al., 2006; Gadagkar. 1 firl_ ill ). where n is the number of matings of each queen, and 2001; Wenseleers and Ratnieks, 20064. Worker policing is J; is the fractional contribution to daughters by the i-th male mate. observed in many social insects, including ants, bees, and wasps. He writes, regarding workers, that "They are on average less Yet the precise conditions for the evolution of worker policing are related to nephews than brothers whenever (promiscuity is still unclear. greater than two) and should prefer that the queen lay all the male Worker policing (Ratnieks. 1988; Ratnieks et al.. 2006; Gadagkar, eggs. Workers would therefore be expected to interfere with each 2001; Wenseleers and Ratnieks. 2006a) and worker sterility (Wilson. other's reproduction." Thus, Starr (1984) was the first to suggest 1971; Hamilton. 1972; Olejarz et al., 2015) are two distinct phe- that workers should raise their nephews (sons of other workers) if the queen mates once, but should only raise their brothers (sons of nomena that are widespread in the eusocial Hymenoptera. In addi- the queen) if the queen mates more than twice. Starr (1984) uses a tion to worker policing, a subset of workers in a colony may forego relatedness-based argument. but he does not provide any calcu- their own reproductive potential to aid in raising their siblings. Prior lation of evolutionary dynamics in support of his argument; he relatedness-based arguments have suggested that queen monogamy uses neither population genetics nor inclusive fitness theory. In a is important for the evolution of a non-reproductive worker caste book on honeybee ecology, Seeley (1985) also proposed, using a relatedness-based argument, that worker policing should occur in • Corresponding author. colonies with multiply mated queens, but that worker policing E-mail oddmss: martin novakPhaivard.edu (MA. Novak). should be absent if queens are singly mated. latp:ndx.clorcag/1010I6Ultbi.20I6.03.091 0022-5193)&2016 Elsevier Ltd. All rights reserved. EFTA_R1_02076880 EFTA02702876 104 1W. OhIan et al /journal of Theoretkal Bletrov 399 (2016)103-116 Woyciechowski and Lomnicki (1987) perform a calculation studied species to date that are multiply mated.) Worker removal based on population genetics and conclude that workers should of worker-laid eggs is much less prevalent in colonies of the raise their nephews (sons of other workers) if the queen mates bumblebee (Velthuis et al., 2002), the stingless bee, (Peters et al.. once, but should only raise their brothers (sons of the queen) if the 1999), and the wasp, Vespula rata (Wenseleers et al.. 2005), which queen mates more than twice—the case of double mating is neu- are predominantly singly mated. (As mentioned above, worker tral with respect to preference. From this result, they claim that, policing has been found only in about 20% of the studied species to under multiple mating of the queen, natural selection should favor date that are singly mated.) There are some studies based on non-reproductive workers. Woyciechowski and Lomnicki (1987) observational evidence that find policing in singly mated species; consider both dominant and recessive alleles affecting worker examples of species with single mating and worker policing are behavior, but they do not consider colony efficiency effects. Vespa crobro (Foster et al.. 2002), Camponotusjlortdanus (Endler et Ratnieks (1988, considers the invasion of a dominant allele for al., 2004), Aphaenogaster smythiesi (Wenseleers and Ratnieks, policing. Using population genetics, he arrives at essentially the 200bh). and DitIC0MM0 (Wenseleers and Ratnieks, 20061), same conclusion as Woyciechowski and Lomnicki (1987): In the Interspecies comparisons are somewhat problematic, because absence of efficiency effects, policing evolves with triple mating even though phylogeny can be controlled for, there are many but not with single mating. But Ramieks also considers colony (known and unknown) ways in which species differ in addition to efficiency effects, focusing mainly on the case where policing mating frequency that may also affect the absence or presence of improves colony efficiency. Since policing occurs alongside other worker policing. Furthermore. many empirical studies are based maintenance tasks (such as cleaning of cells. removal of patho- on genetic analyses of male parentage. (Though studies of some gens, incubation of brood), and since eating worker-laid eggs species are based on actual observational evidence: see. e.g., might allow workers to recycle some of the energy lost from laying Wenseleers and Ratnieks, 2006b.) Regarding species for which the eggs. Ramieks supposes that policing improves colony efficiency. study of policing is based on genetic analyses, policing is often He finds that worker policing with singly mated queens may inferred if males are found to originate predominantly from the evolve if policing improves colony reproductive efficiency. He also queen. But such an inference, in cases where it is made, pre- finds that worker policing with triply mated queens may not supposes that workers actively try to lay male eggs in the first evolve if policing reduces colony reproductive efficiency, but he place. It is therefore not clear how reliably genetic investigations considered this case to be unlikely on empirical grounds. Ratnieks can measure policing. does not study recessive policing alleles. He also does not calculate The small number of attempts at measuring the prevalence of evolutionary stability conditions. worker policing in intraspecific experiments have also returned Both papers (Woyciechowski and Lomnicki. 1987; Ratnieks, conflicting results. Foster and Ratnieks (2000) report that facul- 1988) offer calculations based on population genetics without tative worker policing in the saxon wasp. Dolichovespula saxonica. mentioning or calculating inclusive fitness. These early studies is more common in colonies headed by multiply mated queens. (Starr. 1984; Seeley, 1985; Woyciechowski and Lomnicki. 1987; But their sample size is only nine colonies. The phenomenon was Ratnieks. 1988) were instrumental in establishing the field of reinvestigated by Ronckaert et al. (2011) who report no evidence worker policing. of facultative worker policing depending on queen mating fre- Testing theoretical predictions on the evolution of worker quencies, and argue that the previous result may have been flawed policing in the field or in the lab is difficult Due to the complex- or that there were interpopulational variations. ities inherent in insect sociality, published empirical results are not Many empirical studies have emphasized that factors besides always easy to interpret. While, so far, winter policing has been intracolony relatedness—including the effects of policing on a found in all species with multiple mating that have been studied, it colony's rate of production of offspring—may play a role in has also been found in about 20% of species with singly mated explaining evolution of worker policing (Foster and Ratnieks queens (I lammond and Keller. 2004; Wenseleers and 2001a,c: Hartmann et al_ 2003; Hammond and Keller. 2004; Ratnieks. 2006b; Bonckaert et al.. 2008). Herein lies the difficulty: Wenseleers and Ratnieks. 2006b; Helantera and Sundstrom. 2007; When worker policing is found in multiply mated species and Khila and Abouheif. 2008; Zanette et al. 2012). Yet reliable pub- found to be absent in singly mated species, this is taken as evi- lished data on the effect that policing has on colony reproductive dence supporting the relatedness argument, and when worker efficiency are often hard to find. (For some exceptions, see policing is found in singly mated species, it is explained away as Wenseleers et al.. 2013 and references therein.) not being evidence against the theory, but as having evolved for In this paper, we derive precise conditions for the evolutionary other reasons (such as colony efficiency). See, for example, the invasion and evolutionary stability of police alleles. We consider following quotation by Bonckaert et al. (2008): 'Nevertheless. our any number of matings, changes in the proportion of queen- results are important in that they show that V. germanica forms no derived males, changes in colony efficiency, and both dominant exception to the rule that worker reproduction should be effec- and recessive mutations that affect the intensity of policing, tively policed in a species where queens mate multiple times Our paper is based on an analysis of evolutionary dynamics and (Ratnieks, 1988). Indeed, any exception to this pattern would be a population genetics of haplodiploid species (Nowak et al., 2010; much bigger challenge to the theory than the occurrence of Olejarz et al., 2015). It does not use inclusive fitness theory. Spe- worker policing in species with single mating, which can be cifically, we adapt the mathematical approach that was developed readily explained (Ratnieks, 1988; Foster and Ratnieks. 2001b)." by Olejarz in al. (2015) for the evolution of non-reproductive This is precisely why a careful simultaneous consideration of workers. We derive evolutionary invasion and stability conditions relatedness, male parentage, and colony efficiency is important for for police alleles. Mathematical details are given in Appendix A. understanding worker policing. In Section 2, we present the basic model and state the general We do not aim to provide an exhaustive catalog of all species in result for any number of matings for dominant policing alleles. In which worker policing has been studied. We merely cite some Sections 3-5, we specifically discuss single, double. and triple specific examples to add context Policing is rampant in colonies of mating for dominant policing alleles. We take dominance of the the honeybee (Ratnieks and Visscher. 1989), the wasp Vesputa policing allele to be the more realistic possibility because the vutgaris (Foster and Ratnieks, 2001c), and the wasp Vespula ger- policing phenotype is a gained function. Nonetheless. for com- manica (Bonckaert et al., 2008), which are all multiply mated. (As pleteness, we give the general result for recessive policing alleles mentioned above, worker policing has been found in all of the in Section 6. In Section 7, we discuss how the shape of the EFTA_R1_02076881 EFTA02702877 Olefore et at /Journal of Theorrtkol Biology 399 (20:6) I03-II6 105 efficiency function determines whether or not policing is more a likely to evolve for single or multiple matings. In Section 8. we vimin Males Fertiliad Queens Queens analyze our results for the case where the phenotypic mutation induced by the mutant allele is weak (or, equivalently in our formalism, the case of weak penetrance). In this setting, the PA + (n —m) A +m a p AA,m quantity of interest is the intensity of policing. We locate the evolutionarily singular strategies. These are the values of intensity Aa + (n —m) A +m a Aa.m of policing for which mutant workers with slightly different policing behavior are, to first order in the mutant phenotype. aa + (n —m ) A+ma aafri neither advantageous nor disadvantageous. We then determine if a singular strategy is an evolutionarily stable strategy (ESS). In Section 9. we discuss the relationship between policing and San Cbiab ninisrb• Tem Clusll inclusive fitness theory. together with the limitations of the O•WIIPO Ors•if• INI•oss• Os" Ns Maw' Moo AAA A. • A relatedness-based argument. Section 10 concludes. Ms Aa A •• • ••• A•••• • •• SA0a AAA M•a• A••• Awl* A• A •.0 rs •• 2. The model C ra Come likes a Usua We investigate worker policing in insect colonies with haplo- Oman timain• Porn% ClougAlm• 0.••••• scos- C•i•Ase Sam -ANAA•mAs A (2.•••1••••• diploid genetics. Each queen mates n times. We derive conditions Awn \10-1•0•A•nAs•maa A•• 1.0 A • In•2al• la-m)M•air In-elA•01•••• under which a mutation that effects worker policing can spread in ran a population. We make the simplifying assumption, as do nip I (a) The possible fluting events with haplodiploid genetics are shown. Each Wowiechowski and Loninicki (1987) and Rat nicks (1988). that the queen mates with n malts. m denotes the number of times that a queen mates with mutant type a males and can take values between 0 and n. Thus. there are 3Ot I I) colony's sex ratio is not affected by the intensity of worker types of colonies. (b) If each queen mates with only a single male, then there are six policing. types of colonies. The female and male offspring (right three columns) of each First we consider the case of a dominant mutant allele. Because colony (leftmost column) are shown. For example. M, I colonies arise from a type the policing allele confers a gain of function on its bearer, the M female fluting with a single mutant type a male. AA. I queens produce female offspring of type Aa and male offspring of type A. 50% of the offspring of workers in assumption that it is dominant is reasonable. There are two types AA. I colonies ate of type A. while the remaining 50% of the offspring of workers in of males. A and a. There are three types of females, AA. Aa, and aa. M. I colonies are of type o. (c) The female and male offspring fright three columns) If the mutant allele is dominant then Aa and aa workers kill the of each colony (leftmost column) when each queen mates n times are shown. male eggs of other workers, while M workers do not. (Alter- natively, M workers police with intensity ZAA, while Aa and ao Possible Forms for the Function p, workers police with intensity Zaa = =Zm+w. We consider this case in Section 8.) For n matings, there are 3m+ 1) types of mated queens. We use the notation AA, in; Ac. in; and art m to denote the a genome of the queen and the number, m. of her matings that were 5ei 0.75 with mutant males. a. The parameter m can assume values 0.1....,n. A schematic of the possible mating events is shown in ei, 0.50 Fig. 1(a). 0 There are three types of females, AA. Aa. and aft and there are n+1 possible combinations of males that each queen can mate 0.25 with. (For example, a queen that mates three times (n=3) can mate with three type A males. two type A males and one type a 0_ male, one type A male and two type a males, or three type a 0.25 0.50 0.75 1 males.) Fig. I (b) shows the different colony types and the offspring Fraction of police workers, of each type of colony when each queen is singly mated. Fig. I Ng. 2. The queen's production of male eggs. pt. increases with the fraction of (c) shows the different colony types and the offspring of each type waiters that are policing. Z. This is intuitive, since having a larger worker police of colony when each queen mates n times. The invasion of the force means that a greater amount of worker-laid eggs can be eaten or removed. mutant allele only depends on a subset of colony types. The cal- Three possibilities for a monotonically increasing function p, are shown. culations of invasion conditions are presented in detail in Appendix A. 22 Colony efficiency as a function of policing r, represents the rate at which a colony produces offspring 2.1. Fraction of male offspring produced by the queen (virgin queens and males) if the fraction of police workers is z. (This quantity was also employed by Ratnieks. 1988.) Without loss pa represents the fraction of males that are queen-derived if the of generality, we can set rc, = I. For a given mutation that affects fraction of police workers is z. (This quantity was already the intensity of policing, and for a given biological setting, the employed by Ratnieks. 1988.) The parameter z can vary between efficiency function ra may take any one of a variety of forms 0 and 1. For z=0. there are no police workers in the colony, and for (Fig. 3). z= 1, all workers in the colony are policing. We expect that pa is an Colony efficiency depends on interactions among police work- increasing function of z. Increasing the fraction of police workers ers and other colony members. It also depends on the interactions increases the fraction of surviving male eggs that come from the of colonies and their environment. There are some obvious nega- queen (Fig. 2). tive effects that policing can have on colony efficiency. By the act EFTA_R1_02076882 EFTA02702878 1013 ale/an et at /journal of Theoretical Biology 399 (2016) 103-116 Possible Forms for the Function r, the entire worker population is unclear. It is possible that a frac- tion z < 1 of police workers can effectively police the entire 1.3 population, and adding additional police workers beyond a certain point could result in wasted energy, inefficient use of colony 1.2 resources, additional recognitional errors. etc. These effects may correspond to colony efficiency r, reaching a maximum value for j 1.1 73 some 0 < z <1. I 1 As another possibility, suppose that police workers, when their number is rare, directly decrease colony efficiency by the act of I 0.9 killing male eggs. It is possible that for some z c 1, police workers are sufficiently abundant that their presence can be detected by 0.8 other workers. Assuming the possibility of some type of facultative response, the potentially reproductive workers may behaviorally 0 0.25 0.50 0.75 adapt by reducing their propensity to lay male eggs, instead Fraction of police workers, z directing their energy at raising the queen's offspring. In this scenario, colony efficiency r, may reach a minimum value for some Fig. 3. The functional dependence of colony efficiency. r,, on the fraction of 0<z<1. workers that are policing. z. may take any one of many possibilities. 23. Main results for dominant police alleles of killing eggs, police workers are directly diminishing the number of potential offspring. In the process of identifying and killing We derive the following main results for dominant police nephews, police workers may also be expending energy that could alleles. If the queen mates with n males, then the a allele for otherwise be spent on important colony maintenance tasks (Cole, policing can invade an A resident population provided the fol- 1986; Naeger et al.. 2013). Policing can also be costly if there are lowing "evolutionary invasion condition" holds: recognitional mistakes. i.e.. queen-laid eggs may accidentally pun +pia (rim\ ma) > 2 _ I'l_ _„ .1 trim\ be removed by workers. Recognitional errors could result in ( (1) modifications to the sex ratio. which is an important extension of 2 ro ro 1'0 l ring ro our model but is beyond the scope of this paper. When considering only one mutation. ro can be set as 1 without We can also identify positive effects that policing may have on loss of generality. Why are the four parameters. rim. ruz, p", and colony efficiency. It has been hypothesized that the eggs which are p12, sufficient to quantify the condition for invasion of the mutant killed by police workers may be less viable than other male eggs allele, a? Since we consider invasion of a. the frequency of the (Velthuts et al., 2002: Pirk et al.. 1999: Gadagkar. 2004; Nonacs, mutant allele is low. Therefore, almost all colonies are of type 2006), although this possibility has been disputed (Beekman and AA 0. which means a wild-type queen.M. has mated with n wild- Oldmyd. 2005; Helantera et al.. 2006; Zanetre et al.. 2012). If less- type males, A. and 0 mutant males, a. In addition, the colonies Aa, 0 viable worker-laid eggs are competing with more-viable queen-laid and M. I are relevant. These are all colony types that include male eggs, then policing may contribute positively to overall colony exactly one mutant allele. Colony types that include more than one efficiency. Moreover, policing decreases the incentive for workers to mutant allele (such as Aa.1 or AA 2) are too rare to contribute to expend their energy laying eggs in the first place (Foster and the invasion dynamics. For an Aa.O colony, half of all workers are Ratnieks, 2001a; Wenseleers et al.. 2004a,b: Wenseleers and policing and therefore the parameters r1j2 and p1, 2 occur in Eq. Ratnieks. 2006a). which could be another positive influence on (1). For an AA.1 colony, 1/n of all workers are policing, which colony efficiency. (However, the decrease in incentive for workers to explains the occurrence of r im and p,,,, in Eq. (1). reproduce due to policing would only arise on a short time scale if Next, we ask the convene question: What happens if a popu- there is a facultative response to policing which is unlikely.) lation in which all workers are policing is perturbed by the As another speculative possibility: Could it be that worker egg- introduction of a rare mutant allele that prevents workers from laying and subsequent policing acts as a form of redistribution policing? If the a allele for worker policing is fully dominant, and if within the colony? That is. suppose that it is better for colony colony efficiency is affected by policing then a resident policing efficiency to have many average-condition workers than to have population is stable against invasion by non-police workers if the some in poor condition and some in good condition. Suppose following "evolutionary stability condition" holds: further, as seems realistic. that good-condition workers are more ri pi) +pa„_ ivamtn-2) likely to lay eggs (which are high in nutritional content, of course). > (2 + 0)(2 +2(2+n+npi ) (2) If the average police worker is of condition below the average egg- ran- mom laying worker, then worker egg-laying and policing serves to What is the intuition behind the occurrence of the four para- redistribute condition among the workers, improving overall col- meters, r1. ran _ wo p pi. and pan _ warn? The condition applies to ony efficiency. a population in which all workers are initially policing. Note that. The special case, where policing has no effect on colony effi- because the allele, a. for policing is fully dominant in our treat- ciency and which has informed the conventional wisdom, is ment. non-policing behavior arises if at least two mutant A alleles ungeneric, because policing certainly has energetic consequences for non-policing are present in the genome of the colony. which is for the colony that cannot be expected to balance out completely. the combination of the queen's genome and the sperm she has An early theoretical investigation of colony efficiency effects stored. To study the invasion of a non-policing mutant allele. we regarding invasion of dominant mutations that effect worker must consider all colony types that have 0,1, or 2 mutant A alleles: policing was performed by Ratnieks (1988). these are Oa. n; aa.n —1: Ao.n: aa.n —2; Ao.n-1; and M.n. The Although monotonically increasing or monotonically decreas- colonies aa,n: aa.n —1: Aa.n: aa.n —2: and Mae do not contain ing functions r, are the simplest possibilities, these cases are not non-police workers: the efficiency of those colonies is r,, and the exhaustive. For example, a small or moderate amount of policing fraction of male eggs that originate from the queen in those may be expected to improve colony efficiency. However, the pre- colonies is pi. Both of these parameters occur in Eq. (2:. Colonies of cise number of police workers that are needed to effectively police type Aa.n - 1 produce a fraction of 1/(2r) non-police workers. EFTA_R1_02076883 EFTA02702879 Olejarz et at /Journal of Theorrtkol Biology 399 (2016)103-N6 107 a Single mating. n=1 100 a 101 — r •052 - rp1.061 11s1.0130 — r et•069 10 10 15 Time (104) Fig. 4. Numerical simulations of the evolutionary dynamics of wetter policing team the condition given by Eq. ' I :. The policing allele is dominant. For numerically probing invasion we use the initial condition XA•o= 1-10- 3 and XAA 1=10 3. We set ro= I without loss of generality. Other parameters are: (a)Pi7 =0.75. pl =0.9. and (b) Ave 0.6, r — 1005. and ri -1.01. Police allele ca ' ade and is evolutionarily stable • ca ade. but is not stable C— to not vale but s stable C— c not d nil is not stable 0 1 Frequency of police allele Fig. 5. There are four possibilities for the dynamical behavior in the proximity of two pure equilibria. a n = 1 mating p1,2 = 0.75, p, = 1 b n = 1 mating p,,, = 0.99. p, = 1 1.2 1.2 Stably Colony efficiency, r1 .-- 11 Does Not \ 1.1 Invade Invades co Stable \ Unstable e.= 1 Invades C Unstable / Does Not O O Invade Does Not 0.9 0.9 - Invade Bitable Unstable 0.9 1 1.1 12 10.8 0.9 1 1.1 1.2 Colony efficiency, r v2 Colony efficiency, r1,2 Fig. S. If queens are singly mated (n-1). then a plot of I.' versus r 1,2 dearly shows all four possibilities for the behavior around the two pure equilibria. For (a), we set pi 0.75 and pi 1. For (b). we set pi = 0.99 and p,-1. which explains the occurrence of ri2,102a, and pa,,_ 102„, in Eq. and be stable. The possibilities are shown in Fig. 5. In the cases (2). where policing cannot invade but is stable, or where policing can Numerical simulations of the evolutionary dynamics with a invade but is unstable. Brouwer's fixed-point theorem guarantees dominant police allele are shown in Fig. 4. the existence of at least one mixed equilibrium. In the case where Generally, four scenarios regarding the two pure equilibria are policing can invade but is unstable, police and non-police workers possible: Policing may not be able to invade and be unstable, will coexist indefinitely. policing may not be able to invade but be stable, policing may be We will now discuss the implications of our results for parti- able to invade but be unstable, or policing may be able to invade cular numbers of matings. EF TA_R 1_02076884 EFTA02702880 lea J.W. Otejaa ef al / Journal of Theo:ilk& Swim 399 (2016)103-116 a The police allele invades and is stable. b The non.polico allele invades and is stable. p,2 0.75. r,? 1.0344, re1.0787 pin=0.75. p.=1. rig=1.0344. r,=1.0567 1 1 0.8 0.8 1 0.6 0.8 - a -8 OA I 0.2 - 0 2 3 4 84 88 88 Time (10') Thin (10') The police and non-police alleles are bistable. d The police and non-poise alleles meat p,a 40.75. rflu1.0244. r,a1.0667 pws0.75. pint ri4e1.0444. ?ter 1 0.8 1 0.8 0.6 - •O 0.8 a 0 OA 15 0.4 LL • 0.2 5 10 15 20 5 LL 02 1 00 4 6 8 10 Time (103) Time (10') Flg. 7. Nurnencal simulations of the evolutionary dynamics of worker policing that show the four behaviors in hg (,(a). The policing allele is dominant For each of the four panels. we use the initial conditions: (a) XAto I -10 l and gm - 10 1:(b)X,ai - I -10 s and r eso - 10 3: 01%Am) 0.02 and Xµl— 098 (lower curve). and Xmo = 0 01 and XAA l =0.99 (upper cum): (d)44.0 =1-10 -2 and XAAI =10 -2 (lower curve). and )(mi I —10-l and 4, 10 -2 (upper curve). We set ro • I without loss of sawrality. n = 1 mating The stability condition for a dominant police allele is 1.003 6—pia+3p,I-112 (4) ri > 6+2p, 4.." 1.002 Evolution of policing is highly sensitive to changes in colony effi- 1.001 ciency. For example, let us consider p, ,2 = 0.99 and pi = 1. This means that if half of all workers police then 99% of all males come from the queen. Ifall workers police then all males come from the queen. In this case, efficiency values such as r 1,2 = 1.001 and r, = 1.0031 lead to the Petty Wades bilis unstable evolution of policing. In principle, arbitrarily small increases in colony 0.999 Pollee Insides aid et HAW — Mktg der not Merle and a unstable efficiency can lead to the evolution of policing for single mating. — eons do a Minna, but e ;We A plot of r, versus rif2 for singly mated queens (Fig. 6) illus- 0.998 trates the rich behavior highlighted in Fig. 5. Numerical simula- 0 1/2 1 tions of the evolutionary dynamics are shown in Fig. 7. Fraction of police workers. Another intriguing feature is that increases in colony efficiency Fig. 8. Possible r, efficiency curves for n=1 mating which demonstrate different due to policing do not necessarily result in a higher frequency of behaviors. For this plot, we set pre =099 and m 41. Here. each curve has the police workers at equilibrium Fig. S illustrates this phenomenon. functional form r1 - 1 +ea +922. For exam*. we can have: (blue) policing Invades Four possibilities for the efficiency function r., are shown. Notice that but is unstable, o-0.003. )l -0.0004: (green) policing invades and is stable. (7=0.0026. 4=0: (red) policing does not invade and is unstable. 0=0.0024. p=0: the r2 curve which results in coexistence of police workers and non- (black) policing does not Invade but is stable. 0=0.002. 4=00004. (For Inter- police workers (blue, top) is strictly greater than the r2 curve which pretation of the references to color in this figure caption, the reader is referred to results in all workers policing (green, second from top). How can the web version of this paper.) increased efficiency due to policing possibly result in policing being less abundant at equilibrium? If a mutation for non-policing behavior 3. Single mating is introduced into a resident policing population, then the evolu- tionary success of the non-policing mutation depends on the success For single mating. n= I. the invasion condition for a dominant of Ao. 0 colonies relative to aa. 1. aa. 0. Aa. 1, and AA, 1 colonies. Au. 0 police allele is colonies have an efficiency parameter r,,2. while the other four relevant colonies each have an efficiency parameter r i. Thus, if vi a is 2(2 —r1/2) too large relative to r,, then the non-police allele can invade a resi- > 13) 2(1 —PI ) 441 +PI,.2)rla dent policing population, and there is coexistence. Also notice that the r, curve which results in bistability of police (Recall that ro a 1.) workers and non-police workers (black, bottom) is strictly less than EFTA_R1_02076885 EFTA02702881 J.W. Olefarz et at /Journal of Theowrkal Biology 399 (2016)103-H6 109 n = 2 matings n = 3 mations 1.003 1.003 - noising irniszles WI Is onstobte Peking inv.. WI isunstable — Poland invades and is slab* — Poland }made* and is slat. .7 1.002 painng does nor ',wady end is unstable 1.002 Poicing does not Mrs. and n unstable Poking don not made bulls**. - Pohang does not info. toil is stable •_ 1.001 1.001 0 or a) or 1 0 (.) 0.999 (-> 0.999 0.998 a 0.998 a 1/3 1/2 (2/3) 5/6 1/2 3/4 1 1 Fraction of police workers,: Fraction of police workers,: Fig. 9. Possible r, efficiency curves for n=2 marines which demonstrate different Fts.10. Possible r, efficiency curves forn-3 matings which demonstrate different behaviors Here, each curve has the functional form r, • fir 2. For example. behaviors. For this plot we set p,r, =0.986. pe, = 0.99. Ps,. = 0996. and pi = 1. we can have: (blue) policing invades but is unstable. a= 0.0005. D=-0.0004: Here, each curie has the functional form r, = I -,a -par. For example. we can have: (green) policing invades and is stable. 0=0.000l. /3=0: (red) policing does not (blue) policing invades but is unstable. a= - 0.0006..d -0.0006: (green) policing invade and is unstable. a= -0.000). D=O: (black) policing does not invade but is invades and is stable, a--0.0012. D-0: (red) policing does not invade and is stable. a= -0.0005. rre0.0004. (For interpretation of the references to color in this unstable. a= -00015. 0=0: (black) policing does not invade but is stable. figure caption. the reader is referred to the web version of this paper.) • 0.0021. A -now& Note that the value r2,1affects the population dynamics but does not appear in the conditions for invasion and stability of the police allele. hence the r, curve which results in policing being dominated by non-policing the parentheses on the horizontal axis. (For interpretation of the references to color in this figure caption. the reader is referred to the web version of this paper.) (red, second from bottom). This phenomenon arises in a similar way: if r, .2 is too small relative tor,, then the non-police allele cannot invade a resident policing population, and there is bistability. Just as for single mating, we observe the intriguing feature that increases in colony efficiency due to policing do not necessarily result in a higher frequency of police workers at equilibrium. Fig. 10 illus- trates this phenomenon. Four possibilities for the efficiency function 4. Double mating I-, are shown. Notice that the r, curve which results in coexistence of police workers and non-police workers (blue, top) is strictly greater For double mating. n=2. the Invasion condition fora dominant than the r, curve which results in all workers policing (green, second police allele is given by from top). Also notice that the r, curve which results in bistability of r1,2 > 1 (5) police workers and non-police workers (black, bottom) is strictly less than the r; curve which results in policing being dominated by non- Thus, policing can invade if there is an infinitesimal increase in policing (red, second from bottom). colony efficiency when half of all workers police. Policing cannot invade if there is an infinitesimal decrease in colony efficiency when half of all workers police. 6. Recessive police allele The stability condition for policing is given by r, > r314 (61 We have also derived the conditions for the emergence and evo- lutionary stability of worker policing if the police allele is fully reces- Therefore, the policing allele is stable if the colony efficiency is sive. In this case, AA and Aa workers are phenotypically identical and greater for z=1 (when all workers police) than for z = 3/4 (when do not police, while aa workers do police. (Alternatively. AA and M only three quarters of the workers police). workers police with intensity ZA4=Zed. while no workers police with Four possible efficiency curves r, and the corresponding intensity Ze, =4,4+w = 4„ + w. We consider this case in Section 8.) behavior of the police allele are shown in Fig. 9. 6.1. Emergence of worker policing 5. Triple mating The invasion condition for a recessive police allele, a, is given by For triple mating. n=3. the invasion condition for a dominant rya., > 2(2+n+npo) (9) police allele is given by r0 (2 +n)(2+po)+Tho2,0(n — 2) 4-2(1—pliorla (7) Note that Eq. (9) for invasion of a recessive police allele has the same rIR > 2 +(pin mathematical form as Eq. (2) for evolutionary stability of a dominant +Pnoria police allele. Starting from Eq.:2'n making the substitution z—. 1—z, The stability condition for policing is given by and reversing the inequality, we recover Eq. (9). The intuition behind 101-p5/6-F5pi this correspondence is described in Appendix A. rl > (8) 10+6pi r516 6.2. Stability of worker policing As a numerical example, let us consider p1„.3 = 0.98 and pia =0.99.11x=1/3 of workers police. then 98% of males come A recessive police allele, a. is evolutionarily stable if from the queen. If z = 1/2 of workers police, then 99% of males \ 12( \ _11 _0 nit\ _ come from the queen. In this case, policing cannot invade if ri,3 j 2 (10) nj )kna) =0.9990 and ri a =0.9979. In principle, arbitrarily small reduc- tions in colony efficiency can prevent evolution of policing for Note that Eq. (10) for evolutionary stability of a recessive police triple mating. allele has the same mathematical form as Eq. (I) for invasion of a EFTA_R1_02076886 EFTA02702882 110 J.W. OkJarz er at /Journal of Theorrtkal Norm 399(2016)103-116 0 4 8 12 Time (105) Time (105) Rd. It. Numerical simulations of the evolutionary dynamics of worker policing confirm the condition given by Eq. . The policing allele is recessive. For numerically probing invasion. we use the initial condition Xmo - I -10-2 and Xmo t0-2. We set rot - I without loss of generality. Other parameters ale: (a) p0-0.6. p,,, 0 and (b) Po 036.PIA2 = 0.9. ,,rot 004, and rl = 1.012. Under what conditions does worker policing invade? n = 1 mating n= 2 mat%igs n= 3 mating' decreases NO NO NO monotonically NO NO Increases monotonically reaches a maximum for some 0 < z <1 reaches a minimum for some 0<z<1 Fig 12. Depending on the functional loon of colony efficiency. r1, on the fraction of police workers. z. policing alleles may or may not invade for single, double. or triple mating. Various possibilities of , arc shown. The outcomes hold for both dominant and recessive police alleles. If r, is constant. then policing does not invade for single mating. is neutral for double mating, and invades for triple mating. If r: decreases monotonically. then policing does not invade or invades only (or triple mating. If r: increases monotonically. then policing either invades only for double and triple mating or for single. double. and triple mating. If r, reaches a maximum at an intermediate value 0 <z < I. then policing does not invade or may invade (or triple mating only, for double and triple mating, or for single. double, and triple mating. If r„ reaches a minimum at an intermediate value 0 <z < I. then any pattern is possible. dominant police allele. Starting from Eq. (1), making the sub- We have the following results regarding the invasion and sta- stitution z I — z. and reversing the inequality, we recover Eq. ( 10). bility of police workers. We discuss single (n-1). double (n=2), Again, the intuition behind this correspondence is described in and triple (n-3) mating. All results apply to both dominant and Appendix A. recessive police alleles. They can be instantiated with arbitrarily Numerical simulations of the evolutionary dynamics with a small changes in colony efficiency. recessive police allele are shown in fig. II. 7.1. Evolutionary invasion of policing 7. Shape of the efficiency function. r, (i) Ifr, is strictly constant (which is ungeneric), then policing does not invade for single mating, is neutral for double mating, and The shape of the efficiency function, G. determines whether does invade for triple mating. policing is more likely to evolve for single mating or multiple (ii) If r, is monotonically decreasing, then policing either invades matings. Recall that r, is the colony efficiency (defined as the rate not at all or only for triple mating. of generation of reproductives) if a fraction, z. of all workers per- (iii) If r, is monotonically increasing, then policing either invades form policing. The variable z can assume values between 0 and 1. If for single, double, and triple mating or only for double and no workers police, z= O. then the colony efficiency is at baseline, triple mating. which we set to one; therefore, we have ro= I. Policing can in (iv) If r, reaches an intermediate maximum (which means colony principle increase or decrease colony efficiency (Fig. 12). efficiency is highest for an intermediate fraction of police EF TA_R 1_02076887 EFTA02702883 J.W. Olefare et at /Journal of Theorrtkol Biology 399 (2016) 103-116 111 a Poking invades for n=1 but not for net b Policing does not invade but is stable (for ne1 and ne2) 1.012 1- • 1.010 1.008 0.999 1.006 • 1.004 • 0.998 1 002 • • • 0.997 • • 0.998 0 12 3(4 1 0.996 0 12 314 Fraction of police workers.: Fraction of police workers.: C Policing invades but is unstable d Poking does not invade but is stable (for ne1) ((or n=1 and n=2) Poking invades but is unstable (for ne2) 1.003 • 1.002 • • • • C 1.002 1.001 4 1.001 • I • 1 • 0 1/2 3/4 1 0 1/2 314 1 Fraction of poke workers.: Fraction of police workers.: Fig. 13. Non-monotonic efficiency functions can lead to rich and counterintuitive behavior. We consider invasion and stability of a dominant police allele for single (n I) and double (n-2) mating. The baseline colony efficiency without policing is re - I. Mice other values must be specified: ri and r,. Moreover. we need to specify two values for how the presence of police workers affects the fraction of malt offspring coming from the queen; we choose Viz —0.99 and Pi — 1. A variety of behaviors can be realized by a very small variation in colony efficiency. (a) Policing invades for single mating but not for double mating. (b) Policing does not invade but is stable for single and double mating. (c) Policing invades but is unstable for single and double mating. (d) Policing does not invade but is stable for single mating. while policing invades but Is unstable for double mating. workers). then policing can invade for n= 1.2.3 or n=2.3 or discuss the invasion and stability of a dominant police allele for n=3 or not at all. single and double mating, we need to specify efficiency at three (v) If rz reaches an intermediate minimum (which means colony discrete values for the fraction of police workers present in a efficiency is lowest for an intermediate fraction of police colony: r112. r314. and rt. Note that re, = 1 is the baseline. Moreover. workers), then policing can invade with any pattern of mat- we need to specify the fraction of male offspring coming from the ings. For example, it is possible that policing invades only for queen at two values: P1,2 and pi. For all examples in Fig. 13, we single mating but neither for double nor triple mating. Or it assume pi/2 =0.99 and PI = 1. We show four cases: (a) policing invades for single and double mating but not for triple mating. invades for single mating but not for double mating; (b) for both single and double mating, policing does not invade but is stable: (c) for both single and double mating. policing invades but is 7.2. Evolutionary stability ofpolicing unstable (leading to coexistence of policing and non-policing alleles): (d) policing does not invade but is stable for single mat- ing: policing invades but is unstable for double mating. These (i) If rz is constant. then policing is unstable for single mating, is cases demonstrate the rich behavior of the system. which goes neutral for double mating, and is stable for triple mating. beyond the simple view that multiple matings are always favor- (ii) If r, is monotonically decreasing, then policing is unstable for able for the evolution of policing. single and double mating. For triple mating it can be stable or unstable. (iii) If r, is monotonically increasing, then policing either is always 8. Gradual evolution of worker policing stable or is stable only for double and triple mating. (iv) If r, reaches an intermediate maximum, then policing can be Our main calculation applies to mutations of any effect size. In stable for any pattern of matings. For example, policing can be this section. we calculate the limit of incremental mutation (small stable for single mating but neither for double nor triple mutational effect size). Our calculations in this section are remi- mating. niscent of adaptive dynamics (Nowak and Sigmund, 1990; Hof- (v) If r; reaches an intermediate minimum, then policing can be bauer and Sigmund. 1990; 0ieckmann and Law. 1996; Metz et al.. stable for n = 1.2.3 or n=2.3 or n=3 or not at all. 1996; Geritz et al., 1998). which is usually formulated for asexual and haploid models. The analysis in this section applies both to the 7.3. Examples for single and double mating case of small phenotypic effect and to the case of weak penetrance. Mathematically, we consider the evolutionary dynamics of poli- Fig. 13 gives some interesting examples for how non- cing if the phenotypic mutations induced by the a allele are small. If monotonic efficiency functions can influence the evolution of an allele affecting intensity of policing is dominant, then it is intui- policing for single (n=1) and double (n=2) mating. In order to tive to think of wild-type workers as policing with intensity ZA.R. EF TA_R 1_02076888 EFTA02702884 111 Oidon et al /journal of Theoretical Biology 399(2016)103-116 while mutant workers police with intensity 4. =4, =ZAA 1-w. If an a small amount For selection to favor increased policing, this ratio allele affecting intensity of policing is recessive, then it is intuitive to of marginals must exceed a quantity depending on the current think of wild-type workers as policing with intensity ZA4 =ZA„, values of 12 and P. while mutant workers police with intensity Zo, = ZAA +V/ = Z,1, +w. Notice that the sign of the right-hand side is determined by In the limit of incremental mutation, the fraction. p. of queen- n —2. So we get different behavior for different numbers of derived males and the colony efficiency, r, become functions of the matings: average intensity of policing in the colony, which is Z+ wz. where z is the fraction of mutant workers in the colony. We have • For n =2 (double mating), policing increases from Z if and only if K(Z) > 0. This means that evolution maximizes the value of R. PIZ +vvz).Ra+P( )wz+! Ribv2z2 +O(w3) regardless of the behavior of P. In other words, for double r,-.R(Z+wz)= R(21+ ralwz+I R.(Znv2z2 +O(tv3) (11) mating, evolution maximizes colony efficiency regardless of the We have made the substitutions p,-•P(Z+ wz) and rz -.R(Z+wZ). effect on the number of queen-derived males. and (11) gives the Taylor expansions of these quantities in terms of • For n=1 (single mating), the right-hand side of Eq. (16) is their first and second derivatives at intensity Z. (For conciseness, we positive. So the condition for Z to increase is more stringent will often omit the argument Z from the functions P and R and their than in the n=2 case. Increases in policing may be disfavored derivatives.) Here, wl<1. so that workers with the phenotype even if they increase colony efficiency. corresponding to the mutant allele only have an incremental effect • For n z 3 (triple mating or more than three matings). the right- on colony dynamics. Thus, the expansions (11) are accurate hand side of Eq. (16) is negative. So the condition for Z to approximations. We assume that P > 0. The sign of w can be posi- increase is less stringent than in the n=2 case. Any increase in tive or negative. If w is positive, then the mutant allele's effect is to policing that improves colony efficiency will be favored, and increase the intensity of policing. If w is negative, then the mutant even increases in policing that reduce colony efficiency may be alleles effect is to decrease the intensity of policing. Note that this favored. formalism could also be interprets as describing the case of weak penetrance, in which only a small fraction of all workers that have Eqs. (13) and (15) also allow us to determine the location(s) of the mutant genotype express the mutant phenotype. evolutionarily singular strategies (Geritz et al., 1998). Intuitively, a For considering the dynamics of a dominant police allele with singular strategy is a particular intensity of policing, denoted by weak phenotypic mutation, we introduce the quantity Zs. at which rare workers with slightly different policing behavior are, to first order in w, neither favored nor disfavored by natural Cdom - Pl tI-Ma 112\ 12 (n12) C m)] (12) selection. The parameter measuring intensity of policing, Z. can 2 tiro, ro k ro re take values between 0 (corresponding to no policing) and 1 (cor- If C.,„„ > 0, then increased intensity of policing is selected, and if responding to full policing). There are several possibilities: There Cam <0. then increased intensity of policing is not selected. This may not exist a singular strategy for intermediate intensity of is just a different way of writing ( I ). policing; in this case, there is either no policing (V = 0) or full We substitute (11) into (121 and collect powers of w. To first policing (r =1). If there exists a singular strategy for 0 < < I. order in w. we get then there are additional considerations: There may be convergent CdomWrn-2)PR+ R 2(2+n+nP)ri evolution toward intensity r or divergent evolution away from 4n ÷O(W2) (13) intensity In a small neighborhood for which Z x2*. further analysis is needed to determine if the singular strategy corre- For considering the dynamics of a recessive police allele with sponding to r is an ESS. weak phenotypic mutation, we introduce the quantity To determine the location(s) of evolutionarily singular strate- 2(2+ n+npo) gies, we set the quantity in square brackets that multiplies w in Cm - titan (14) (131 and I to zero. yielding ro (2 + n)(2 + po) + plia„,(n - 2) If Cm( > 0, then increased intensity of policing is selected, and if RV') We) +fn-21— 0 (17) Cmc <0, then increased intensity of policing is not selected. This is P(r) 2(2-F n+riP(r)) just a different way of writing (9). Eq. (17) gives the location(s) of singular strategies for both domi- We substitute ( I I) into (14) and collect powers of w. To first nant and recessive mutations that affect policing. order in w, we get For a given singular strategy V. there is convergent evolution - 2),R+2(2 +n+nP)K] toward Z* if Crec m W +O(1V2) (15) 4nRa 4.n+ n11) Notice that ( 13) and (15) are, up to a multiplicative factor, the same to first order in w. dE IY(Z) ?W .° 2(2 -i-i iiP(Z)).1 - z• < ° Using Eqs. (13) and (IS), the condition for policing to increase There is divergent evolution away from 2° if the opposite from a given level Z is inequality holds. It is helpful to consider some examples. If the functions f(Z) /rat R(Z) and R(Z) are known for a given species. then the behavior of > —(n-2) (16) 19Z) 2(2 +n+nPan worker policing with gradual evolution can be studied. It is pos- Policing decreases from a given level Z if the opposite inequality sible that policing is at maximal intensity, Z. = 1 (Fig. 14(a)), is holds. We have explicitly written the 2dependencies in Eq. (16) to nonexistent, o(Fig. I4(b)). is bistable around a critical value of emphasize that the quantities P.P. R. and /V are all functions of the intensity. 0 <2* < 1 (Fig. 14(c)), or exists at an intermediate value intensity of policing. Z. of intensity, 0 <V < 1 (Fig. 14(d)). The left-hand side of Eq. (16) can be understood as a ratio of Note that a singular strategy may or may not be an evolutio- marginal effects. To be specific, the left-hand side gives the ratio of narily stable strategy (ESS). (For example. it is possible that there is the marginal change in efficiency over the marginal increase in the convergent evolution toward a particular singular strategy Zs proportion of queen-derived males, if policing were to increase by which is not an ESS. In this case, once Z 2', evolutionary EF TA_R 1_02076889 EFTA02702885 J.W. Olefarz n at /Journal of Theowrkol Biology 399 (2016)103- fie II) a There is full policing. (1-41) b There is no poking. (Z=0) • 1.20 1.15 5 1.20 1.15 .= 1.10 1.10 g 1.05 1.05 1 1 S0.8 0.8 0.2 02 0.4 0.6 0.6 °O 0.2 0.4 0.6 0.8 Intensity of policing. Intensity of poking, C d There is imermediate policing. .0.7986...) • 120 120 • 1.15 - 1.15 - 1.10 I 1.10- 8 1.05 - 1.05 LT, 1 • W E5 0.8 0.8 0.6 I I OA 3 0.2 n •E's: 0.6 04 8. 02 00 o 0.2 0.4 0.6 0.8 Intensity of policing, 7 Intensity of policing. 2 Fig. 14. Several simple examples of functions F(2) and 11(2) are shown. For single mating the corresponding dynamics of policing Intensity with gradual evolution are also shown. We use the forms Pat= 1 - P +P7 and 2(21= I +Ca+ (1/2)C222. For eadi of the four panels. we set: (a) F" = 0.5. C. =0.2. Co =O. corresponding to Zs = I: (b) P - 0.8. CI -0.1, C2 - O. corresponding to r-tx (c) P -0.8, ei =0.12, C2 rP 0, corresponding to bistability around - 1/3: (d) P -OA. CI -02. C2 - - 018, corre- sponding to an intermediate level of policing around Z. w 0.7986.... branching may occur; Gen tz et al.. 1998) To determine if (17) is an policing, the singular strategy (17) represents a local ESS if ESS. we must look at second-order terms in (12) and (14). (n2 - 4)1'R2 +2(n2 + 4n-4) YR'R+8nPRI2 +2(n2 +n2P+4)/eR <0 For a dominant police allele, we return to (12) with the sub- stitutions (11). We focus on a singular strategy given by (17). For a (21) singular strategy, Cam, is zero to first order in w. To second order in We may alternatively write (21) by substituting for /11 using (17): w. we get (2 +n+nP)21(n2 — 4), 12+2(r,2 +rr2P+4),I tn2 -41P.R2 + 2(n2 + 4n -4)Fiftl Coon, w2 — (n2 —4)(n2 +n2P+4n —4)122R < 0 (22) 16n2R2 Similarly, for a recessive allele that affects intensity of policing, the +8nPR'2 + 2012 +n2P+ 4001 +OW) (18) singular strategy (17) represents a local ESS if 16n2R2 (n-2X2+n+nPfR—(n— 2)2112R +2(2 +n+nP)211 <0 (23) We may alternatively write (18) by substituting for ir using (17): Here. R P, P, R. fe. and le are all functions of the intensity of Can,_ w2 [(2 + n+ nP)2I(n2 — 4),R + 20r2 + n2P+4fle] policing, Z. The local ESS conditions (22) and (23) are quite opaque 16n2R(2+n+ni2)2 and do not allow for simple analysis. Notice that, although the (n —4)(n2+Ii2P+4n-4)P2R1 locations of evolutionarily singular strategies are the same for +000) (19) dominant and recessive mutations that influence policing, the 16n2R(2+n+nP)2 conditions for a singular strategy to be a local ESS are different. For a recessive police allele, we return to (14) with the sub- stitutions (11). We focus on a singular strategy given by (17). For a singular strategy. Cm is zero to first order in w. To second order in 9. Policing and inclusive fitness theory w, we get It has been claimed that policing is a test case of inclusive fit- = w2 tn —2R2+ n+nP)P1—ur — 2)2PQR ness theory (Abbot et .11.. 2011). But the first two papers to theo- Crec 16n2R(2+n+nP)l retically establish the phenomenon (Woyciechowski and Lomnicki. 2(2+n+ nP)2R* ] 1987: Ratnicks. 1988) use standard population genetics; they do + +0(Q) (20) not mention the term "inclusive fitness", and they do not calculate 16n2Ra+n+nP)2 inclusive fitness. Therefore, the claims that theoretical investiga- Inspection of(18) and (20) allows us to determine if a singular tions of worker policing emerge from inclusive fitness theory or strategy is an ESS. If the bracketed quantity multiplying 14,2 is that empirical studies of policing test predictions of inclusive fit- negative, then mutations that change policing in either direction ness theory are incorrect. are disfavored. If the bracketed quantity multiplying w2 is positive, In light of known and mathematically proven limitations of then mutations that change policing in either direction are inclusive fitness theory (Nowak et al.. 2010; Allen et al.. 2013), it is favored. Thus, for a dominant allele that affects intensity of unlikely that inclusive fitness theory can be used to study general EF TA_R 1_02076890 EFTA02702886 114 1W. Olden et al /journal of Theoretkal &elegy 399 (2016) 103-116 questions of worker policing. Inclusive fitness theory assumes that releasing ourselves from the confines of a mathematically limited each individual contributes a separate, well-defined portion of theory, we expand the possibilities of scientific discovery. fitness to itself and to every other individual. It has been shown repeatedly (Cavalli-Sforza and Feldman. 1978; Uyenoyama and Feldman, 1982; Matessi and Karlin. 1984; Nowak ct al., 2010; van it Discussion Veelen et al.. 2014). that this assumption does not hold for general evolutionary processes. Therefore, inclusive fitness is a limited We have derived analytical conditions for the invasion and concept that does not exist in most biological situations. stability of policing in situations where queens mate once or Our work shows that the evolution of worker policing depends several times and where colony efficiency can be affected by on the effectiveness of egg removal (pz) and the consequences of policing. In the special case where policing has no effect on colony colony efficiency (r2). Each of these effects can be nonlinear (not efficiency, our results confirm the traditional view that policing the sum of contributions from separate individuals), with impor- does not evolve for single mating, is neutral for double mating, and tant consequences for the fate of a policing allele. Moreover, the does evolve for triple mating or more than three matings. If colony invasion and stability conditions involve the product of p- and r- efficiency depends linearly or monotonically on the fraction of values, indicating a nontrivial interaction between these two workers that are policing, then our results support the view that effects which does not reduce to a simple sum of costs and ben- multiple mating is favorable to evolution of policing (Ratnieks. efits. We also found that there are separate conditions for invasion 1988). Our results also show that non-monotonic relations in and stability. with neither implying the other. Inclusive fitness colony dynamics and small changes in colony efficiency necessi- theory, which posits a single, linear condition for the success of a tate a more careful analysis. trait, is not equipped to deal with these considerations. We find that policing can evolve in species with singly mated Attempts to extend inclusive fitness theory to more general queens if it causes minute increases in colony efficiency. We find evolutionary processes (Queller. 1992; Frank. 1983; Gardner et al.. that policing does not evolve in species with multiply mated 2011) rely on the incorrect interpretation of linear regression queens if it causes minute decreases in colony efficiency. For non- coefficients (Allen et al.. 2013; see also Birch and Okasha. 2014). monotonic efficiency functions, it is possible that single mating This misuse of statistical inference tools is unique to inclusive allows evolution of policing while multiple mating opposes evo- fitness theory. and differs from legitimate uses of linear regression lution of policing. in quantitative genetics and other areas of science. It was also Our analysis is the first to give precise conditions for both the recently discovered that even in situations where inclusive fitness invasion and stability of policing for both dominant and recessive does exist, it can give the wrong result as to the direction of nat- mutations that effect policing. We study the evolutionary invasion ural selection (Tarnita and Taylor, 2014). and evolutionary stability of policing both analytically and Relatedness-based arguments are often seen in conjunction numerically. For any number of matings, there are four possible with inclusive fitness but there is a crucial difference. Consider the outcomes (see Fig. 5): (i) policing can invade and is stable; (ii) following statement: if the queen is singly mated, then workers policing can invade but is unstable, leading to coexistence; (iii) share more genetic material with sons of other workers than with policing cannot invade but is stable, leading to bistability; (iv) sons of the queen. This statement is not wrong and could be useful policing cannot invade and is unstable. We give precise conditions in formulating evolutionary hypotheses. Such hypotheses can then for all outcomes for both dominant and recessive police alleles. All be checked using exact mathematical methods. outcomes can be achieved with arbitrarily small changes in colony The problem arises when one attempts to formulate the efficiency. quantity of inclusive fitness by partitioning fitness into contribu- Our calculations are not based on any assumption about the tions from different individuals and reassigning these contribu- strength of phenotypic mutation induced by an allele. The condi- tions from recipient to actor. A worker does not make separate tions I . . 91. and (10) also describe the dynamics of mutations contributions to fitnesses of others. and therefore does not have that have an arbitrarily small phenotypic effect on colony Inclusive fitness". Arguments such as "the worker maximizes her dynamics. This facilitates investigation of the evolution of complex inclusive fitness by not policing" are meaningless, since they are social behaviors that result from gradual accumulation of many based on maximizing a nonexistent quantity. Moreover, even mutations (Kapheim et al.. 2016). We derive a simple relation. Eq. when evolution leads individuals to maximize some quantity, that (17), for the location(s) of evolutionarily singular strategies. We quantity is not necessarily inclusive fitness (Okasha and Martens. also derive precise conditions for a singular strategy to be an ESS. 2015; Lehmann et al., 2015), These results are applicable for understanding both the case of It is true that genes (alleles) can be favored by natural selection weak phenotypic effect and the case of weak penetrance. if they enhance the reproduction of copies of themselves in other Our analysis does not use inclusive fitness theory. Given the individuals. But that argument works out on the level of genes and known limitations of inclusive fitness (Nowak et al.. 2010; Allen et can be fully analyzed using population genetics. Inclusive fitness al.. 2013). it is unlikely that inclusive fitness theory could provide a only arises when the individual is chosen as the level of analysis. general framework for analyzing the evolution of worker policing. which is a problematic choice for many cases of complex family or In summary, the main conclusions of our paper are: (i) The population structure (Akcay and Van Clew. 2016). prevalent relatedness-based argument that policing evolves under Bourke (20111 has proposed that inclusive fitness remains valid multiple mating but not under single mating is not robust with as a concept even when it is nonexistent as a quantity. But why is respect to arbitrarily small variations in colony efficiency; (ii) for such an uninstantiable concept useful? The mathematical theory of non-monotonic efficiency functions, it is possible that policing evolution is clear and powerful. Exact calculations of evolutionary evolves for single mating, but not for double or triple mating; (iii) dynamics (Antal et al.. 2009; Allen and Nowak. 2014; Fu et al.. 2014; careful measurements of colony efficiency and the fraction of Haven and Doebelt, 2004; Szabo and Kith. 2007; Antal and queen-derived males are needed to understand how natural Scheming. 2006; Traulsen et al., 2008; van Veelen et al.. 2014; selection acts on policing; (iv) contrary to what has been claimed Simon ct al.. 2013) demonstrate that inclusive fitness is not needed (Abbot et al.. 2011), the phenomenon of worker policing is no for understanding any phenomenon in evolutionary biology. This empirical confirmation of inclusive fitness theory: the first two realization is good news for all whose primary goal is to understand mathematical papers on worker policing (Woyacchowski and evolution rather than to insist on a particular method of analysis. By lomnicki. 1987; Ratnieks. 1988) do not use inclusive fitness theory. EFTA_R1_02076891 EFTA02702887 J.W. °Wars et ol. / Journal of Theoretkal Biology 399 (20:6)103-n6 115 The present paper, which also does not use inclusive fitness the- Gadagkar. It. 2001. 'Ilk Social Biology of Ropalidia hlargiluu: low.u4 Under- ory. is the first detailed analysis of policing for any number of standing the Evolution of EuSOCiality. Harvard University Press. Cambridge. MA. Cadagkar. R, 2004. Why do honey bee workers destroy each other's eggs? J. Mout coatings and taking into account effects on colony efficiency. 29(3). 213-217. Cardneu. A.. West, SA. Wild. G.. 2011. The generical theory of kin selection. J. Evol. Riot 24. 1020-1043. C,eritz. SAIL. Kisdi. E., Meszena. G.. Metz. JAL 1998. Evolutionarily singular stra- Aclmowledgments tegies and the adaptive growth and branching of the evolutionary tree. Evol. Earl. 12. 35-57. We are grateful to the referees and editor for helpful comments Hamilton. W.D.. 1972. Altruism and 'elated phenomena, mainly in social Insects. Annu. Rev. Ecol. Syst. 31. 193 232. that have significantly benefited this manuscript. This publication liammond, RI_ Keller. I.. not Conflict over male parentage in social insects. MOS was made possible through the support of a grant from the John Biol. 2 (e248). 1472 1482. Templeton Foundation. The opinions expressed in this publication Han:Tuna, A., Wantu, J.. 'Torres. JA„ Heiner. J., 2003. Worker policing without genetic conflicts in a clonal ant. Proc. Nail. Acad. So. USA IOU 12836-12840. are those of the authors and do not necessarily reflect the views of Hawn. C.. Doebeli, M.. 2004. Spatial structure often inhibits the evolution of the John Templeton Foundation. cooperation in the snowdrift game. Nature 428. 643-646. I ielantria. II.. TolfiSki. A.. Wenseleers. 1', %Hoicks. F.LW.. 2006. Worker policing in the common wasp liespncla sultans is not aimed al Improving colony hygiene. lusectes Soc. 53. 399-402. Appendix A. Supplementary data titian:era. H.. Sundstrons L. 2007. Worker reproduction in formica ants. Am. Nat. 170. E14-F25. Ifolbaues J.. Sigmund. le. 1990. Adaptive dynamics and evolutionary stability. Appl. Supplementary data associated with this paper can be found in Math. Lett. 3. 75 79. the online version at http://dx.doi.org/10.1015/Kitbi.2016.03.001. Hughes. W,O.H.. Oldroyd. lilt. Beekman. M.. Ratnieks. FLW.. 2008. Ancestral monogamy shows kin selection is key to the evolution of eusociality. Science 320. 1213-1216. Kapheim. KM.. Pan, /I, ti. C.. SalzIwig. S,1-, Ruin 1).. et al„ 2015. Cenomic signalores References of evolutionary riansitions from solitary to group living. Science 348. 1139-1143. Khila, A.. Abouheif. E.. 2008. Reproductive constraint is a developmental mechan- Abbot. C, er al.. 2011. Inclusive fitness theory and eusociality. Nature 471. E1-F9. ism that maintains social harmony in advanced ant societies. Proc. Natl. Acad. Alkay. E. Van Cleve. J.. 2016. There is no fitness but fitness. and the lineage is its Sci. U.S.A. 105. 17889-17889. bearer. Philos Trans. R. Soc. 13371. 20150085. Lehmann. I., Alger. I.. Weihull, J.. 2015. Does evolution lead to maximizing beha- Allen. Et, Nowak. MA.. Wilson, E,0.. 2013. Limitations of inclusive fitness. Proc. Nail, vior? Evolution 60 1858 1873. Mad. 50. U.SA 110.20135-20139. Matessi, C. Karlin. S.. 1984.0n the evolution of altruism by kin selection. Proc. Natl. Allen. B.. Nowak. MA„ 2014. Games on graphs. EMS Sun'. Math, Sci. I (I ). 115-151. Acad. So. U.SA. 81. 1754 1758, natal. I.. Scheming. 2006. Fixation of strategies for an evolutionary game in finite Metz. J.A.J. Cecil,. SA.11. Meszena. C. Jacobs. F.JA. van Ikerwaarden. IS.. 1996. populations. Bull, Math. Biol. 68, 1923-1944. Adaptive dynamics, a geometrical slimly of the consequences of nearly faithful Antal. T.. ()Musks It,. Wakeley. J.. Taylor. 1W.. Nowak. MA.. 2009. Evolution of repregluction. In: van Sums S.J.. Verduyn Luna S.M. (Eds.). Stochastic and cooperation by phenotypic similarity, Proc. Natl. Mad. Sci. USA. 106 (21), Spatial Structures of Dynamical Systems. KNAW Verhandelingen. Afel.. 8597 8600. Amsterdam. pp. 183-231. Beekman. M.. Oldroyd, B.R. 2005. Honeybee workers use cues mho than egg via• Nancy. N.L.. Peso, M.. Even. N.. Barron. A.B.. Robinson. G.E.. 2013. Altivistic behavior Wiry (or policing. Biol. Lets I. 129-132. by egg-laying worker honeybees. Curt. Biol. 23. 1574-1578. Birch. J.. Okasha S., 2014, Kin selection and its critics. koScience 65. 22-32. Nonacs, It. 2006. Nepotism and brood reliability in the suppression of worker Bonckaert. W.. Vuefinekt. K. Billen, J., Hammond. RL. Kellen. L. Wenseleers. T.. reproduction in the emocul hymenoptera. Biol. lett. 2, 577-574. 2008. Worker policing in the German wasp Vespula germaniro. Behay. Ecol. 19. Nowak. MA. Sigmund, K.. 1990. The evolution of stochastic strategies in the pris- 272-278. oner's dilemma. Acta Appl. Math. 20. 247 265. gonekarts W. van Zweden. JS„ &Entine. P.. Billets J. Wenseleers. L. 2011. Colony Nowak, MA., tonna. Cl., Wilson. CO.. 2010. The evolution of eusociality. Nature stage and not facultative policing explains pattern of worker reproduction in 466,1057-1062. the Saxon wasp. Mol. Ecol. 20. 3455 3468. Okasha, S.. Martens. L. 2016. Hamilton's rule, inclusive fitness maximization. and Boomuna. IL 2007. Kin selection versus sexual selection: why the ends do not the goal of individual behaviour in symmetric two-prayer games. J. Evol. Biol. Meet (WE Biol. 17. R673-K6133. 29. 473-482. BoomIma, J,f.. 2009. lifetime monogamy and the evolution of euuniality. Dicker, J.W„ Allen. B., Vellet, C., Nowak, MA, 2015. The evolution of non Trans. R. Soc. B 364. 3191 3207. Bourke. A.F.G.. 2011. The validity and value of inclusive fitness theory. Proc. R. Soc. B reproductive workers in insect colonies with haplocliplold genetics elite 4. 278. 3313-3320. e08918. Uvalli-Sforza. LI.. Feldman. MW.. 1978.1)aiwinian selection and lheor. Peters. JAL. Quelkr. D.C.. Imperatriz-Fonseca. V.L. Roulade. D.W_ Strassmann. Popul. Biol. 14. 268-280. 1999. Mate number, kin selection and social conflicts in stingless bees and Cole. Rt. 1986. the social behavior 01 Scow :home ollardynii wvenoptera. For honeybees, Proc. R. Soc. loud. B 266. 379 384. micidae): rime budgets and the evolution of worker reproduction. Behay. Ecol. Pint, C.W.W., Neumann, P. Hepburn. R.. hloritz. R.I.A.. Tautr. J.. 1999. Fgg viability Sociothol, IS. 165-173. and worker policing in honey bees. Proc. Natl. Acad. Sol. USA 101, 8649 8651. C.K. West, S.A., Davis. KF,., Griffin. AS., 20ln Promiscuity and the Queller. D.C.. 1992. A general model for kin selection. Evolution 46. 376-380. evolutionary transition to complex societies, Nature 466. 969-972. Queller, D.C., Strassmann, J.E.. 1998, Kin selection and .4(ICUI insects. Rinscience 48. Dieckmann, U.. law, R.. 1996. The dynamical theory of coevolution: a derivation 165-175. from stochastic ecological processes. J. Math. Biol. 34. 579-612. Ratnieks. F.I.W., 1988. Reproductive harmony via mutual policing by workers in Endkr. A.. t.iebig. J.. Schmitt. T. Patter. It. Jones. G.K. Schroer, Holldobler. B.. cusocial hymenoptera. Ant Nat. 132. 217-236. 2004. Surface hydrocarbons of queen eggs regulate woiket teproduction in a Ratnieks. FLW..1Asscher. P.K. 1989. Worker policing in the honeybee. Nature 342. social insect. Proc Natl. Mad. Sc,. U.S.A. 101. 2945-2950. 796-797. Poster. KR.. Ratnieks. ELW.. 2000. Facultative worker policing in a wasp. Nature Ratrucks. ELW., Foster. Kit, Wenseleers, T. 2006. Conflict resolution in insect 407.6312-693. societies. Annu. Rev. Entomol. SI, 581-608, Foster. KR_ Raimeks. F.LW.. 2001a. Convergent evolution of worker policing by egg Seeley. 1.D.. 1985.11oneybee Ecology: A Study of Adaptation in Social Life. Princeton eating in the honeybee and common wasp. Proc. R. Soc. Land. 8268, 169-174. University Press. Princeton. NJ. Foster. KR.. Ratmeks. F.LW.. 2001b. The effect of sex-allocation biasing on the Simon, B. l'sr:Int. IA.. Doeben M. 7013 general theory of group evolution of worker policing in hymenopteran societies. Ant Nat. 158.615-624. stir • • • • , h7, 1561- I572. Foster. K.R.. Rainglis r.t.w.. 2001c. Paternity. tepreduCtion and conflict in vespine Start C.K., 1984. In. Smith. R.L (Ed.). Sperm Competition. Kinship• and Sociality in wasps: a model system for testing kin selection predictions. &bar. Ecol. the Acoleate I Ivmenoplera. Academic Press. Orlando, Florida. pp. 427-464. Sotiobiol. 50. 1-8. Szabo. C., .' Evolutionary gum. tin glimils Illy. Rep. 446. 97 216. Foster, 1CR.. Gulliver. J.. Ratnieks. F.LW„ 2002. Worker policing in the European Tamita, C.E. Taylor. P.D. 2014. Measures of relative fitness of social behaviors in hornet Wspo Erato°. Insectes Soc. 49.41-44. finite structured population models. Am. Na. 184(4) in press. FroMes K.R.. Wenseleers. T. Ratineks. F.LW. 2006. Kin selection is the key to Iraulsen. A., Shoreils N. Nowak, M A. 2008. Analytical results for individual and altruism. trends Ecol. Ewa 21. 57-60. group selection of any intensity. MI. Math. Biol. 70, 1410-1424. Frank. S,A.. 1983. Foundations of Social Evolution. Princeton University Press. Dyenoyania, M.K., Feldman, M.. 1982. Population genetic theory of kin selection. II, Princeton. NJ. The multiplicative model. Am. Nat. 120, 614 627. It. E. Wicker. S.D.. Nowak. MA. 2014. The risk-return trade-olf between solitary van Verlen. M., WO. S.. Simon. 8.. 2014. A simple model of group selection that and eusonal trproduction. Fxol. Lett 19 (Ile 74-84. unnol be analyzed with inclusive fitness. J. lheor. Biol. 360. 279-289. EFTA_R1_02076892 EFTA02702888 116 J.W. °kiwi et al /Journal of Theoretical Biology 399(2016)103-116 Winans, H.H.W.. de Anus, Alves, Lt. Impeistinz-lonseca. V.L. Jose. M.. 2002. Wenseleers, r, Itatowks. [LW.. 20066. Comparative analysis of worker reproduc- Worker bees and the (ate of their eggs. Proc. Exp. Appl. Eniomol. NEV tion and policing in eusocul hyownopteia supports relatedness theory. Am. Amsterdam 13. 97-102. Nat. 168. E163-E179. Wenseleers. T.. Helantera, H_ Han. A. Ratnieks. F.LW..200.4a. Winker reproduction Wenseleers, T.. Itelantera. H.. Alves. U.A., Otienez-Cusman. E.. ' ,ample. P.. 2013. and policing in insert societies: an FS5 analysis. J. (vol. Riot. 17. 1035-1047. Towards greater realism in inclusive fitness models: the case of worker Wenseleers. T. Han. A6. Ratnieks. F.LW.. 2004b. When sesistaixe is useless: reproduction in insect societies. lhol. let'. 9, 20130334. policing and the evolution of reproductive acquiescence in insect societies. Am. Wilson. Ell.. 1971. the Insect Societies. Harvard trowel soy Press. Cambridge, MA. Nat. 164, E154 E167. tArowierhowski. M.. lommtki. A.. 1987. Multiple mating or queens and the sterility Wenseleers. T.. &Wrack. N.S. given. K.. tofilski. A, Nascimenio. ES., Hari. A.G.. of workers among eusocial hymenoptera. J. 'fluor. Riot. 128. 317 327. Burke. TA. Archer. Ratnieks. 2005. A test of worker policing theory 2anette. IRS. Miller. 5.0.1. Stria, C.M.A. Almond. E.J.. Huggins. T.1.. Jordan. W.C.. in an advanced eusocul wasp, Ye:sputa ndo. Evolution 59. 1306-1314. Boinke.A.F.C.. 2012. Reproductive conflict in bumblebees and the evolution of Wenseleers. T.. Ratnieks. F.LW., 2006a. Enforced altruism in insect societies. Nature worker policing. Evolution 66, 3765-3777. 444. 50. EFTA_R1_02076893 EFTA02702889