EFTA01070997

Journal of Theoretical Biology 399 (2016) 103-116 Contents lists available at ScienceDirect Jam. 1or Theoretical lhoOgy Journal of Theoretical Biology journal homopago::::vw.elsevier.com/locate/yjtbi Evolution of worker policing CrossMatk Jason W. Olejarza, Benjamin Allen b•a'c, Carl Veller aA, Raghavendra Gadagkai Martin A. Nowaka•d-gs ▪ Program for Evolutionary Dynamic. Harvard University, Cambridge. MA 02738, USA b Department of Mathematics. Emmanuel Cortege, Boston, MA 02115. USA Center for Mathematical Sciences and Applications. Harvard University, Cambridge. MA 02738, USA ▪ Department of Organismic and Evolutionary Biology. Harvard University, Cambridge, MA 0213B, USA ' Centre for Ecological Sciences and Centre for Contemporary Studies. Indian Institute of Science, Bangalore 560 012. India Indian National Science Academy, New Delhi 170 002. India 2 Department of Mathematics, Harvard University, Cambridge. AM 02138, USA ARTICLE INFO ABSTRACT Ankle history. Workers in insect societies are sometimes observed to kill male eggs of other workers, a phenomenon Received 2 February 2015 known as worker policing. We perform a mathematical analysis of the evolutionary dynamics of policing. Received in revised form We investigate the selective forces behind policing for both dominant and recessive mutations for dif- 23 January 2016 ferent numbers of matings of the queen. The traditional, relatedness-based argument suggests that Accepted 2 March 2016 Available online II March 2016 policing evolves if the queen mates with more than two mates, but does not evolve if the queen mates with a single male. We derive precise conditions for the invasion and stability of policing alleles. We find Keywords: that the relatedness-based argument is not robust with respect to small changes in colony efficiency Sociobiology caused by policing. We also calculate evolutionarily singular strategics and determine when they are Natural selection evolutionarily stable. We use a population genetics approach that applies to dominant or recessive Evolutionary dynamics Models/simulations mutations of any effect size. e. 2016 Elsevier Ltd. All rights reserved. 1. Introduction (Hughes et al., 2008: Cornwallis et al.. 2010: Queller and Strassmann, 1998: Foster et al.. 2006: Boomsma, 2007, 2009). In contrast• it is In populations with haplodiploid genetics. unfertilized female believed that polygamy—not monogamy—is important for the evo- workers are capable of laying male eggs. Thus, in a haplodiploid lution of police workers. colony, male eggs can in principle originate from the queen or Several papers have studied the evolution of policing. Starr from the workers. Worker policing is a phenomenon where female (1984) explores various topics in the reproductive biology and workers kill the male eggs of unmated female workers (Ratnieks, sociobiology of eusocial Hymenoptera. He defines promiscuity as 1988: Ratnieks and Visscher.1989: Ratnieks et al„ 2006: Gadagkar, 1/(E7_,J1), where n is the number of matings of each queen. and 2001: Wenseleers and Ratnieks, 2006a). Worker policing is .h is the fractional contribution to daughters by the i-th male mate. observed in many social insects, including ants, bees, and wasps. He writes, regarding workers, that "They are on average less Yet the precise conditions for the evolution of worker policing are related to nephews than brothers whenever (promiscuity is still unclear. greater than twol and should prefer that the queen lay all the male Worker policing (Ratnieks, 1988: Rah-licks et al.. 2006: Gadagkar, eggs. Workers would therefore be expected to interfere with each 2001: Wenseleers and Ratnieks, 2006a) and worker sterility (Wilson, others reproduction.' Thus, Starr (1984) was the first to suggest that workers should raise their nephews (sons of other workers) if 1971: Hamilton, 1972: Olejarz et al.. 2015) are two distinct phe- the queen mates once, but should only raise their brothers (sons of nomena that are widespread in the eusocial Hymenoptera. In addi- the queen) if the queen mates more than twice. Starr (1984) uses a tion to worker policing. a subset of workers in a colony may forego relatedness-based argument, but he does not provide any calcu- their own reproductive potential to aid in raising their siblings. Prior lation of evolutionary dynamics in support of his argument: he relatedness-based arguments have suggested that queen monogamy uses neither population genetics nor inclusive fitness theory. In a is important for the evolution of a non-reproductive worker caste book on honeybee ecology. Seeley (1985) also proposed, using a relatedness-based argument, that worker policing should occur in 'Corresponding author. colonies with multiply mated queens, but that worker policing E-mail address: martin_nowaloPharvard.edu (MA Nowak). should be absent if queens are singly mated. http:fidx.dokorg/10.1016/mtbi.2016.03.001 0022-51930 2016 Elsevier Ltd. All rights reserved. EFTA01070997 104 OkJorz et al /Journal of theatrical Biology 399 (2016) 103-116 Woyciechowski and Lomnicki (1987) perform a calculation studied species to date that are multiply mated.) Worker removal based on population genetics and conclude that workers should of worker-laid eggs is much less prevalent in colonies of the raise their nephews (sons of other workers) if the queen mates bumblebee (Velthuis et al.. 2002), the stingless bee. (Peters et al.. once, but should only raise their brothers (sons of the queen) if the 1999), and the wasp, Vespula rub (Wenseleers et al.. 2005). which queen mates more than twice—the case of double mating is neu- are predominantly singly mated. (As mentioned above, worker tral with respect to preference. From this result, they claim that. policing has been found only in about 20% of the studied species to under multiple mating of the queen, natural selection should favor date that are singly mated.) There are some studies based on non-reproductive workers. Woyciechowski and Lomnicki (1987) observational evidence that find policing in singly mated species: consider both dominant and recessive alleles affecting worker examples of species with single mating and worker policing are behavior, but they do not consider colony efficiency effects. Vespa crabro (Foster et al., 2002), Camponotusfloridanus (Endler et Ratnieks (1988) considers the invasion of a dominant allele for al.. 2004). Aphaenogasrer smythiesi (Wenseleers and Ratnieks. policing. Using population genetics, he arrives at essentially the 2006b), and Diacamma (Wenseleers and Ratnieks, 2006b). same conclusion as Woyciechowski and Lomnicki (1987): In the Interspecies comparisons are somewhat problematic, because absence of efficiency effects, policing evolves with triple mating even though phylogeny can be controlled for, there are many but not with single mating. But Ratnieks also considers colony (known and unknown) ways in which species differ in addition to efficiency effects, focusing mainly on the case where policing mating frequency that may also affect the absence or presence of improves colony efficiency. Since policing occurs alongside other worker policing. Furthermore. many empirical studies are based maintenance tasks (such as cleaning of cells, removal of patho- on genetic analyses of male parentage. (Though studies of some gens. incubation of brood), and since eating worker-laid eggs species are based on actual observational evidence: see, e.g., might allow workers to recycle some of the energy lost from laying Wenseleers and Ratnieks. 2006b.) Regarding species for which the eggs, Ratnieks supposes that policing improves colony efficiency. study of policing is based on genetic analyses, policing is often He finds that worker policing with singly mated queens may inferred if males are found to originate predominantly from the evolve if policing improves colony reproductive efficiency. He also queen. But such an inference. in cases where it is made. pre- finds that worker policing with triply mated queens may not supposes that workers actively try to lay male eggs in the first evolve if policing reduces colony reproductive efficiency, but he place. It is therefore not clear how reliably genetic investigations considered this case to be unlikely on empirical grounds. Ratnieks can measure policing. does not study recessive policing alleles. He also does not calculate The small number of attempts at measuring the prevalence of evolutionary stability conditions. worker policing in intraspecific experiments have also returned Both papers (Woyciechowski and Lomnicki. 1987: Ratnieks, conflicting results. Foster and Ratnieks (2000) report that facul- 1988) offer calculations based on population genetics without tative worker policing in the saxon wasp. Dotichovespula scuronica, mentioning or calculating inclusive fitness. These early studies is more common in colonies headed by multiply mated queens. (Starr, 1984: Seeley, 1985: Woyciechowski and Lomnicki, 1987: But their sample size is only nine colonies. The phenomenon was Ratnieks. 1988) were instrumental in establishing the field of reinvestigated by Bonckaert et al. (2011) who report no evidence worker policing. of facultative worker policing depending on queen mating fre- Testing theoretical predictions on the evolution of worker quencies. and argue that the previous result may have been flawed policing in the field or in the lab is difficult. Due to the complex- or that there were interpopulational variations. ities inherent in insect sociality, published empirical results are not Many empirical studies have emphasized that factors besides always easy to interpret. While, so far, worker policing has been intracolony relatedness—including the effects of policing on a found in all species with multiple mating that have been studied, it colony's rate of production of offspring—may play a role in has also been found in about 20% of species with singly mated explaining evolution of worker policing (Foster and Ratnieks queens (Hammond and Keller, 2004: Wenseleers and 2001a,c: Hartmann et al., 2003: Hammond and Keller, 2004: Ratnieks. 2006b: Bonckaert et al.. 2008). Herein lies the difficulty: Wenseleers and Ratnieks. 2006b: Helantera and Sundstrom, 2007: When worker policing is found in multiply mated species and Khila and Abouheif, 2008: Zanette et al.. 2012). Yet reliable pub- found to be absent in singly mated species, this is taken as evi- lished data on the effect that policing has on colony reproductive dence supporting the relatedness argument, and when worker efficiency are often hard to find. (For some exceptions, see policing is found in singly mated species, it is explained away as Wenseleers et al., 2013 and references therein.) not being evidence against the theory, but as having evolved for In this paper. we derive precise conditions for the evolutionary other reasons (such as colony efficiency). See, for example, the invasion and evolutionary stability of police alleles. We consider following quotation by Bonckaert et al. (2008): "Nevertheless, our any number of matings, changes in the proportion of queen- results are important in that they show that V. germanico forms no derived males. changes in colony efficiency, and both dominant exception to the rule that worker reproduction should be effec- and recessive mutations that affect the intensity of policing. tively policed in a species where queens mate multiple times Our paper is based on an analysis of evolutionary dynamics and (Ratnieks. 1988). Indeed, any exception to this pattern would be a population genetics of haplodiploid species (Nowak et al., 2010: much bigger challenge to the theory than the occurrence of Olejarz et al.. 2015). It does not use inclusive fitness theory. Spe- worker policing in species with single mating, which can be cifically. we adapt the mathematical approach that was developed readily explained (Ratnieks. 1988: Foster and Ratnieks, 2001b)' by Olejarz et al. (2015) for the evolution of non-reproductive This is precisely why a careful simultaneous consideration of workers. We derive evolutionary invasion and stability conditions relatedness, male parentage, and colony efficiency is important for for police alleles. Mathematical details are given in Appendix A. understanding worker policing. In Section 2, we present the basic model and state the general We do not aim to provide an exhaustive catalog of all species in result for any number of matings for dominant policing alleles. In which worker policing has been studied. We merely cite some Sections 3-5. we specifically discuss single. double, and triple specific examples to add context Policing is rampant in colonies of mating for dominant policing alleles. We take dominance of the the honeybee (Ratnieks and Visscher, 1989), the wasp Vespula policing allele to be the more realistic possibility because the vulgaris (Foster and Ratnieks. 2001c), and the wasp Vespula ger- policing phenotype is a gained function. Nonetheless. for com- manica (Bonckaert et al.. 2008), which are all multiply mated. (As pleteness. we give the general result for recessive policing alleles mentioned above, worker policing has been found in all of the in Section 6. In Section 7. we discuss how the shape of the EFTA01070998 J.W. OleJeri et al. /Journal of theoretical Biology 399 (2016) 103-116 105 efficiency function determines whether or not policing is more a likely to evolve for single or multiple matings. In Section 8. we Virgin n Males Fertilized Queens Queens analyze our results for the case where the phenotypic mutation induced by the mutant allele is weak (or. equivalently in our AA +(n —m) A +m a AA,m formalism, the case of weak penetrance). In this setting, the quantity of interest is the intensity of policing. We locate the evolutionarily singular strategies. These are the values of intensity Aa +(n -m) A +m a Aa,m of policing for which mutant workers with slightly different policing behavior are, to first order in the mutant phenotype, aa +(n —m) A +m a aa,m neither advantageous nor disadvantageous. We then determine if a singular strategy is an evolutionarily stable strategy (ESS). In Section 9. we discuss the relationship between policing and b OM Ousts MO•Onellos MO inclusive fitness theory, together with the limitations of the Oman Gomm Oen mina Man On VIManelms MO AA A A relatedness-based argument. Section 10 concludes. M /121 As A A•• MA AA•Pa An 5A•• AM *aim An Ana ma As A•• M1 la a 2. The model C IsehOINOIMOE • TOM We investigate worker policing in insect colonies with haplo- Omen Omega Omen Oman I Clain- MINIONIfielt• I la na AnmPa oil A ft...inns diploid genetics. Each queen mates n times. We derive conditions Mon 05-MM•Mann A • Ilp fan-r••ta•h•Ael.- under which a mutation that effects worker policing can spread in 16 -nIsa•maa In-MA•ln•mla a population. We make the simplifying assumption, as do Flg. I (a) The possible mating events with haplodiploid genetics are shown. Each Woyciechowski and Lomnicki (1987) and Ratnieks (1988), that the queen mates with n males. in denotes the number of times hat a queen mates with mutant type a males and can take values between 0 and n Thus, there are 3(n +1) colony's sex ratio is not affected by the intensity of worker types of colonies. (b)Ifeach queen mates with only a single male, then there are six policing. types of colonies. The female and male offspring (right hree columns) of each First we consider the case of a dominant mutant allele. Because colony (leftmost column) are shown. For example. AA. I co onies arise from a type AA female mating with a single mutant type a male. M.1 queens produce female the policing allele confers a gain of function on its bearer, the offspring of type Aa and male offspring of type A. 50% of the offspring of workers in assumption that it is dominant is reasonable. There are two types M. I colonies are of type A, while the remaining 50% of the offspring of workers in of males. A and a. There are three types of females. AA, Ac. and aa. M. I colonies are of type a. (c) The female and male offspring (right three columns) If the mutant allele is dominant, then Aa and aa workers kill the of each colony (leftmost column) when each queen mates it times are shown. male eggs of other workers, while AA workers do not. (Alter- natively. AA workers police with intensity 44 while Aa and aa Possible Forms for the Function p : workers police with intensity ZA0 = Z 2a = 44+w. We consider this case in Section 8.) For n matings, there are 3(n + 1) types of mated 0 queens. We use the notation AA. m: Aa,m: and aa,m to denote the 0 2 0.75 E genome of the queen and the number. m. of her matings that were with mutant males, a. The parameter m can assume values rn 0,1...., n. A schematic of the possible mating events is shown in gi 0.50 Fig. 1(a). There are three types of females, AA. Act, and ca. and there are n+1 possible combinations of males that each queen can mate wo 0.25 with. (For example, a queen that mates three times (n=3) can O mate with three type A males, two type A males and one type a 0 male, one type A male and two type a males, or three type a LI- 0 0.25 0.50 0.75 males.) Fig. 1(b) shows the different colony types and the offspring Fraction of police workers, z of each type of colony when each queen is singly mated. Fig. 1 Fig 2. The queen's production of male eggs, p., increases with the fraction of (c) shows the different colony types and the offspring of each type workers that are policing, 2. This is intuitive, since having a larger worker police of colony when each queen mates n times. The invasion of the force means that a greater amount of worker-laid eggs can be eaten of removed. mutant allele only depends on a subset of colony types. The cal- Three possibilities for a monotonically increasing function pt are shown. culations of invasion conditions are presented in detail in 22. Colony efficiency as a function of policing Appendix A. re represents the rate at which a colony produces offspring 2.1. Fraction of male offspring produced by the queen (virgin queens and males) if the fraction of police workers is z. (This quantity was also employed by Ratnieks. 1988.) Without loss Pr represents the fraction of males that are queen-derived if the of generality, we can set ro = 1. For a given mutation that affects fraction of police workers is z. (This quantity was already the intensity of policing, and for a given biological setting, the employed by Ratnieks. 1988.) The parameter z can vary between efficiency function re may take any one of a variety of forms 0 and 1. For z=0. there are no police workers in the colony, and for (Fig. 3). z=1. all workers in the colony are policing. We expect that pr is an Colony efficiency depends on interactions among police work- increasing function of z. Increasing the fraction of police workers ers and other colony members. It also depends on the interactions increases the fraction of surviving male eggs that come from the of colonies and their environment. There are some obvious nega- queen (Fig. 2). tive effects that policing can have on colony efficiency. By the act EFTA01070999 106 OkJarz et al. 1puma, of theoretical Biology 399 (2016)193-116 Possible Forms for the Function r the entire worker population is unclear. It is possible that a frac- tion ze 1 of police workers can effectively police the entire 1.3 population, and adding additional police workers beyond a certain point could result in wasted energy, inefficient use of colony 1.2 resources, additional recognitional errors. etc. These effects may 0 correspond to colony efficiency rz reaching a maximum value for 1.1 some 0 < z < 1. As another possibility, suppose that police workers, when their number is rare, directly decrease colony efficiency by the act of z 0.9 killing male eggs. It is possible that for some z < 1. police workers are sufficiently abundant that their presence can be detected by 0.8 other workers. Assuming the possibility of some type of facultative response, the potentially reproductive workers may behaviorally 0 0.25 0.50 0.75 adapt by reducing their propensity to lay male eggs. instead Fraction of police workers, z directing their energy at raising the queen's offspring. In this scenario. colony efficiency rz may reach a minimum value for some Hg. 3. The functional dependence of colony efficiency. rz. on the fraction of 0 ez e 1. workers that are policing. z, may take any one of many possibilities. 2.3. Main results for dominant police alleles of killing eggs, police workers are directly diminishing the number of potential offspring. In the process of identifying and killing We derive the following main results for dominant police nephews, police workers may also be expending energy that could alleles. If the queen mates with n males, then the a allele for otherwise be spent on important colony maintenance tasks (Cole. policing can invade an A resident population provided the fol- 1986: Naeger et al.. 2013). Policing can also be costly if there are lowing -evolutionary invasion condition" holds: recognitional mistakes. Le, queen-laid eggs may accidentally be removed by workers. Recognitional errors could result in modifications to the sex ratio, which is an important extension of Ptin +Pia Cm) (rig) >2 _ ro ro CI/2) _ ro _ Pito Cin) ro (1) our model but is beyond the scope of this paper. When considering only one mutation. ro can be set as 1 without We can also identify positive effects that policing may have on loss of generality. Why are the four parameters. r im. r la, pun. and colony efficiency. It has been hypothesized that the eggs which are pia. sufficient to quantify the condition for invasion of the mutant killed by police workers may be less viable than other male eggs allele, a? Since we consider invasion of a. the frequency of the (Velthuis et al.. 2002: Pirk et al., 1999: Cadagkat 2004: Nonacs. mutant allele is low. Therefore, almost all colonies are of type 2006), although this possibility has been disputed (Beekman and AA, 0. which means a wild-type queen. AA, has mated with n wild- Oldroyd. 2005: Helantera et al., 2006: Zanette et al., 2012). If less- type males. A, and 0 mutant males, a. In addition, the colonies Act 0 viable worker-laid eggs are competing with more-viable queen-laid and AA, 1 are relevant These are all colony types that include male eggs, then policing may contribute positively to overall colony exactly one mutant allele. Colony types that include more than one efficiency. Moreover, policing decreases the incentive for workers to mutant allele (such as Aa, 1 or AA,2) are too rare to contribute to expend their energy laying eggs in the first place (Foster and the invasion dynamics. For an Ac. 0 colony, half of all workers are Ratnieks, 2001a: Wenseleers et al.. 2004a,b: Wenseleers and policing, and therefore the parameters r112 and p1/2 occur in Eq. Ratnieks, 2006a), which could be another positive influence on (1). For an AA. 1 colony, 1/n of all workers are policing, which colony efficiency. (However, the decrease in incentive for workers to explains the occurrence of r un and pim in Eq. (1). reproduce due to policing would only arise on a short time scale if Next. we ask the converse question: What happens if a popu- there is a facultative response to policing, which is unlikely.) lation in which all workers are policing is perturbed by the As another speculative possibility: Could it be that worker egg- introduction of a rare mutant allele that prevents workers from laying and subsequent policing acts as a form of redistribution policing? If the a allele for worker policing is fully dominant. and if within the colony? That is, suppose that it is better for colony colony efficiency is affected by policing, then a resident policing efficiency to have many average-condition workers than to have population is stable against invasion by non-police workers if the some in poor condition and some in good condition. Suppose following 'evolutionary stability condition" holds: further, as seems realistic, that good-condition workers are more +n)(2 +Pi )+Pan- oita,o(n —2) (2) likely to lay eggs (which are high in nutritional content. of course) If the average police worker is of condition below the average egg- roan-mart 2(2+n+npi ) laying worker, then worker egg-laying and policing serves to What is the intuition behind the occurrence of the four para- redistribute condition among the workers, improving overall col- meters, ri, ran _ wan). Pt. and Pan-102M? The condition applies to ony efficiency. a population in which all workers are initially policing. Note that. The special case, where policing has no effect on colony effi- because the allele. a, for policing is fully dominant in our treat- ciency and which has informed the conventional wisdom. is ment. non-policing behavior arises if at least two mutant A alleles ungeneric, because policing certainly has energetic consequences for non-policing are present in the genome of the colony, which is for the colony that cannot be expected to balance out completely. the combination of the queen's genome and the sperm she has An early theoretical investigation of colony efficiency effects stored. To study the invasion of a non-policing mutant allele. we regarding invasion of dominant mutations that effect worker must consider all colony types that have 0.1. or 2 mutant A alleles: policing was performed by Ratnieks (1988). these are aa.n: aa,n — 1: Aa,n: n-2: Aa,n-1: and AA, n. The Although monotonically increasing or monotonically decreas- colonies aa.n: mai —1: Aa,n: aa, n —2: and AA, n do not contain ing functions rz are the simplest possibilities, these cases are not non-police workers: the efficiency of those colonies is r 1, and the exhaustive. For example, a small or moderate amount of policing fraction of male eggs that originate from the queen in those may be expected to improve colony efficiency. However, the pre- colonies is pi. Both of these parameters occur in Eq. (2). Colonies of cise number of police workers that are needed to effectively police type At n —1 produce a fraction of 1/(2n) non-police workers. EFTA01071000 J.W. Oleforz er al. /Journal of theoretical Biology 399 (2016) M3-116 107 Single mating. n=1 Double mating, n=2 0 10 10 1 To 104 '1104 .6 10 4 4 10 1. !1.1. 104 .7 2 6 Time (104) Fig. a Numerical simulations of the evolutionary dynamics of worker policing confirm the condition given by Eq. (1). The policing allele is dominant. For numerically probing invasion, we use the initial condition XA40 —1-10 s and Kai — 10 a. We set ra —1 without loss of generality. Other parameters are: (a) pi,2 —0.75, pi —0.9, and rig -1A1: (b)pl a —0.6, p,—0.8. — 1.005, and r,-1.01. Police allele can invade and is evolutionarily stable can invade, but is not stable cannot invade. but is stable cannot invade and is not stable 0 1 kequency of police allele S. There are four possibilities for the dynamical behavior in the proximity of two pure equilibria a n= 1 mating P1/2 = 0.75. p1 =1 b n = 1 mating pv2 = 0.99. Pi = I 1.2 1.2 Invades Stable Colony efficiency. r1 Does Not 1.1 Stable Invade Invades Stable Does Not Unstable Invade 1 Invades Stable Unstable Does Not a Invade 8 0.9 Unstable 0.9 Does Not Invade 0.8 0.8 08 0.9 1 1.1 12 08 0.9 1 1.1 12 Colony efficiency. r„2 Colony efficiency. r112 Mg. 6. If queens are singly mated (n-1). then a plot of ri versus rin clearly shows all four possibilities for the behavior around the two pure equilibria. For (a). we set pi,2 —0.75 and p, — I. For (b), we set pc.2 —0.99 and pi — I. which explains the occurrence of rota _ I va," and pan _ i)!2,0 in Eq. and be stable. The possibilities are shown in Fig. 5. In the cases (2). where policing cannot invade but is stable, or where policing can Numerical simulations of the evolutionary dynamics with a invade but is unstable, Brouwer's fixed-point theorem guarantees dominant police allele are shown in Fig. 4. the existence of at least one mixed equilibrium. In the case where Generally, four scenarios regarding the two pure equilibria are policing can invade but is unstable, police and non-police workers possible: Policing may not be able to invade and be unstable, will coexist indefinitely. policing may not be able to invade but be stable, policing may be We will now discuss the implications of our results for parti- able to invade but be unstable, or policing may be able to invade cular numbers of matings. EFTA01071001 108 Okjarz er aL /journal of theoretical Biology 399 (2016)103-n6 a The police allele invades and is stable. b The non-police allele invades and is stable. p„.2=0.75. 1.0344. r1=1.0767 p.„2=0.75, p1=1, r„e1.0344, r,=1.0567 rue Frequency of policing allele, a to 0.8 co :9. 0.6 • 1 o 0.4 8. 0.2 1 2 3 4 82 84 86 88 Time (101) Time (101) The police and non-police alleles are bistable. d The police and non-police alleles coexist. pw=0.75, p1=1. r„2=1.0244. r1.1.0667 pm l3.75. p1=1, re 1.0444. r1=1.0667 1 1 1 0.8 0.8 2' 0.8 0.8 0.4 8 0.4 0.2 0.2 2 LL LL OB 10 15 2 4 8 10 Time(10) Time (104) Fig. 7. Numerical simulations of the evolutionary dynamics of worker policing that show the four behaviors in fig. 6(a). The policing allele is dominant. For each of the four panels. we use the initial conditions: la) Xm,0 and443 -10-1; (b)X„,2 — 1 —10"1 and X„0 (C)440 0.02 and X2A.I — 0.98 (lower cum), and XAA0 —0.01 and X,,,„, —0.99 (upper cum): (d)X.„,„ — 1-10-2 and X.,," — 10-2 (lower curve), and X„,,, —1-10 2 and Xmo —10 2 (upper curie) We set re —1 without loss of generality. The stability condition for a dominant police allele is n = 1 mating 1.003 6 —pia +3pt r1> (4) 6+2p1 1.002 Evolution of policing is highly sensitive to changes in colony effi- 1.001 ciency. For example, let us consider pin = 0.99 and pi =1. This means that if half of all workers police then 99% of all males come from the queen. If all workers police then all males come from the queen. In this 1 case, efficiency values such as r112 = 1.001 and rt = t0031 lead to the O evolution of policing. In principle, arbitrarily small increases in colony O 0.999 efficiency can lead to the evolution of policing for single mating. A plot of rt versus r112 for singly mated queens (Fig. 6) illus- 0.998 trates the rich behavior highlighted in Fig. 5. Numerical simula- O 1/2 1 tions of the evolutionary dynamics are shown in Fig. 7. Fraction of poice workers, Another intriguing feature is that increases in colony efficiency Fig a Possible rz efficiency curves for n-1 mating which demonstrate different due to policing do not necessarily result in a higher frequency of behaviors. For this plot. we set pi.,7 0.99 and p, — 1. Here, each curve has the police workers at equilibrium Fig. 8 illustrates this phenomenon. functional form r, — I + art/IS. For example. we can have: (blue) policing invades Four possibilities for the efficiency function rz are shown. Notice that but is unstable, a—0.003, p--0.O004: (green) policing invades and is stable, the r, curve which results in coexistence of police workers and non- a—0.0026,1-0: (red) policing does not invade and is unstable, a — 0.0024. p—0: (black) policing does not invade but is stable, a—0.002. 0-0.0004. (For inter- police workers (blue, top) is strictly greater than the r z curve which pretation of the references to color in this figure caption, the reader is referred to results in all workers policing (green, second from top). How can die web version of this paper.) increased efficiency due to policing possibly result in policing being less abundant at equilibrium? If a mutation for non-policing behavior 3. Single mating is introduced into a resident policing population, then the evolu- tionary success of the non-policing mutation depends on the success For single mating. n=1. the invasion condition for a dominant ofM. 0 colonies relative to aa. 1, aa.0, Aa, 1, and AA, 1 colonies. Aa. 0 police allele is colonies have an efficiency parameter r 112. while the other four relevant colonies each have an efficiency parameter rt. Thus, if r112 is 2(2 — rip) too large relative to rt, then the non-police allele can invade a resi- > (3) 2(1-1:01)+(P +Ptp) p dent policing population, and there is coexistence. Also notice that the r z curve which results in bistability of police (Recall that r0 = 1.) workers and non-police workers (black, bottom) is strictly less than EFTA01071002 J.W. OleJarz er al. /Journal of Theoretical Biology 399 (2016) 103-116 109 n = 2 matings n=3 matings 1.003 1.003 POIleng InvedeS but IS °Datable Pa cog yaks INA is unstable Poems' invades and is stable - POrcna ades and is stable 1.002 P011eng We, nOlInvaile BM unSbable 1.002 Porteng does not invade and is unstable PoIrceig does not invade but is stable PcI cog does not invade txt 4 Stable 1.001 ro 1.001 g. 1 1 O 0 8 0.999 0 0.999 0.998 0.998 0 1/2 3/4 1 0 1)3 112 (213) 516 1 Fraction of police workers, z Fraction of police workers, z Fig. 9. Possible r, efficiency curves for n-2 matings which demonstrate different Fig. 10. Possible rt efficiency curves for n-3 matings which demonstrate different behaviors. Here, each curve has the functional form r, — az+rs.z2. For example, behaviors. For this plot we set pl.., — 0.986, p,,a — 0.99, p, — 0.996, and pi —1. we can have: (blue) poking invades but is unstable. a-0.0005. p--0.0004: Here, each curve has the functional form r, -1+a2 pz2. For example. we can have: (green) policing invades and is stable. a-0.0001, p-0; (red) policing does not (blue) policing invades but is unstable. a— —0.0006. p- -0.0006: (green) policing invade and is unstable, a — 0.0001, 0-0; (black) policing does not invade but is invades and is stable, a--0.0012, p-0; (red) policing does not invade and is stable, a — -0.0005.p-0.0004. (For interpretation of the references to color in this unstable, a - 0.0015, 0-0; (black) policing does not invade but is stable, figure caption, the reader is referred to the web version of this paper.) a- -0.0021, /1-0.0006. Note that the value r2,., affects the population dynamics but does not appear in the conditions for invasion and stability of the police allele, hence the r, curve which results in policing being dominated by non-policing the parentheses on the horizontal axis. (For interpretation of the references to color in this figure caption, the reader is referred to the web version of this paper.) (red, second from bottom). This phenomenon arises in a similar way: if ri ;2 is too small relative tor,, then the non-police allele cannot invade a resident policing population, and there is bistability. Just as for single mating we observe the intriguing feature that increases in colony efficiency due to policing do not necessarily result in a higher frequency of police workers at equilibrium. Fig. 10 illus- trates this phenomenon. Four possibilities for the efficiency function 4. Double mating r, are shown. Notice that the r, curve which results in coexistence of police workers and non-police workers (blue, top) is strictly greater For double mating. n=2. the invasion condition for a dominant than the r, curve which results in all workers policing (green, second police allele is given by from top). Also notice that the r, curve which results in bistability of ria >1 (5) police workers and non-police workers (black, bottom) is strictly less than the r, curve which results in policing being dominated by non- Thus, policing can invade if there is an infinitesimal increase in policing (red, second from bottom). colony efficiency when half of all workers police. Policing cannot invade if there is an infinitesimal decrease in colony efficiency when half of all workers police. 6. Recessive police allele The stability condition for policing is given by rl > F3/4 (6) We have also derived the conditions for the emergence and evo- lutionary stability of worker policing if the police allele is fully reces- Therefore, the policing allele is stable if the colony efficiency is sive. In this case. M and Aa workers are phenotypically identical and greater for z=1 (when all workers police) than for z= 3/4 (when do not police, while aa workers do police. (Alternatively, M and Aa only three quarters of the workers police). workers police with intensity 44= Zilch while aa workers police with Four possible efficiency curves r, and the corresponding intensity Zoo =Zvi+ W = ZA0 -1-w. We consider this case in Section 8.) behavior of the police allele are shown in Fig. 9. 6.1. Emergence of worker policing 5. Triple mating The invasion condition for a recessive police allele, a, is given by For triple mating n=3. the invasion condition for a dominant Two > 2(2+n+npo) ro (9) police allele is given by (2 +n)(2 +Po) +Pin2A)(n — 2) 4- 20 - PO115 Note that Eq. (9) for invasion of a recessive police allele has the same rig> (7) mathematical form as Eq. (2) for evolutionary stability of a dominant 2+(Pia+Ptp), 113 police allele. Starting from Eq. (2), making the substitution z -.1-z. The stability condition for policing is given by and reversing the inequality, we recover Eq. (9). The intuition behind 10+Josio +Spi this correspondence is described in Appendix A. ri > F5/6 (8) 10+6pi 62. Stability of worker policing As a numerical example, let us consider pv3 =0.98 and p112 =0.99. If z= 1/3 of workers police, then 98% of males come A recessive police allele, a. is evolutionarily stable if from the queen. If z = 1/2 of workers police, then 99% of males [2( \ 11 (1 pi ri >cfa_iv.+pia come from the queen. In this case, policing cannot invade if ri/3 (10) Vrn-run/ I. Via./ ' kria) 2 = 0.9990 and ria =0.9979. In principle, arbitrarily small reduc- tions in colony efficiency can prevent evolution of policing for Note that Eq. (10) for evolutionary stability of a recessive police triple mating. allele has the same mathematical form as Eq. (1) for invasion of a EFTA01071003 110 J.W. OkJazz et al. &Journal of theoretical Biology 399 (2016) 103-116 Single mating, n=1 Double mating, n=2 b le - ru4=1.002 - ru4=1.001 To To - r =0.999 2 - r =0.998 rr 16 20 0 8 12 16 20 Time (105 F1/4 .11. Numerical simulations of the evolutionary dynamic of worker policing confirm the condition given by Eq. (9). The policing allele is recessive. For numerically probing invasion, we use the initial condition XAm, - 1 -10-2 and XAki - 10-e. We set I'D 1 without loss of generality. Other parameters are: (a)p0-0.6. en -0.8, and 6-1.06: (b) Po - 035. R14 —0.9. ri,2 - recra, and G -1.012. Under what conditions does worker policing invade? n = 2 footings as 3 matings n a 1 Mathlg is constant (equal to 1) NEUTRAL a decreases monotonically YES Increases monotonically NO YES NO YESa YES YES YES NO reaches a maximum for NO YES scaitt0<z< I NO YES YES YES NO NO NO YES NO NO NO YES NO reaches a minimum for NO NO YES some 0< z <1 NO YES YES ki tL YESA NO YES YES YES NO YES YES YES Fig. 12. Depending on the funttional form of colony efficiency. G. on the fraction of police workers. z. policing alleles may or may not invade for single, double, of triple mating. Various possibilities of r, are shown. The outcomes hold for both dominant and recessive police alleles. If G is constant, then policing does not invade for single mating, is neutral for double mating and invades for triple mating. If rz decreases monotonically, then policing does not invade of invades only for triple mating, If i, increases monotonically. then policing either invades only for double and triple mating or for single, double, and tnpk mating. If G reaches a maximum at an intermediate value 0 <z < 1, then policing does not invade of may invade for tnpk mating only. for double and triple mating or for single, double, and triple mating, If rz reaches a minimum at an intermediate value 0 <z < 1. then any pattern is possible. dominant police allele. Starting from Eq. (1). making the sub- We have the following results regarding the invasion and sta- stitution z-.1-z. and reversing the inequality, we recover Eq. , 10 '. bility of police workers. We discuss single (n=1). double (n=2). Again, the intuition behind this correspondence is described in and triple (n=3) mating. All results apply to both dominant and Appendix A. recessive police alleles. They can be instantiated with arbitrarily Numerical simulations of the evolutionary dynamics with a small changes in colony efficiency. recessive police allele are shown in Fig. 11. 7.1. Evotutionary invasion of policing 7. Shape of the efficiency function, r j, (i) If r2 is strictly constant (which is ungeneric), then policing does not invade for single mating, is neutral for double mating, and The shape of the efficiency function. rz. determines whether does invade for triple mating. policing is more likely to evolve for single mating or multiple (ii) If rz is monotonically decreasing, then policing either invades matings. Recall that r, is the colony efficiency (defined as the rate not at all or only for triple mating. of generation of reproductives) if a fraction, z. of all workers per- (iii) If rz is monotonically increasing, then policing either invades form policing. The variable z can assume values between 0 and 1.If for single, double, and triple mating or only for double and no workers police. z=0. then the colony efficiency is at baseline. triple mating. which we set to one: therefore, we have ro =1. Policing can in (iv) If rz reaches an intermediate maximum (which means colony principle increase or decrease colony efficiency (Fig. 12). efficiency is highest for an intermediate fraction of police EFTA01071004 Olefarz er al. /Journal of Theoretkaf Biology 399 (2016) 103-116 a Policing invades for n=1 but not for n=2 b Policing does not evade but is stable (for n=1 and n=2) 1.012 1.010 1.088 1.006 g. 0.999 1.004 1 0.998 _g 1.002 • 8 -• • 6 0.997 0.998 0 1/2 3/4 1 0.996 0 1/2 3/4 Fraction of police workers.: Fraction of police workers.: Policing invades but is unstable d Policing does not invade but is stable (fix n=1) (for n=1 and n=2) Policing invades but is unstable (for n=2) 1.003 • 1.002 • • C g 1.002 .2 € 1.001 0 1.001 1- • 0 1/2 3/4 0 1/2 314 Fraction of police workers.: Fraction of police workers,: Fig. 13. Non-monotonic efficiency funcnons can lead to rich and counterintuitive behavior. We consider invasion and stability of a dominant police allele for single (n-1) and double (n-2) mating. The baseline colony efficiency without policing is ro — 1.Three other values must be specified: r,„2, 44. and r1. Moreover, we need to specify two values for how the presence of police workers affects the fraction of male offspring coming from the queen: we choose oil —0.99 and p, — I. A variety of behaviors can be realized by a very small variation in colony efficiency. (a) Policing invades for single mating but not for double mating. (b) Policing does not invade but is stable for single and double mating. (c) Policing invades but is unstable for single and double mating. (d) Policing does not invade but is stable for single mating, while policing invades but is unstable for double mating. workers), then policing can invade for n = 1,2,3 or n=2.3 or discuss the invasion and stability of a dominant police allele for n=3 or not at all. single and double mating, we need to specify efficiency at three (v) If rz reaches an intermediate minimum (which means colony discrete values for the fraction of police workers present in a efficiency is lowest for an intermediate fraction of police colony: r ia. r314. and r 1. Note that ro =1 is the baseline. Moreover. workers), then policing can invade with any pattern of mat- we need to specify the fraction of male offspring coming from the ings. For example, it is possible that policing invades only for queen at two values: po and pi. For all examples in Fig. 13, we single mating but neither for double nor triple mating. Or it assume pla =0.99 and pi = 1. We show four cases: (a) policing invades for single and double mating but not for triple mating. invades for single mating but not for double mating: (b) for both single and double mating. policing does not invade but is stable: (c) for both single and double mating, policing invades but is 72. Evolutionary stability of policing unstable (leading to coexistence of policing and non-policing alleles): (d) policing does not invade but is stable for single mat- ing: policing invades but is unstable for double mating. These (i) If rz is constant, then policing is unstable for single mating, is cases demonstrate the rich behavior of the system, which goes neutral for double mating, and is stable for triple mating. beyond the simple view that multiple matings are always favor- (ii) If rz is monotonically decreasing, then policing is unstable for able for the evolution of policing. single and double mating. For triple mating it can be stable or unstable. (iii) If rz is monotonically increasing, then policing either is always 8. Gradual evolution of worker policing stable or is stable only for double and triple mating. (iv) If rz reaches an intermediate maximum, then policing can be Our main calculation applies to mutations of any effect size. In stable for any pattern of matings. For example, policing can be this section, we calculate the limit of incremental mutation (small stable for single mating but neither for double nor triple mutational effect size). Our calculations in this section are remi- mating. niscent of adaptive dynamics (Nowak and Sigmund, 1990; Hof- (v) If rz reaches an intermediate minimum, then policing can be bauer and Sigmund, 1990; Dieckmann and Law, 1996; Metz et al., stable for n = 1.2.3 or n=2,3 or n=3 or not at all. 1996; Geritz et al., 1998), which is usually formulated for asexual and haploid models. The analysis in this section applies both to the 7.3. Examples for single and double mating case of small phenotypic effect and to the case of weak penetrance. Mathematically, we consider the evolutionary dynamics of poli- Fig. 13 gives some interesting examples for how non- cing if the phenotypic mutations induced by the a allele are smalL If monotonic efficiency functions can influence the evolution of an allele affecting intensity of policing is dominant, then it is intui- policing for single (n=1) and double (n=2) mating. In order to tive to think of wild-type workers as policing with intensity 4.,, EFTA01071005 112 Okjorz et al /journal of theorerkol Biology 399 (2016) 10-116 while mutant workers police with intensityIm =Zoo =ZoAl-w. If an a small amount. For selection to favor increased policing, this ratio allele affecting intensity of policing is recessive, then it is intuitive to of marginals must exceed a quantity depending on the current think of wild-type workers as policing with intensity ZM =140, values of R and P. while mutant workers police with intensity 1,0 = ZAA = ZA0 + w. Notice that the sign of the right-hand side is determined by In the limit of incremental mutation, the fraction, p. of queen- n-2. So we get different behavior for different numbers of derived males and the colony efficiency. r, become functions of the matings: average intensity of policing in the colony, which is Z+wz, where z is the fraction of mutant workers in the colony. We have • For n=2 (double mating), policing increases from Z if and only if R'(Z)> 0. This means that evolution maximizes the value of it, -.p(Z+ wz)= P(z)+PV)wz+1 PV)w22+0(w3) regardless of the behavior of P. In other words, for double r,-.R(Zl-wz)= Rail+ K(Z)wz+1 R.(Z)w2z2 +O(w3) (11) mating, evolution maximizes colony efficiency regardless of the We have made the substitutions p,—.P(Zl-wz) and r1—.5t(Z+wz). effect on the number of queen-derived males. and (11) gives the Taylor expansions of these quantities in terms of • For n=1 (single mating), the right-hand side of Eq. (16) is their first and second derivatives at intensity 1 (For conciseness, we positive. So the condition for Z to increase is more stringent will often omit the argument Z from the functions P and R and their than in the n=2 case. Increases in policing may be disfavored derivatives.) Here. I wi 41. so that workers with the phenotype even if they increase colony efficiency. corresponding to the mutant allele only have an incremental effect • For n a 3 (triple mating or more than three matings), the right- on colony dynamics. Thus, the expansions (11) are accurate hand side of Eq. (16) is negative. So the condition for Z to approximations. We assume that P > 0. The sign of w can be posi- increase is less stringent than in the n=2 case. Any increase in tive or negative. If w is positive, then the mutant allele's effect is to policing that improves colony efficiency will be favored, and increase the intensity of policing. If w is negative, then the mutant even increases in policing that reduce colony efficiency may be allele's effect is to decrease the intensity of policing. Note that this favored. formalism could also be interpreted as describing the case of weak penetrance, in which only a small fraction of all workers that have Eqs. (13) and (15) also allow us to determine the location(s) of the mutant genotype express the mutant phenotype. evolutionarily singular strategies (Geritz et al.. 1998). Intuitively, a For considering the dynamics of a dominant police allele with singular strategy is a particular intensity of policing, denoted by weak phenotypic mutation, we introduce the quantity r. at which rare workers with slightly different policing behavior are, to first order in w, neither favored nor disfavored by natural Qom = Pun +Pia 01 Oa) [2 _ trial _ _Pim) ( 1. .1)1 (12) selection. The parameter measuring intensity of policing. Z. can 2 ro 1/41 re / ro I f ro take values between 0 (corresponding to no policing) and 1 (cor- If Corm > 0. then increased intensity of policing is selected, and if responding to full policing). There are several possibilities: There Com <0. then increased intensity of policing is not selected. This may not exist a singular strategy for intermediate intensity of is just a different way of writing (1). policing: in this case, there is either no policing (Z* =0) or full We substitute (11) into (12) and collect powers of w. To first policing (r = 1). If there exists a singular strategy for 0 <Z• <1. order in w we get then there are additional considerations: There may be convergent Wr-2)KRA-2(2+n+nP)R1 evolution toward intensity Z" or divergent evolution away from Cdom = +O(w2) (13) intensity Z. In a small neighborhood for which Za-a*, further 4nR analysis is needed to determine if the singular strategy corre- For considering the dynamics of a recessive police allele with sponding tor is an ESS. weak phenotypic mutation, we introduce the quantity To determine the location(s) of evolutionarily singular strate- Aram 2(2-i-nl-npo) gies. we set the quantity in square brackets that multiplies w in Crec — (14) (13) and (15) to zero, yielding ro (2+n)(2 +Po)+Pita,o(n —2) If Crec > a then increased intensity of policing is selected, and if Kr) (Z*) +(n-2) —0 (17) Crec < 0, then increased intensity of policing is not selected. This is per) 2(2+n A- nP(n) just a different way of writing (9). Eq. (17) gives the location(s) of singular strategies for both domi- We substitute (11) into (14) and collect powers of w To first nant and recessive mutations that affect policing. order in w. we get For a given singular strategy r. there is convergent evolution —2)PRA-2(2+n-EnP)K1 toward r if Cre, — +O0412) (15) 4nR(2+n+nP) el [K(Z) R(Z) 1 +(n 2 <0 Notice that (13) and (15) are, up to a multiplicative factor, the same P(Z) n+nP(Z)) 2(2+ dZ .1 to first order in w Using Eqs. (13) and (15), the condition for policing to increase There is divergent evolution away from r if the opposite from a given level Z is inequality holds. It is helpful to consider some examples. If the functions P(Z) 1O2.) > R(Z) and R(Z) are known for a given species, then the behavior of (16) P(Z) -1n-2) 2(2+ n+nP(Z)) worker policing with gradual evolution can be studied. It is pos- Policing decreases from a given level Z if the opposite inequality sible that policing is at maximal intensity. r =1 (Fig. 14(a)). is holds. We have explicitly written the Z dependencies in Eq. (16) to nonexistent, r =0 (Fig. 14(b)). is bistable around a critical value of emphasize that the quantities P.P.& and K are all functions of the intensity. 0 e r < 1 (Fig. 14(c)). or exists at an intermediate value intensity of policing, Z. of intensity, 0 <Z. < I (Fig. 14(d)). The left-hand side of Eq. (16) can be understood as a ratio of Note that a singular strategy may or may not be an evolutio- marginal effects. To be specific, the left-hand side gives the ratio of narily stable strategy (ESS). (For example, it is possible that there is the marginal change in efficiency over the marginal increase in the convergent evolution toward a particular singular strategy r proportion of queen-derived males, if policing were to increase by which is not an ESS. In this case, once .zzo. evolutionary EFTA01071006 J.W. OleJarz et al. /puma/ of Theoregicaf Biology 399 (2016) 103-116 113 a There is full policing. (I-1) b There is no policing. (1.0) 120 S 1.20 re 1.15 • 1.15 = 1.10 1.10 gw L05I g Los 1 1 O .7 0.8 0.8 0.6 0.6 2 0.4 0.4 c- 0.2 8 2 & 0.2 0 00 0.2 0.4 0.6 0.8 0.2 0.4 0.6 0.8 Intensity of policing. Z Intensity of policing, Z C There is histabdity. (1-113) d There is intermediate policing. (1.0.7986...) S 120 S 1.20- a 1.15 1.15 - 1.10 11.10- ig 1.05 g ILI 1 0.8 IF; 0.8 0.6 °'. 0.6 0.4 I 0.4 g 0.2 2 0.2 CIO 0.2 0.4 0.6 0.8 °O 02 OA 0.6 0.8 Intensity of policing, Z Intercity of policing. Z Fig. 14. Several simple examples of functions P(Z) and R(Z) are shown. For single mating, the corresponding dynamics of policing intensity with gradual evolution are also shown. We use the forms FM- 1-IF +102 and R(Z)-l+CIZI-(1/2)C2.22. For each of the four panels, we set: (a) fr -0.5, C, -0.2, C2 0, corresponding to r -1: (b) P - OZ. CI -0.1, C2 -0, corresponding to 2" - (c) P -0.8, Ci -0.12, C2 -0, corresponding to bistability around Zs - 1/3: (d)f - - 0.2. C2 - 0.18, corre- sponding to an intermediate level of policing around r ft 0.7986.... branching may occur: Geritz et al., 1998) To determine if (17) is an policing, the singular strategy (17) represents a local ESS if ESS, we must look at second-order terms in (12) and (14). (n2 —4)1:Die + 2(n2 +4n —4)Pfrft+8nW2 +2(n2 + n2P+4)R-R <0 For a dominant police allele, we return to (12) with the sub- stitutions (11). We focus on a singular strategy given by (17). For a (21) singular strategy, Cdom is zero to first order in w. To second order in We may alternatively write (21) by substituting for R' using (17): w. we get (2 +n +nP)2[(n2 — 4)P. R + 2 (n2 +rt2P+4)r] (n2 — 4)P.R2 + 2(n2 + 4n —4),1212 Cam = w2 — (n2 —4) (n2 A- n2P+ 4n — 4)112R <0 (22) 16n2R2 Similarly, for a recessive allele that affects intensity of policing, the 8nPR42 +2(n2 + n2P+4)R-R1 +Ow') (18) singular strategy (17) represents a local ESS if 16n2R2 (n —2)(2+n+ nP)P. R — (n — 2)20 /2+2(2 + n + nP)2R. <0 (23) We may alternatively write (18) by substituting for RP using (17): Here, P. P. R. R. and 12' are all functions of the intensity of Qom w2 [(2+n+nP)2((n2 —4)"R +2(n2 n2P 4)12. policing, Z The local ESS conditions (22) and (23) are quite opaque 16n2R(2+n+nP)2 and do not allow for simple analysis. Notice that, although the (n —4)(n2 +n2P+4n-4)PeR locations of evolutionarily singular strategies are the same for +000) (19) dominant and recessive mutations that influence policing, the 16n2R(2+n+nP)2 conditions for a singular strategy to be a local ESS are different For a recessive police allele, we return to (14) with the sub- stitutions (11). We focus on a singular strategy given by (17). For a singular strategy, Crec is zero to first order in w. To second order in 9. Policing and inclusive fitness theory w. we get It has been claimed that policing is a test case of inclusive fit- crec [(n — 2)(2 +n + nP)P R — (n — 2y2P2R ness theory (Abbot et al.. 2011). But the first two papers to theo- = 16n2R(2 +n + nP)2 retically establish the phenomenon (Woyciechowski and Lomnicki, 2(2 + n + nP)2R. 1987: Ratnieks. 1988) use standard population genetics: they do +O(W3) (20) not mention the term "inclusive fitness", and they do not calculate 16n2R(2+n nP)2 inclusive fitness. Therefore, the claims that theoretical investiga- Inspection of (18) and (20) allows us to determine if a singular tions of worker policing emerge from inclusive fitness theory or strategy is an ESS. If the bracketed quantity multiplying w2 is that empirical studies of policing test predictions of inclusive fit- negative, then mutations that change policing in either direction ness theory are incorrect are disfavored. If the bracketed quantity multiplying w2 is positive, In light of known and mathematically proven limitations of then mutations that change policing in either direction are inclusive fitness theory (Nowak et al., 2010: Allen et al., 2013). it is favored. Thus, for a dominant allele that affects intensity of unlikely that inclusive fitness theory can be used to study general EFTA01071007 114 J.W. Odejorz er aL /puma( of Theorerkol Biology 399 (2026) la3-116 questions of worker policing. Inclusive fitness theory assumes that releasing ourselves from the confines of a mathematically limited each individual contributes a separate, well-defined portion of theory, we expand the possibilities of scientific discovery. fitness to itself and to every other individual. It has been shown repeatedly (Cavalli-Sforza and Feldman, 1978; Uyenoyama and Feldman, 1982; Matessi and Karlin, 1984; Nowak et al., 2010; van 10. Discussion Veelen et al.. 2014). that this assumption does not hold for general evolutionary processes. Therefore, inclusive fitness is a limited We have derived analytical conditions for the invasion and concept that does not exist in most biological situations. stability of policing in situations where queens mate once or Our work shows that the evolution of worker policing depends several times and where colony efficiency can be affected by on the effectiveness of egg removal (p1) and the consequences of policing. In the special case where policing has no effect on colony colony efficiency (rz). Each of these effects can be nonlinear (not efficiency, our results confirm the traditional view that policing the sum of contributions from separate individuals), with impor- does not evolve for single mating, is neutral for double mating, and tant consequences for the fate of a policing allele. Moreover, the does evolve for triple mating or more than three matings. If colony invasion and stability conditions involve the product of p- and r- efficiency depends linearly or monotonically on the fraction of values, indicating a nontrivial interaction between these two workers that are policing, then our results support the view that effects which does not reduce to a simple sum of costs and ben- multiple mating is favorable to evolution of policing (Ratnieks. efits. We also found that there are separate conditions for invasion 1988). Our results also show that non-monotonic relations in and stability, with neither implying the other. Inclusive fitness colony dynamics and small changes in colony efficiency necessi- theory, which posits a single, linear condition for the success of a tate a more careful analysis. trait, is not equipped to deal with these considerations. We find that policing can evolve in species with singly mated Attempts to extend inclusive fitness theory to more general queens if it causes minute increases in colony efficiency. We find evolutionary processes (Queller. 1992; Frank, 1983; Gardner et al., that policing does not evolve in species with multiply mated 2011) rely on the incorrect interpretation of linear regression queens if it causes minute decreases in colony efficiency. For non- coefficients (Allen et al., 2013; see also Birch and Okasha, 2014). monotonic efficiency functions, it is possible that single mating This misuse of statistical inference tools is unique to inclusive allows evolution of policing, while multiple mating opposes evo- fitness theory, and differs from legitimate uses of linear regression lution of policing. in quantitative genetics and other areas of science. It was also Our analysis is the first to give precise conditions for both the recently discovered that even in situations where inclusive fitness invasion and stability of policing for both dominant and recessive does exist, it can give the wrong result as to the direction of nat- mutations that effect policing. We study the evolutionary invasion ural selection (Tarnita and Taylor, 2014). and evolutionary stability of policing both analytically and Relatedness-based arguments are often seen in conjunction numerically. For any number of matings, there are four possible with inclusive fitness, but there is a crucial difference. Consider the outcomes (see Fig. 5): (i) policing can invade and is stable; (ii) following statement: if the queen is singly mated, then workers policing can invade but is unstable, leading to coexistence; (iii) share more genetic material with sons of other workers than with policing cannot invade but is stable. leading to bistability; (iv) sons of the queen. This statement is not wrong and could be useful policing cannot invade and is unstable. We give precise conditions in formulating evolutionary hypotheses. Such hypotheses can then for all outcomes for both dominant and recessive police alleles. All be checked using exact mathematical methods. outcomes can be achieved with arbitrarily small changes in colony The problem arises when one attempts to formulate the efficiency. quantity of inclusive fitness by partitioning fitness into contribu- Our calculations are not based on any assumption about the tions from different individuals and reassigning these contribu- strength of phenotypic mutation induced by an allele. The condi- tions from recipient to actor. A worker does not make separate tions (1). (2). (9), and (10) also describe the dynamics of mutations contributions to fitnesses of others. and therefore does not have that have an arbitrarily small phenotypic effect on colony Inclusive fitness". Arguments such as "the worker maximizes her dynamics. This facilitates investigation of the evolution of complex inclusive fitness by not policing' are meaningless. since they are social behaviors that result from gradual accumulation of many based on maximizing a nonexistent quantity. Moreover, even mutations (Kapheim et al.. 2016). We derive a simple relation, Eq. when evolution leads individuals to maximize some quantity, that (17). for the location(s) of evolutionarily singular strategies. We quantity is not necessarily inclusive fitness (Okasha and Martens, also derive precise conditions for a singular strategy to be an ESS. 2015; Lehmann et al., 2015). These results are applicable for understanding both the case of It is true that genes (alleles) can be favored by natural selection weak phenotypic effect and the case of weak penetrance. if they enhance the reproduction of copies of themselves in other Our analysis does not use inclusive fitness theory. Given the individuals. But that argument works out on the level of genes and known limitations of inclusive fitness (Nowak et al., 2010; Allen et can be fully analyzed using population genetics. Inclusive fitness al.. 2013). it is unlikely that inclusive fitness theory could provide a only arises when the individual is chosen as the level of analysis. general framework for analyzing the evolution of worker policing. which is a problematic choice for many cases of complex family or In summary, the main conclusions of our paper are: (i) The population structure (Akcay and Van Cleve. 2016). prevalent relatedness-based argument that policing evolves under Bourke (2011) has proposed that inclusive fitness remains valid multiple mating but not under single mating is not robust with as a concept even when it is nonexistent as a quantity. But why is respect to arbitrarily small variations in colony efficiency; (ii) for such an uninstantiable concept useful? The mathematical theory of non-monotonic efficiency functions, it is possible that policing evolution is clear and powerful Exact calculations of evolutionary evolves for single mating, but not for double or triple mating; (iii) dynamics (Antal et al.. 2009; Allen and Nowak, 2014; Fu et al.. 2014; careful measurements of colony efficiency and the fraction of Hauert and Doebeli, 2004; Szabo and Fath, 2007; Antal and queen-derived males are needed to understand how natural Scheuring, 2006; Traulsen et al., 2008; van Veelen et al.. 2014; selection acts on policing; (iv) contrary to what has been claimed Simon et al.. 2013) demonstrate that inclusive fitness is not needed (Abbot et al.. 2011), the phenomenon of worker policing is no for understanding any phenomenon in evolutionary biology. This empirical confirmation of inclusive fitness theory; the first two realization is good news for all whose primary goal is to understand mathematical papers on worker policing (Woyciechowski and evolution rather than to insist on a particular method of analysis. By Lomnicki, 1987; Ratnieks. 1988) do not use inclusive fitness theory. EFTA01071008 J.W. Olejorz et al. /puma/ of Theoretkof Biology 399 (2016) 103-116 115 The present paper, which also does not use inclusive fitness the- Gadagkar. R. 2001. The Social Biology of Ropalidta Marginata: Toward Under- ory. is the first detailed analysis of policing for any number of standing the Evolution of Eusociality. Harvard University Press. Cambridge. MA Gadagkar. It. 2004. Why do honey bee workers destroy each other's eggs/ J. Biosci. matings and taking into account effects on colony efficiency. 29 (3). 213-217. Gardner. A_ West. SA. Wild. C.. 2011. The genetical theory of kin selection. J. (vol. Biol. 24.1020-1043. Gentz. SAH.. Kisdi. E.. Meszena. C.. Metz. JAJ.. 1998. Evolutionarily singular stra- Acknowledgments tegies and the adaptive growth and branching of the evolutionary tree. Evol. (col. 12.35-57. We are grateful to the referees and editor for helpful comments Hamilton. W.D. 1972. Altruism and related phenomena, mainly in social insects. Annu. Rev. EcoL Syst. 31.193-232. that have significantly benefited this manuscript. This publication Hammond. R.L. Keller. L. 2004. Conflict over male parentage in social insects. PLoS was made possible through the support of a grant from the John 8101.2 (e248). 1472-1482. Templeton Foundation. The opinions expressed in this publication Hartmann. A. Wantia. J.. Torres. JA_ Heinze. J.. 2003. Worker policing without genetic conflicts in a clonal ant. Proc. Natl. Acad. Sit. USA 100.12836-12840. are those of the authors and do not necessarily reflect the views of Hauen. C.. Doebeli. M_ 2004. Spatial structure often inhibits the evolution of the John Templeton Foundation. cooperation in the snowdrift game. Nature 428.643-646. Helantera. H.. Tofilsla. A.. Wenseleers. T. Ratnieks. ELW.. 2006. Worker policing in the common wasp Vespulo wagons is not aimed at improving colony hygiene. Insectes Soc. 53.399-401 Appendix A. Supplementary data Helantera. H.. Sandstrom. L 2007. Worker reproduction in formica ants. Am. Nat. 170. (14-E25. Hofbauer. J.. Sigmund. K.. 1990. Adaptive dynanucs and evolutionary stability. Appl. Supplementary data associated with this paper can be found in Math. Len. 3.75-79. the online version at http://dx.doi.org/10.1016njtbi2016.03.001. Hughes. W.O.H.. Oldroyd. B.R. Beekman. M.. Ratnieks. F.LW.. 2008. Ancestral monogamy shows kin selection is key to the evolution of eusociality. Science 320. 1213-1216. Kapheim. K.M.. Pan. H.. U. C.. Salzberg. S.L.. Pula. D.. et al.. 2015. Cenonuc signatures References of evolutionary transitions from solitary to group living. Science 348. 1139-1143. Khila. A.. Abouheif. E_ 2008. Reproductive constraint is a developmental mechan- Abbot. It. et al_ 2011. Inclusive fitness theory and eusociality. Nature 471. (1-(9. ism that maintains social harmony in advanced ant societies. Proc. Natl. Acad. Akcay. L. Van Cleve. J.. 2016. There is no fitness but fitness. and the lineage is its Sci. U.S.A. 105.17884-17889. bearer. Philos. Trans. R. Soc. B 371.20150085. Lehmann. L. Alger. I.. Weibull. J_ 2015. Does evolution lead to maximizing beha- Allen. B.. Nowak. MA.. Wilson. E.O.. 2013. Limitations of inclusive fitness. Proc. Natl. vior/ Evolution 69. 1858-1873. Acad. Sci. USA. 110.20135-20139. Matessi. C.. Karlin. S.. 1984.0n the evolution of altruism by kin selection. Proc. Natl. Allen. B.. Nowak. MA. 2014. Games on graphs. EMS Surv. Math. Sc. 1 (1). 115-151. Mad. Sa. USA 81.1754-1758. Antal. T.. Scheming. I.. 2006. Fixation of strategies for an evolutionary game in finite Metz. JAJ.. Gentz. SA.H.. Meszena. G.. Jacobs. EJA. van Heerwaarden. J.S.. 1996. populations. BuIL Math. Biol. 68.1923-1944. Adaptive dynamics. a geometncal study of the consequences of nearly faithful Antal. T.. Ohtsula. H. Wakeley. J.. Taylor. P.U.. Nowak. M.A. 2009. Evolution of reproduction. In: van Stnen. S.J.. Verduyn Lund SM. (Eds.). Stochastic and cooperation by phenotypic similanty. Proc. Natl. Mad. Sa. USA. 106 (21). Spatial Structures of Dynamical Systems. KNAW Verhandelingen. Afd.. 8597-8600. Amsterdam. pp. 183-231. Beekman. M.. Oldroyd. B.P.. 2005. Honeybee workers use cues other than egg via- Naeger. N.L. Peso. M.. Even. N.. Barron.A.B.. Robinson. G.E.. 2013. Altruistic behavior bility for policing. Biol. Lett. I. 129-132. by egg-laying worker honeybees. Curt. BioL 23.1574-1578. Birch. J.. °kasha. S.. 2014. Kin selection and its critics. BioScrence 65.22-32. Nonacs. P.. 2006. Nepotism and brood reliability in the suppression of worker Bonckaert. %V_ Vuerincicc K.. Billen. J.. Hammond. R.L. Keller. L. Wenseleers. T.. reproduction in the eusocial Hymenoptera. Biol. Lett. 2.577-579. 2008. Worker policing in the German wasp Wspulo gembonko. Behay. (col. 19. Nowak. MA.. Sigmund. K.. 1990. The evolution of stochastic strategies in the pris- 272-278. oners dilemma. Acta AppL Math. 20.247-265. Bonckaen. W.. van Zweden. J.S.. d'Ettorre. P.. Billen. J.. Wenseleers. T.. 2011. Colony Nowak. MA. Tarrora. C.E. Wilson. E.0.. 2010. The evolution of eusociality. Nature stage and not facultative policing explains pattern of worker reproduction in 466. 1057-1061 the Saxon wasp. Mol. EcoL 20.3455-3468. °kasha. S.. Martens. J_ 2016. Hamilton's rule. inclusive fitness maximization, and Boomsma. J.J. 2007. Kin selection versus sexual selection: why the ends do not the goal of individual behaviour in symmetric two-player games. J. (vol. Biol. meet. Curr. Biol. 17. R673-R683. 29.473-482. Boonisma. J.J.. 2009. Lifetime monogamy and the evolution of eusociality. ▪ J.W.. Allen. B.. yeller. C.. Nowak. M.A.. 2015. The evolution of non- Trans. IL Soc. B 164.3191-3207. Bourke. A.F.C.. 2011. The validity and value of inclusive fitness theory. Proc. FL Soc B reproductive workers in insect colonies with haplodiploid generics. elzfe 4. 278.3313-3320. e08918. Cavalli-Sforza. LL. Feldman. M.W.. 1978. Darwinian selection and 'altruism". Theor. Peters. J.M.. Queller. D.C. Imperarriz-Fonseca. V.L. Roubik. D.W.. Strassmann. J.E.. Popul. BioL 14.268-280. 1999. Mate number. kin selection and social conflicts in stingless bees and Cole. Bj.. 1986. The social behavior of Leptothorax olkodyce: (Hymenoptera. For- honeybees. Proc. It Soc. Lond. B 266.379-384. nuadae): time budgets and the evolution of worker reproduction. Behay. (col. Pirk. C.W.W.. Neumann. P.. Hepburn. R. N10111z, R.F.A.. Tautz. J_ 1999. Egg viability SociobioL 18.165-173. and worker policing in honey bees. Proc. Natl. Mad. Sci. U.S.A. 101.8649-8651. Cornwallis. CK.. West. S.A.. Davis. K.E.. Gnffin. AS.. 2010. Pronuscuity and the Queller. U.C.. 1992. A general model for kin selection. Evolution 46.376-380. evolutionary transition to complex societies. Nature 466.969-972. Queller. D.C.. Strassmann. J.E.. 1998. Kin selection and social insects. Bioscience 48. Dieckmann. U.. Law. R.. 1996. The dynamical theory of coevolution: a denvanon 165-175. from stochastic ecological processes. J. Math. Biol. 34.579-612. Barracks. F.L.W_ 1988. Reproductive harmony via mutual policing by workers in Endler. A.. Liebig. J.. Schnutt. T.. Parker. J.E. Jones. C.R.. Schreier. P. Holldobler. B.. eusocial hymenoptera. Am. Nat 132.217-236. 2004. Surface hydrocarbons of queen eggs regulate worker reproduction in a Barracks. F.L.W.. Visscher. HK.. 1989. Worker policing in the honeybee. Nature 342. social insect. Proc. Natl. Acad. Sci. USA 101.2945-2950. 796-797. Foster. K.R. Ratnieks. F.LW.. 2000. Facultative worker policing in a wasp. Nature Ratnieks. F.LW.. Foster. K.R. Wenseleers. T_ 2006. Conflict resolution in insect 407.692-693. societies. Annu. Rev. EntomoL 51.581-608. Foster. K.R. Ratnieks. F.LW.. 2001a. Convergent evolution of worker policing by egg Seeley. T.D.. 1985. Honeybee Ecology: A Study of Adaptation in Social Life. Princeton eating in the honeybee and common wasp. Proc. R. Soc. Lend. B 268.169-174. University MI5. Princeton. NJ. Foster. K.R. Ratnieks. F.LW.. 2001b. The effect of sex-allocation biasing on the Simon. B.. Fletcher. JA. Doebeli. M_ 2013. Towards a general theory of group evolution of worker policing in hymenopteran societies. Am. Nat. 158.615-624. selection. Evolution 67.1561-1572. Foster. K.R.. Ratnieks. F.LW.. 2001c. Paternity. reproduction and conflict in vespine Starr, C.K., 1984. In: Smith, RI. (Ed.). Sperm Competition, Kinship, and Sociality in wasps: a model system for testing kin selection predictions. Behay. (col. the Aculeate Hymenoptera. Academic Press, Orlando, Florida, pp. 427-464. SociobioL 50.1-8. Szabo. G.. Fath. C.. 2007. Evolutionary games on graphs. Phys. Rep. 446.97-216. Foster. KR.. Gulliver. J.. Ratnieks. F.LW.. 2002. Worker policing in the European Tarnita, C.E., Taylor, P.D. 2014. Measures of relative fitness d social behaviors in homer Vespo <wino. Insectes Soc. 49.41-44. finite structured population models. Am. Nat. 184(4) in press. Foster. K.R.. Wenseleers. T_ Ratnieks. F.LW.. 2006. Kin selection is the key to Traulsen. A.. Shoresh. N.. Nowak. MA.. 2008. Analytical results for individual and altruism. Trends (col. (vol. 21.57-60. group selection of any intensity. Bull. Math. BioL 70.1410-1424. Frank. SA_ 1983. Foundations of Social Evolution. Pnnceron University Press. Uyenoyama. M.K.. Feldman. M.. 1982. Population genetic theory of kin selection. II. Princeton. NJ. The multiplicative model. Am. Nat. 120.614-627. Fu. F.. Rocker. S.U.. Nowak M.A.. 2014. The nsk-retum trade-off between solitary van Veelen. M.. Luo. S.. Simon. B.. 2014. A simple model of group selection that and eusocial reproduction. Ecol. Lett. 18 (1). 74-84. cannot be analyzed with inclusive fitness. J. Theor. Biol. 160.279-289. EFTA01071009 116 J.W. Olejarz er al. /Journal of Theoreckal Bioko. 399 (2016)103-126 Velthuis. H.H.W.. de Araujo Alves. D.. Imperatriz-Fonseca. V.L. Jose. M.. 2002. Wenseleers. T.. Ratmeks. F.LW.. 2006b. Comparative analysts of worker reproduc- Worker bees and the fare of their eggs. Proc. Exp. Appl. Enromol. NEV tion and policing in eusocial hymenoptera supports relatedness theory. Am. Amsterdam 13.97-102. Nat 168. E163-E179. Wenseleers. T.. Helantera. H.. Hart. A.. Rarnieks. ELW.. 2004a. Worker reproduction Wenseleers. T.. Helantera. H.. Alves. DA. Duenez-Cuzman. E.. Panto. P.. 2013. and policing in insect societies: an ESS analysis. J. Evol. Biol. 17. 1035-1047. Towards greater realism in inclusive fitness models: the case of worker Wenseleers. T.. Hart. A.G.. Ratmeks. ELW.. 2004b. When resistance is useless: reproduction in insect societies. Biol. Lett. 9. 20130334. policing and the evolution of reproductive acquiescence in insect societies. Am. WdSOA E.0.. 1971. The Insect Societies. Harvard University Press. Cambridge. MA Nat. 164. E154-E167. Woyaechowski. M.. Lommcki. A.. 1987. Multiple manng of queens and the sterility Wenseleers. T.. Badcock. N.S. Erven. K.. Tofilski. A. Nascimenro. ES. Hart. A.C.. of workers among eusocial hymenoptera. J. Theor. Biol. 128. 317-327. Burke. T.A. Archer. M.E. Ratnieks. F.LW.. 2005. A test of worker policing theory Zanette. LRS.. Miller. S.U.L. Farrel. C.M.A.. Almond. Ej.. Huggins. T.J_ Jordan. W.C.. in an advanced eusocial wasp. Vespula tufa. Evolution 59. 1306-1314. Bourke. AEC.. 2012. Reproductive conflict in bumblebees and the evolution of Wenseleers. T.. Rainreks. F.LW.. 2006a. Enforced altruism in insect societies. Nature worker policing. Evolution 66. 3765-3777. 444. 50. EFTA01071010