-
Notifications
You must be signed in to change notification settings - Fork 264
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Moran process #534
Moran process #534
Changes from 3 commits
5269133
9a3494d
9de867e
fc831f3
1146f3a
5b28eec
b487bde
9a950d8
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,112 @@ | ||
# -*- coding: utf-8 -*- | ||
from collections import Counter | ||
import random | ||
|
||
import numpy as np | ||
|
||
from .deterministic_cache import DeterministicCache | ||
from .match import Match | ||
from .player import Player | ||
from .random_ import randrange | ||
|
||
|
||
def fitness_proportionate_selection(scores): | ||
"""Randomly selects an individual proportionally to score.""" | ||
csums = np.cumsum(scores) | ||
total = csums[-1] | ||
r = random.random() * total | ||
|
||
for i, x in enumerate(csums): | ||
if x >= r: | ||
return i | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this can be done simpler with http://docs.scipy.org/doc/numpy-1.10.0/reference/generated/numpy.random.choice.html There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm.. I'm not sure which is more simple.
Edit: I'm not sure this argument makes sense in this context, people usually prevent nonnegative values by exponentiating or some other such transformation. As a purely mathematical function it's fine to have negative values but not for this application. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I think it is. Leave it. |
||
|
||
class MoranProcess(object): | ||
def __init__(self, players, turns=100, noise=0): | ||
self.turns = turns | ||
self.noise = noise | ||
self.players = list(players) # initial population | ||
self.winner = None | ||
self.populations = [] | ||
self.populations.append(self.population_distribution()) | ||
self.score_history = [] | ||
|
||
@property | ||
def _stochastic(self): | ||
""" | ||
A boolean to show whether a match between two players would be | ||
stochastic | ||
""" | ||
if self.noise: | ||
return True | ||
else: | ||
return any(p.classifier['stochastic'] for p in self.players) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
def __next__(self): | ||
"""Iterate the population: | ||
- play the round's matches | ||
- chooses a player proportionally to fitness (total score) to reproduce | ||
- choose a player at random to be replaced | ||
- update the population | ||
""" | ||
# Check the exit condition, that all players are of the same type. | ||
population = self.populations[-1] | ||
classes = set(p.__class__ for p in self.players) | ||
if len(classes) == 1: | ||
self.winner = str(self.players[0]) | ||
raise StopIteration | ||
scores = self._play_next_round() | ||
# Update the population | ||
# Fitness proportionate selection | ||
j = fitness_proportionate_selection(scores) | ||
# Randomly remove a strategy | ||
i = randrange(0, len(self.players)) | ||
# Replace player i with clone of player j | ||
self.players[i] = self.players[j].clone() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is this going to be ok? Can we have a test with strategies we've had issues with before? In the same way we have a test for tournaments with strategies There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought about that -- I was hoping you could work some hypothesis magic :). I think we should be ok as far as strategy pairings go since I'm using the match class. So if they pass their clone tests and match pairings they should be fine here. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Sure thing: I'll send you a PR (probably in time for your tomorrow morning). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. |
||
self.populations.append(self.population_distribution()) | ||
|
||
def _play_next_round(self): | ||
"""Plays the next round of the process. Every player is paired up | ||
against every other player and the total scores are recorded.""" | ||
N = len(self.players) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. In this version the population size is constant but there are many variations in which that's not the case. I can go ahead and change it if you'd like.... I'm guessing we'd have another class for variants. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No no, if it can vary than I'm with you! Keep it like this :) 👍 There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The standard is not to vary, so I changed it. |
||
scores = [0] * N | ||
for i in range(N): | ||
for j in range(i + 1, N): | ||
player1 = self.players[i] | ||
player2 = self.players[j] | ||
player1.reset() | ||
player2.reset() | ||
match = Match((player1, player2), self.turns, noise=self.noise) | ||
match.play() | ||
match_scores = np.sum(match.scores(), axis=0) / float(self.turns) | ||
scores[i] += match_scores[0] | ||
scores[j] += match_scores[1] | ||
self.score_history.append(scores) | ||
return scores | ||
|
||
def population_distribution(self): | ||
"""Returns the population distribution of the last iteration.""" | ||
player_names = [str(player) for player in self.players] | ||
counter = Counter(player_names) | ||
return counter | ||
|
||
next = __next__ # Python 2 | ||
|
||
def __iter__(self): | ||
return self | ||
|
||
def reset(self): | ||
"""Reset the process to replay.""" | ||
self.winner = None | ||
self.populations = [self.populations[0]] | ||
self.score_history = [] | ||
|
||
def play(self): | ||
"""Play the process out to completion.""" | ||
while True: # O_o | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why this inline comment? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's not necessary |
||
try: | ||
self.__next__() | ||
except StopIteration: | ||
break | ||
|
||
def __len__(self): | ||
return len(self.populations) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -12,3 +12,9 @@ def random_choice(p=0.5): | |
if r < p: | ||
return Actions.C | ||
return Actions.D | ||
|
||
def randrange(a, b): | ||
"""Python 2 / 3 compatible randrange.""" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you write a bit more saying what this takes and returns (numpy syntax? If we're going to stick to that.). There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I could, but it's the same as There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yup, I know why it's here. Just asking for a 1 line saying: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done. |
||
c = b - a | ||
r = c * random.random() | ||
return a + int(r) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# -*- coding: utf-8 -*- | ||
import random | ||
import unittest | ||
|
||
import axelrod | ||
from axelrod import MoranProcess | ||
from axelrod.moran import fitness_proportionate_selection | ||
|
||
|
||
class TestMoranProcess(unittest.TestCase): | ||
|
||
def test_fps(self): | ||
self.assertEqual(fitness_proportionate_selection([0, 0, 1]), 2) | ||
random.seed(1) | ||
self.assertEqual(fitness_proportionate_selection([1, 1, 1]), 0) | ||
self.assertEqual(fitness_proportionate_selection([1, 1, 1]), 2) | ||
|
||
def test_stochastic(self): | ||
p1, p2 = axelrod.Cooperator(), axelrod.Cooperator() | ||
mp = MoranProcess((p1, p2)) | ||
self.assertFalse(mp._stochastic) | ||
p1, p2 = axelrod.Cooperator(), axelrod.Cooperator() | ||
mp = MoranProcess((p1, p2), noise=0.05) | ||
self.assertTrue(mp._stochastic) | ||
p1, p2 = axelrod.Cooperator(), axelrod.Random() | ||
mp = MoranProcess((p1, p2)) | ||
self.assertTrue(mp._stochastic) | ||
|
||
def test_exit_condition(self): | ||
p1, p2 = axelrod.Cooperator(), axelrod.Cooperator() | ||
mp = MoranProcess((p1, p2)) | ||
mp.play() | ||
self.assertEqual(len(mp), 1) | ||
|
||
def test_two_players(self): | ||
p1, p2 = axelrod.Cooperator(), axelrod.Defector() | ||
random.seed(5) | ||
mp = MoranProcess((p1, p2)) | ||
mp.play() | ||
self.assertEqual(len(mp), 5) | ||
self.assertEqual(mp.winner, str(p2)) | ||
|
||
def test_three_players(self): | ||
players = [axelrod.Cooperator(), axelrod.Cooperator(), | ||
axelrod.Defector()] | ||
random.seed(5) | ||
mp = MoranProcess(players) | ||
mp.play() | ||
self.assertEqual(len(mp), 7) | ||
self.assertEqual(mp.winner, str(axelrod.Defector())) | ||
|
||
def test_four_players(self): | ||
players = [axelrod.Cooperator() for _ in range(3)] | ||
players.append(axelrod.Defector()) | ||
random.seed(10) | ||
mp = MoranProcess(players) | ||
mp.play() | ||
self.assertEqual(len(mp), 9) | ||
self.assertEqual(mp.winner, str(axelrod.Defector())) | ||
|
||
def test_reset(self): | ||
p1, p2 = axelrod.Cooperator(), axelrod.Defector() | ||
random.seed(8) | ||
mp = MoranProcess((p1, p2)) | ||
mp.play() | ||
self.assertEqual(len(mp), 4) | ||
self.assertEqual(len(mp.score_history), 3) | ||
mp.reset() | ||
self.assertEqual(len(mp), 1) | ||
self.assertEqual(mp.winner, None) | ||
self.assertEqual(mp.score_history, []) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,3 +16,4 @@ Contents: | |
noisy_tournaments.rst | ||
ecological_variant.rst | ||
command_line.rst | ||
moran.rst | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
Moran Process | ||
============= | ||
|
||
The strategies in the library can be pitted against one another in the | ||
[Moran process](https://en.wikipedia.org/wiki/Moran_process), a population | ||
process simulating natural selection. The process works as follows. Given an | ||
initial population of players, the population is iterated in rounds consisting | ||
of: | ||
- matched played between each pair of players, with the cumulative total | ||
scores recored | ||
- a player is chosen to reproduce proportional to the player's score in the | ||
round | ||
- a player is chosen at random to be replaced | ||
|
||
The process proceeds in rounds until the population consists of a single player | ||
type. That type is declared the winner. To run an instance of the process with | ||
the library, proceed as follows:: | ||
|
||
>>> import axelrod as axl | ||
>>> players = [axl.Cooperator(), axl.Defector(), | ||
... axl.TitForTat(), axl.Grudger()] | ||
>>> mp = axl.MoranProcess(players) | ||
>>> mp.play() | ||
>>> mp.winner # doctest: +SKIP | ||
|
||
Defector | ||
|
||
You can access some attributes of the process, such as the number of rounds:: | ||
>>> len(mp) # doctest: +SKIP | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You don't. Have built these locally (see my PR on your branch). |
||
6 | ||
|
||
The sequence of populations:: | ||
>>> import pprint | ||
>>> pprint.pprint(mp.populations) # doctest: +SKIP | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Given the base print below, do we want the same here for consistency? (minor) |
||
[Counter({'Defector': 1, 'Cooperator': 1, 'Grudger': 1, 'Tit For Tat': 1}), | ||
Counter({'Defector': 1, 'Cooperator': 1, 'Grudger': 1, 'Tit For Tat': 1}), | ||
Counter({'Defector': 2, 'Cooperator': 1, 'Grudger': 1}), | ||
Counter({'Defector': 3, 'Grudger': 1}), | ||
Counter({'Defector': 3, 'Grudger': 1}), | ||
Counter({'Defector': 4})] | ||
|
||
The scores in each round:: | ||
>>> import pprint | ||
>>> pprint.pprint(mp.score_history) # doctest: +SKIP | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On
Perhaps do that for consistency? |
||
[[6.0, 7.0800000000000001, 6.9900000000000002, 6.9900000000000002], | ||
[6.0, 7.0800000000000001, 6.9900000000000002, 6.9900000000000002], | ||
[3.0, 7.04, 7.04, 4.9800000000000004], | ||
[3.04, 3.04, 3.04, 2.9699999999999998], | ||
[3.04, 3.04, 3.04, 2.9699999999999998]] | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you add a sentence or two explaining the connection/difference to the ecological variant (what insights are gained from one over the other?)? Is the ecological variant 'kind off' a moran process in expectation for a given cost function? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. They are pretty different. As far as I know the ecological variant used by Axelrod gets little attention while the Moran process is in thousands of papers. One is not a special case of the other AFAIK but I'll admit to not having studied the Axelrod version much. The Moran process limits to the replicator equation as the population size heads toward infinity. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Or something like that? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That check box confused the hell out of me: like "I don't remember writing that..." but yeah I've seen the addition. Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add a bit more to the docstring here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, but I'm not sure what to add really beyond the name of the function. Wikipedia has an extensive explanation.
It's sort of a weighted
randrange
-- if you put in a constant vector then it would berandrange(0, len(scores))
. It's possible that there is a more efficient implementation.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I realise this is trivial, just trying to future proof things as much as possible.