add implementation of a generic memory two + 2 strategies #1171

Nikoleta-v3 · 2018-04-05T13:11:30Z

A generic class for memory two players, similar to the MemoryOnePlayer class.
If transition rates are not given the the player is set to default which is Cooperator.

I have added the equivalent tests and a reference from issue #1063.

Moreover, this implements two strategies of interest from #1063:

AON2 and Delayed AON1

Each strategy has 16 states. I have added tests for most of the states.

Close #1063 A generic class for memory two players, similar to the MemoryOnePlayer class. If transition rates are not given the the player is set to default which is Cooperator. I have added the equivilent tests and a reference from issue #1063. Moreover, this implements two strategies of interest from #1063: - AON2 and Delayed AON1 Each strategy has 16 states. I have added tests for most of the states.

Nikoleta-v3 · 2018-04-05T13:11:40Z

Note that in the paper Memory-n strategies of direct reciprocity there are:

four possible transition vectors which are equivalent to AON_2. I chosed the one with the highest cooperation rate against itself. I did generate the fingerprints of all four vectors.

AON_2

second vector

third vector

fourth vector

three possible transition vectors which are equivalent to Delayed AON_1. I chosed the one with the highest cooperation rate against itself again, and the fingerprints:

Delayed AON_1

second vector

third vector

The fingerprints appear to be similar.

Nikoleta-v3 · 2018-04-05T13:14:20Z

Finally the tests are going to fail because of:

axelrod/strategies/memorytwo.py:98: error: Invalid index type "Tuple[Tuple[Any, ...], Tuple[Any, ...]]" for "Dict[Tuple[Tuple[Any], Tuple[]], float]"; expected type "Tuple[Tuple[Any], Tuple[]]"

which is because of the type hints. I am not really sure why it's breaking 😅 I would appreciate some help.

drvinceknight

Some minor requests from me @Nikoleta-v3, will take a look at the type hints now.

drvinceknight · 2018-04-05T13:27:04Z

axelrod/strategies/_strategies.py

@@ -57,7 +57,7 @@
    MemoryOnePlayer, ALLCorALLD, FirmButFair, GTFT, SoftJoss,
    StochasticCooperator, StochasticWSLS, WinStayLoseShift, WinShiftLoseStay,
    ReactivePlayer)
-from .memorytwo import MEM2
+from .memorytwo import MemoryTwoPlayer, AON2, DelayedAON1, MEM2


AON2 and DelayedAON1 need to be added to the all_players list.

drvinceknight · 2018-04-05T13:30:14Z

axelrod/strategies/memorytwo.py

+        'manipulates_state': False
+    }
+
+    def __init__(self, initial: Action = C) -> None:


As these strategies are defined in the literature, do we need initial to be an option?

I agree, but there are many strategies in the library (from the literature) already which have initial as an option.

Axelrod/axelrod/strategies/memoryone.py

Lines 98 to 123 in 5e09643

class WinStayLoseShift(MemoryOnePlayer):

"""

Win-Stay Lose-Shift, also called Pavlov.

Names:

- Win Stay Lose Shift: [Nowak1993]_

- WSLS: [Stewart2012]_

- Pavlov: [Kraines1989]_

"""

name = 'Win-Stay Lose-Shift'

classifier = {

'memory_depth': 1, # Memory-one Four-Vector

'stochastic': False,

'makes_use_of': set(),

'long_run_time': False,

'inspects_source': False,

'manipulates_source': False,

'manipulates_state': False

}

def __init__(self, initial: Action = C) -> None:

four_vector = (1, 0, 0, 1)

super().__init__(four_vector)

self._initial = initial

Sure but in this case, they're very specific strategies and we go down the line of needing two initial moves to be truly generic and then probably overwriting the repr method to not display them as they're defined in the literature?

I suggest just removing the option for these two strategies. (If a user really wants to they can always inherit and overwrite.)

drvinceknight · 2018-04-05T13:37:27Z

axelrod/strategies/memorytwo.py

+    In essence is a strategy that starts of by cooperating and will cooperate
+    again only after the states (CC, CC), (CD, CD), (CD, DD), (DD, CD),
+    (DC, DC) and (DD, DD).
+


Could we specifically write the vector here and also write the other vectors reported as equivalent (but different) in [Hilbe2017] (mentioning the reported self cooperation rate?).

Same for AON2.

Add information about the vectors which are reported as equivilent. I am not sure I can say that they are actually different in the documentation without saying why and without adding the fingerprints.

drvinceknight · 2018-04-05T14:17:11Z

I would appreciate some help.

@Nikoleta-v3 I've opened https://github.com/Nikoleta-v3/Axelrod/pull/8 that fixes the type hint issue (for me).

I've essentially been less specific in some places with the hints and been explicit in the Dict, that inline comment is actually a variable type hint (see http://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html) which is there specifically annotating we've got a dictionary mapping tuples to floats.

marcharper · 2018-04-05T14:18:39Z

axelrod/strategies/memorytwo.py

+
+        states = [(hist[:2], hist[2:]) for hist in list(itertools.product((C, D), repeat=4))]
+
+        self._sixteen_vector = dict(zip(states, map(float, sixteen_vector)))


I think the automatically derived type here is not correct. Maybe if you wrap map(...) with tuple(...) it will work since it won't be a list of tuples until actually used in a list/tuple context, which the type checker may not be able to determine.

Alternatively you could explicitly set the type (perhaps better to do in the __init__ function probably).

…to implement-mem-two

drvinceknight · 2018-04-05T14:26:42Z

axelrod/strategies/memorytwo.py

+    rounds in which the focal player cooperated while at least one coplayer defected,
+    the strategy responds by defecting the following k rounds.
+
+    In [Hilbe2017]_ the following vectors are reported as equivalent to AON2


(Very minor request:)

reported as "equivalent" ... cooperation rate (note that these a not the same)

(Same for the other one).

drvinceknight · 2018-04-05T14:35:55Z

axelrod/tests/strategies/test_memorytwo.py

+        self.assertFalse(self.p1.classifier['stochastic'])
+        self.assertFalse(self.p2.classifier['stochastic'])
+
+class TestMemoryTwoPlayer(unittest.TestCase):


@Nikoleta-v3 could we add some tests of the player with stochastic sixteen_vector (so using TestPlayer.versus_test with init_kwargs). Seeded tests, including same history/different seed to show that we indeed have stochastic behaviour?

drvinceknight · 2018-04-05T14:37:22Z

axelrod/tests/strategies/test_memorytwo.py

+        self.assertEqual(self.p2.name,
+                         "Generic Memory Two Player: (1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0)")
+
+    def test_stochastic_classification(self):


Can we rename this to test_deterministic_classification and then also include a test_stochastic_classification where we use sixteen_vector where the values aren't all in [0, 1].

drvinceknight · 2018-04-05T14:42:56Z

axelrod/strategies/memorytwo.py

+        states = [(hist[:2], hist[2:])
+                  for hist in list(itertools.product((C, D), repeat=4))]
+
+        self._sixteen_vector = dict(zip(states, sixteen_vector)) # type: Dict[tuple, float]


@marcharper this no longer maps float on to states, it's a change I suggested to @Nikoleta-v3 when trying to get the type hints to work https://github.com/Nikoleta-v3/Axelrod/pull/8

I couldn't see why we needed to map(float but we do do it for memory_one so just highlighting in case this is something I've forgotten/missed (if indeed it's not needed, let's get rid of it from memory_one too).

Added tests against two opponents. For different seed the behaviour of the memory two strategy changes.

marcharper · 2018-04-06T00:57:21Z

axelrod/strategies/memorytwo.py

+    4. [1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1], self cooperation
+    rate: 0.952
+
+    AON2 is implemented using vector 1 due it's self cooperation rate.


nit: it's --> its

marcharper · 2018-04-06T00:58:20Z

axelrod/strategies/memorytwo.py

@@ -148,6 +162,18 @@ class DelayedAON1(MemoryTwoPlayer):
    Delayed AON1 a memory two strategy also introduced in [Hilbe2017]_ and belongs to the
    AONk family. Note that AON1 is equivalent to Win Stay Lose Swift.


Swift -> Shift

marcharper · 2018-04-06T01:01:16Z

axelrod/strategies/memorytwo.py

+class MemoryTwoPlayer(Player):
+    """
+    Uses a sixteen-vector for strategies based on the last two rounds of play,
+    1.  P(C|CC, CC)


Maybe better as (or something similar to):
Based on the 16 conditional probabilities P(X | I,J,K,L) where X, I, J, K, L in [C, D] and I, J are the players last two moves and K, L are the opponents last two moves.

I think I'd prefer them explicitly listed, the ordered isn't clear otherwise?

Unless your suggestion is to use your sentence to clarify what the list is and also list them which is a great call 👍

marcharper · 2018-04-06T01:04:50Z

axelrod/strategies/memorytwo.py

+        'manipulates_state': False
+    }
+
+    def __init__(self, sixteen_vector: Tuple[float, float, float, float,


Could use Tuple[float, ...] for brevity but it wouldn't enforce length 16

marcharper · 2018-04-06T01:05:47Z

axelrod/strategies/memorytwo.py

+
+    def set_initial_sixteen_vector(self, sixteen_vector):
+        if sixteen_vector is None:
+            sixteen_vector = (1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)


tuple([1] * 16)

marcharper · 2018-04-06T01:08:24Z

axelrod/strategies/memorytwo.py

+    to cooperate after rounds with mutual cooperation (provided the last k actions
+    of the focal player were actually consistent).
+
+    2.Error correcting. A strategy is error correcting after at most k rounds if,


missing space in 2.Error

marcharper · 2018-04-06T01:08:39Z

axelrod/strategies/memorytwo.py

+
+    2.Error correcting. A strategy is error correcting after at most k rounds if,
+    after any history, it generally takes a group of players at most k + 1 rounds
+    to re establish mutual cooperation.


re-establish

marcharper · 2018-04-06T01:10:23Z

axelrod/strategies/memorytwo.py

+    to re establish mutual cooperation.
+
+    3. Retaliating. A strategy is retaliating for at least k rounds if, after
+    rounds in which the focal player cooperated while at least one coplayer defected,


coplayer = opponent ? (if so, I like it)

I think this is supposed to say that the opponent defected while the player was cooperating, as written it makes it seem like there are multiple coplayers.

marcharper · 2018-04-06T01:10:55Z

axelrod/strategies/memorytwo.py

+    the strategy responds by defecting the following k rounds.
+
+    In [Hilbe2017]_ the following vectors are reported as "equivalent" to AON2
+    with their respective self cooperation rate (note that these a not the same):


a -> are
self cooperation -> self-cooperation (and elsewhere)

marcharper · 2018-04-06T01:11:21Z

axelrod/strategies/memorytwo.py

+
+    AON2 is implemented using vector 1 due it's self cooperation rate.
+
+    In essence is a strategy that starts of by cooperating and will cooperate again


marcharper · 2018-04-06T01:11:46Z

axelrod/strategies/memorytwo.py

+class DelayedAON1(MemoryTwoPlayer):
+    """
+    Delayed AON1 a memory two strategy also introduced in [Hilbe2017]_ and belongs to the
+    AONk family. Note that AON1 is equivalent to Win Stay Lose Swift.


Swift -> Shift

marcharper · 2018-04-06T01:12:16Z

axelrod/strategies/memorytwo.py

+
+    name = 'AON2'
+    classifier = {
+        'memory_depth': 2,  # Memory-one Four-Vector


16 vector. Not sure we need a comment here since it's a memory two strategy by definition, or maybe that's a more useful comment

I agree, I don't think the comment is useful here. 👍

marcharper · 2018-04-06T01:13:40Z

axelrod/strategies/memorytwo.py

+
+    name = 'Delayed AON1'
+    classifier = {
+        'memory_depth': 2,  # Memory-one Four-Vector


marcharper · 2018-04-06T01:14:46Z

axelrod/tests/strategies/test_memorytwo.py

+            self.assertEqual(len(warning), 1)
+            self.assertEqual(warning[-1].category, UserWarning)
+            self.assertEqual(str(warning[-1].message),
+                              "Memory two player is set to default, Cooperator.")


remove extra space

marcharper · 2018-04-06T01:15:58Z

axelrod/tests/strategies/test_memorytwo.py

+        'manipulates_state': False
+    }
+
+    def test_class_classification(self):


Does this test have to be explicit? Or is it covered by the generic player tests?

marcharper

Looks very nice overall, lots of minor comments, approving since there are no substantial issues

marcharper · 2018-04-06T01:18:29Z

Nice work so far :) Thanks for adding these strategies!

Nikoleta-v3 · 2018-04-06T10:09:45Z

Thank you for the feedback @marcharper and thank you both for the review. I believe I addressed everything.

drvinceknight · 2018-04-06T10:15:13Z

Thanks @Nikoleta-v3, you've addressed all my requests. Will leave it to @marcharper for final ok 👍

Nikoleta-v3 mentioned this pull request Apr 5, 2018

Agenda 06-03-2018 Nikoleta-v3/Thesis#60

Closed

12 tasks

drvinceknight requested changes Apr 5, 2018

View reviewed changes

Nikoleta-v3 added 2 commits April 5, 2018 14:46

add AON2 and Delayed AON1 to all strategies

56ee6c9

add to the documentation of the strategies

874c341

Add information about the vectors which are reported as equivilent. I am not sure I can say that they are actually different in the documentation without saying why and without adding the fingerprints.

marcharper reviewed Apr 5, 2018

View reviewed changes

drvinceknight and others added 3 commits April 5, 2018 15:21

Fix type hint. (#8)

cd78ad8

Merge branch 'implement-mem-two' of github.com:Nikoleta-v3/Axelrod in…

858f533

…to implement-mem-two

remove initial action option

46b1e1d

drvinceknight requested changes Apr 5, 2018

View reviewed changes

minor tweak to documentation

de5d8d2

drvinceknight requested changes Apr 5, 2018

View reviewed changes

drvinceknight reviewed Apr 5, 2018

View reviewed changes

Nikoleta-v3 added 3 commits April 5, 2018 15:59

add to tests initialisation of stochastic mem two

b60e194

fix doc test

d98965e

test stochastic behaviour of memory two player

f3a8b84

Added tests against two opponents. For different seed the behaviour of the memory two strategy changes.

marcharper reviewed Apr 6, 2018

View reviewed changes

axelrod/strategies/memorytwo.py Outdated

name = 'Delayed AON1'

classifier = {

'memory_depth': 2, # Memory-one Four-Vector

Copy link

Member

marcharper Apr 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

marcharper reviewed Apr 6, 2018

View reviewed changes

marcharper approved these changes Apr 6, 2018

View reviewed changes

address comments on pr by Marc

0590028

drvinceknight approved these changes Apr 6, 2018

View reviewed changes

marcharper merged commit cf65f33 into Axelrod-Python:master Apr 6, 2018

	class WinStayLoseShift(MemoryOnePlayer):
	"""
	Win-Stay Lose-Shift, also called Pavlov.

	Names:

	- Win Stay Lose Shift: [Nowak1993]_
	- WSLS: [Stewart2012]_
	- Pavlov: [Kraines1989]_
	"""

	name = 'Win-Stay Lose-Shift'
	classifier = {
	'memory_depth': 1, # Memory-one Four-Vector
	'stochastic': False,
	'makes_use_of': set(),
	'long_run_time': False,
	'inspects_source': False,
	'manipulates_source': False,
	'manipulates_state': False
	}

	def __init__(self, initial: Action = C) -> None:
	four_vector = (1, 0, 0, 1)
	super().__init__(four_vector)
	self._initial = initial


		states = [(hist[:2], hist[2:]) for hist in list(itertools.product((C, D), repeat=4))]

		self._sixteen_vector = dict(zip(states, map(float, sixteen_vector)))

		@@ -148,6 +162,18 @@ class DelayedAON1(MemoryTwoPlayer):
		Delayed AON1 a memory two strategy also introduced in [Hilbe2017]_ and belongs to the
		AONk family. Note that AON1 is equivalent to Win Stay Lose Swift.


		AON2 is implemented using vector 1 due it's self cooperation rate.

		In essence is a strategy that starts of by cooperating and will cooperate again

add implementation of a generic memory two + 2 strategies #1171

add implementation of a generic memory two + 2 strategies #1171

Uh oh!

Conversation

Nikoleta-v3 commented Apr 5, 2018

Uh oh!

Nikoleta-v3 commented Apr 5, 2018

Uh oh!

Nikoleta-v3 commented Apr 5, 2018

Uh oh!

drvinceknight left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

drvinceknight commented Apr 5, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

drvinceknight Apr 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

drvinceknight Apr 5, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marcharper Apr 6, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

marcharper Apr 6, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

drvinceknight Apr 5, 2018 •

edited

Loading

drvinceknight Apr 5, 2018 •

edited

Loading

marcharper Apr 6, 2018 •

edited

Loading

marcharper Apr 6, 2018 •

edited

Loading