Automatically set strategy classes (and transformers) repr to include parameters. #802 #953

Chadys · 2017-03-31T15:35:57Z

Following the discussion in #802, I did the dynamic naming of strategies, while adding (and using) an init_params class method in Player to get a dict of the init parameters of a strategy (where the values are either given or default one are used). I also fixed #947 while I was at it.

…od-Python#802)

drvinceknight · 2017-03-31T16:56:57Z

It looks like tests are failing because of ordering of the dictionaries being different based on Python versions? (I've looked at this very briefly: https://travis-ci.org/Axelrod-Python/Axelrod/jobs/217220481). Perhaps you can sort the keys?

Chadys · 2017-03-31T17:02:06Z

Yes I'm mostly sure I know where the problem is, I'll get to it.

…xelrod-Python#802)

Chadys · 2017-03-31T17:20:54Z

I made a list instead of a dict, so order should now be ok for both python 3.5 and 3.6. Also, for consistency sake, I should make ParameterisedTestPlayer and TestOpponent be written in the same format, so which format of the two should I use ? I'm more in favor of the in line class declaration I used for ParameterisedTestPlayer since the class is so small, but you tell me :)

Chadys · 2017-03-31T17:42:47Z

Ok so now I have another problem on python 3.5 :

hypothesis.errors.FailedHealthCheck: Data generation is extremely slow: Only produced 6 valid examples in 1.39 seconds (0 invalid ones and 0 exceeded maximum size). Try decreasing size of the data you're generating (with e.g.average_size or max_leaves parameters).

For those :

ERROR: test_decorator_with_stochastic_strategies (axelrod.tests.unit.test_property.TestProbEndSpatialTournament)
ERROR: test_decorator_with_stochastic_strategies (axelrod.tests.unit.test_property.TestProbEndTournament)
ERROR: test_decorator_with_stochastic_strategies (axelrod.tests.unit.test_property.TestSpatialTournament)
ERROR: test_property_serial_play (axelrod.tests.unit.test_tournament.TestTournament)

https://travis-ci.org/Axelrod-Python/Axelrod/jobs/217256127#L709
Apparently the error is caused by the call to the function associated with the hypothesis.given decorator.
Is it a time performance test ?

drvinceknight · 2017-03-31T17:48:05Z

Have you rebased on to master? That's just a health check when hypothesis can't find an example, I thought I'd fixed it by reducing the example size but maybe not. I'll re kick off the build now (nothing that's the fault of your patch).

…

On Fri, 31 Mar 2017, 18:42 Julie Rymer, ***@***.***> wrote: Ok so now I have another problem on python 3.5 : hypothesis.errors.FailedHealthCheck: Data generation is extremely slow: Only produced 6 valid examples in 1.39 seconds (0 invalid ones and 0 exceeded maximum size). Try decreasing size of the data you're generating (with e.g.average_size or max_leaves parameters). For those : ERROR: test_decorator_with_stochastic_strategies (axelrod.tests.unit.test_property.TestProbEndSpatialTournament) ERROR: test_decorator_with_stochastic_strategies (axelrod.tests.unit.test_property.TestProbEndTournament) ERROR: test_decorator_with_stochastic_strategies (axelrod.tests.unit.test_property.TestSpatialTournament) ERROR: test_property_serial_play (axelrod.tests.unit.test_tournament.TestTournament) https://travis-ci.org/Axelrod-Python/Axelrod/jobs/217256127#L709 Apparently the error is caused by the call to the function associated with the hypothesis.given decorator. Is it a time performance test ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#953 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACCGWgAwcm-b1d8WcIrv0xUjIeGETZFQks5rrTsXgaJpZM4Mv0w7> .

drvinceknight · 2017-03-31T17:50:10Z

Also, have you seen the paremterized player test on #952?

I realise I'm not answering your question about that fully, I'll take a closer look later this evening (just taking the 🐕 for a walk :))

Chadys · 2017-03-31T18:11:03Z

Yes I'm doing fetch/merge regularly.
Still the same error but only one left !
Yes I saw, there will be some merge conflicts but once we agree on a format I'll be able to handle it.
No problem, good walk !

drvinceknight · 2017-03-31T19:14:42Z

Yes I'm doing fetch/merge regularly.

Damn... That means my fix aiming to remove that timeout for it didn't work... I'll wait and see if it happens again...

drvinceknight · 2017-03-31T19:18:59Z

In axelrod/tests/unit/test_property on line 237 can you bump the value of min_prob_end to 0.7:

>>235     @given(tournament=prob_end_spatial_tournaments(strategies=stochastic_strategies,                                                            
  236                                                    max_size=3,                                                                                  
~ 237                                                    min_prob_end=0.7))

That might help this stop happening...

drvinceknight · 2017-03-31T19:21:46Z

Also, for consistency sake, I should make ParameterisedTestPlayer and TestOpponent be written in the same format, so which format of the two should I use ? I'm more in favor of the in line class declaration I used for ParameterisedTestPlayer since the class is so small, but you tell me :)

I would prefer the same format of TestOpponent, I agree that the class is small but I'd plead readability (so the way it's done on #952 is preferred please).

drvinceknight · 2017-03-31T19:24:51Z

In axelrod/tests/unit/test_property on line 237 can you bump the value of min_prob_end to 0.7:

(Sorry about this, it's obviously not your fault and just a thing that happens with Hypothesis. Hopefully that'll fix it but I'll keep an eye on it.)

drvinceknight · 2017-03-31T19:26:38Z

Actually! Sorry for all these messages @Chadys! New suggestion. Remove all of these lines:

  235     @given(tournament=prob_end_spatial_tournaments(strategies=stochastic_strategies,                                                            
  236                                                    max_size=3,                                                                                  
  237                                                    min_prob_end=0.7))                                                                           
  238     @settings(max_examples=10, timeout=0)                                                                                                       
  239     def test_decorator_with_stochastic_strategies(self, tournament):                                                                            
  240         self.assertIsInstance(tournament, axelrod.ProbEndSpatialTournament)                                                                     
  241         stochastic_player_names = [str(s()) for s in stochastic_strategies]                                                                     
  242         for p in tournament.players:                                                                                                            
  243             self.assertIn(str(p), stochastic_player_names)

Coverage will not be affected and it's not at all a necessary test.

drvinceknight · 2017-03-31T19:30:41Z

Scrap, all that: sending you a PR to this branch :)

Due to timeouts on travis as discussed on Axelrod-Python#953. - Reduced size in tournament tests. - Removed some tests for the property based tests themselves (this where not necessary and just tested a specific subset of strategies).

drvinceknight · 2017-03-31T19:46:17Z

Chadys#1

Chadys 802 vk

Chadys · 2017-03-31T22:13:52Z

I did the merge and made ParameterisedTestPlayer a class like in #952.
We have some tests for parameters that are redundant with each other, I can do the merge between the two after his PR is closed, since I suppressed init_args (*args and **kwargs are now both cached in init_kwargs), so the conflicts might be more work for him than for me.

marcharper

Looks good overall, just a few questions about some of the changes.

marcharper · 2017-04-01T02:46:44Z

axelrod/player.py

+        sig = inspect.signature(cls.__init__)
+        # the 'self' parameter needs to be removed or the first *args will be assigned to it
+        self_param = sig.parameters.get('self', None)
+        if self_param is not None:


Does this conditional need to be present?

There should always be a 'self' parameters to get rid of, but since I'm calling list.remove which would fail if I call it with None, better be safe than sorry I guess. But I agree that there is no reason an __init__ would ever have no 'self' parameter so I guess I could remove this test.

marcharper · 2017-04-01T02:48:05Z

axelrod/player.py

+        prefix = ': '
+        gen = (value for value in self.init_kwargs.values() if value is not None)
+        for value in gen:
+            name = ''.join([name, prefix, str(value if not isinstance(value, float) else round(value, 2))])


Technically strategies won't have unique names since Random(0.12) and Random(0.121) will get the same name. Are we all ok with that?

I would prefer it if we didn't round. Does that make things messy in some cases?

Originally, the __repr__ of Random was doing a round. When I print all strategies __repr__ with default values I get srings a bit long for those :

ZD-Extort-2: 0.1111111111111111, 0.5 ZD-Extort-2 v2: 0.125, 0.5, 1 ZD-Extort-4: 0.23529411764705882, 0.25, 1

So maybe use a bigger value for round or, if you prefer to be really precise and you don't mind __repr__ like my example above, I can suppress it

I suppose ultimately the floats in the __repr__ are being truncated anyway. I'm relatively indifferent between:

Leaving without any rounding (or in essence using the default rounding of a python float);

Perhaps rounding to 4 digits?

I prefer less aesthetically pleasing in favor of more uniqueness in this case

marcharper · 2017-04-01T02:51:50Z

axelrod/strategies/gobymajority.py

        self.name = (
            'Go By Majority' + (self.memory > 0) * (": %i" % self.memory))
        if self.soft:
            self.name = "Soft " + self.name
        else:
            self.name = "Hard " + self.name

+    def __repr__(self):


This is the default __repr__ from Player, yes? If so I think we don't need to add it again explicitly.

When I don't override __repr__ here I get that :

Soft Go By Majority: inf, True Soft Go By Majority: 10 Soft Go By Majority: 20 Soft Go By Majority: 40 Soft Go By Majority: 5 ... Hard Go By Majority: inf Hard Go By Majority: 10 Hard Go By Majority: 20 Hard Go By Majority: 40 Hard Go By Majority: 5

The 'True' is redundant with the 'Soft' name, so I guess I should effectively get rid of this override but also get rid of this lines (gobymajority.py L54-59) :

self.name = ( 'Go By Majority' + (self.memory > 0) * (": %i" % self.memory)) if self.soft: self.name = "Soft " + self.name else: self.name = "Hard " + self.name

and add the 'Soft' manually in the names of the GoByMajority_number ? But that means the GoByMajority main class will no longer be call 'Soft Go By Majority' or 'Hard Go By Majority' but 'Go By Majority: True' or 'Go By Majority: False', are we okay with that ?

'Go By Majority: True' or 'Go By Majority: False', are we okay with that ?

I'm fine with that.

I think we should prefer "Soft" since it's the name in literature

Knowing that it would just apply for the main GoByMajority, the SoftGoByMajority40 would still be called 'Soft Go By Majority 40', the same for all the other versions, and it would avoid an override of __repr__

For now I did the 'Go By Majority: True' option but if everyone agree on the other one, I will put back the override

On reflection I agree with @marcharper, keeping in line with the literature is good.

do I still keep the ': inf' though ?

Perhaps it's not needed for the base one.

marcharper · 2017-04-01T03:01:11Z

axelrod/tests/strategies/test_player.py

+    classifier = _test_classifier
+
+    def __init__(self, arg_test1='testing1', arg_test2='testing2'):
+        super().__init__()


Do we not need to pass the arguments through to the super init?

No since it calls Player.__init__ which takes no argument

marcharper · 2017-04-01T03:01:51Z

axelrod/tests/strategies/test_player.py

@@ -270,7 +307,7 @@ def attribute_equality_test(self, player, clone):
    def test_clone(self):
        # Test that the cloned player produces identical play
        player1 = self.player()
-        if str(player1) in ["Darwin", "Human"]:
+        if str(player1)[:6] in ["Darwin", "Human:"]:


Why does Human have a : now?

Because Human now get its str in the form 'Human: human_name, symbol1, symbol2' so I can't test the exact name so I did a split and I needed the same number of letters for Darwin and Human, hence the "Human:". But I could replace that line by :

if player1.name in ["Darwin", "Human"]:

it is more explicit.
But that got me to identify a little bug :

In [5]: h= axl.Human('coco') In [6]: str(h) Out[6]: 'coco: coco, C, D'

So the question is : Do we want str(h) in that case to return 'Human: coco, C, D' or 'coco: C, D' ? I'll correct my code with the solution choosen

I'd vote for: Human: coco, C, D.

drvinceknight · 2017-04-01T06:26:34Z

#952 has been merged now (with the parametrized test player).

Chadys · 2017-04-01T11:21:48Z

I'll do a fetch/merge and correct conflicts later today :)

This reverts commit 180e28c.

Chadys · 2017-04-01T20:02:50Z

I did all the asked changes and merged with the latest version 😄

drvinceknight · 2017-04-02T06:39:20Z

I'm taking @marcharper's approved review as an indication that this is good to go. 👍

Chadys added 14 commits March 27, 2017 11:34

include transformers parameters (Axelrod-Python#802)

5dfd22b

Automatically add parameters in player.__repr__ (Axelrod-Python#802)

8ebd43f

Put Transformers' args after transformed player's args in name (Axelr…

b189a93

…od-Python#802)

Updated tests with new players' name (Axelrod-Python#802)

08477d9

Merge remote-tracking branch 'upstream/master' into 802

239492e

update .gitignore to include Jetbrain editor settings

86ed15d

added test_init_params for Player (Axelrod-Python#802)

c150911

init_param *args management (Axelrod-Python#802)

bf83ba6

added more test for init_params and init_kwargs (Axelrod-Python#802)

8296a89

remove *init_args use (Axelrod-Python#802)

a40ecc0

better strategy transformers arg repr (Axelrod-Python#802)

6d2dfb4

Fix bug in testing human with new repr (Axelrod-Python#802)

6bd8b04

add comment explaining init_params (Axelrod-Python#802)

eff83ec

better organisation of ParameterisedTestPlayer (Axelrod-Python#802)

0c9519b

Chadys added 2 commits March 31, 2017 19:14

added common classifier for TestOpponent and ParameterisedTestPlayer (A…

f5f76be

…xelrod-Python#802)

Resolve bad ordering of parameters for python3.5 (Axelrod-Python#802)

a10e2f0

Merge branch '802' of https://github.com/Chadys/Axelrod into Chadys-802

e14dd08

Reduce quantity/size of property based tests.

d70a2ac

Due to timeouts on travis as discussed on Axelrod-Python#953. - Reduced size in tournament tests. - Removed some tests for the property based tests themselves (this where not necessary and just tested a specific subset of strategies).

Chadys added 2 commits March 31, 2017 21:55

Merge pull request #1 from Axelrod-Python/Chadys-802-vk

b7b8ac5

Chadys 802 vk

ParameterisedTestPlayer as class (Axelrod-Python#802)

e744914

marcharper requested changes Apr 1, 2017

View reviewed changes

Chadys added 6 commits April 1, 2017 17:18

round to 4 digits (Axelrod-Python#802)

fa898a4

remove __repr__ override in GoByMajority (Axelrod-Python#802)

180e28c

correct __repr__ bug for human (Axelrod-Python#802)

f603acb

Merge remote-tracking branch 'upstream/master' into 802

c06dca4

no round in __repr__ (Axelrod-Python#802)

66ac41f

Revert "remove __repr__ override in GoByMajority (Axelrod-Python#802)"

1151f2c

This reverts commit 180e28c.

marcharper approved these changes Apr 1, 2017

View reviewed changes

drvinceknight merged commit d0c1636 into Axelrod-Python:master Apr 2, 2017

This was referenced Apr 2, 2017

Automatically set strategy classes (and transformers) __repr__ to include parameters. #802

Closed

The __repr__ for the human strategy includes an extra C and D. #958

Closed

Automatically set strategy classes (and transformers) __repr__ to include parameters. #802 #953

Automatically set strategy classes (and transformers) __repr__ to include parameters. #802 #953

Conversation

Chadys commented Mar 31, 2017

drvinceknight commented Mar 31, 2017

Chadys commented Mar 31, 2017

Chadys commented Mar 31, 2017

Chadys commented Mar 31, 2017

drvinceknight commented Mar 31, 2017 via email

drvinceknight commented Mar 31, 2017

Chadys commented Mar 31, 2017 • edited Loading

drvinceknight commented Mar 31, 2017

drvinceknight commented Mar 31, 2017

drvinceknight commented Mar 31, 2017

drvinceknight commented Mar 31, 2017

drvinceknight commented Mar 31, 2017

drvinceknight commented Mar 31, 2017

drvinceknight commented Mar 31, 2017

Chadys commented Mar 31, 2017

marcharper left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chadys Apr 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Chadys Apr 1, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drvinceknight commented Apr 1, 2017

Chadys commented Apr 1, 2017

Chadys commented Apr 1, 2017

drvinceknight commented Apr 2, 2017

Automatically set strategy classes (and transformers) repr to include parameters. #802 #953

Automatically set strategy classes (and transformers) repr to include parameters. #802 #953

Chadys commented Mar 31, 2017 •

edited

Loading

Chadys Apr 1, 2017 •

edited

Loading

Chadys Apr 1, 2017 •

edited

Loading