Refactor all strategy tests

[drvinceknight]

With the merge of #875 we have a new method for testing strategies called versus_test (http://axelrod.readthedocs.io/en/latest/tutorials/contributing/strategy/writing_test_for_the_new_strategy.html). The goal is for this to eventually replace the currently used responses_test.

This will take a slow refactor of all strategy tests. Ideally any PRs towards this issue should only do a small number of modules at a time to ensure that nothing is missed during the review process.

EDIT: Once all of these are complete, we can refactor the TestPlayer class to remove unused methods.

[ghost]

Hi I would like to help with this. Its my first contribution, any guidance how to start, I will much appreciate it.

[drvinceknight]

Hi @alajara, welcome!

We have introduced a new testing framework for strategies, you can read about this here: http://axelrod.readthedocs.io/en/latest/tutorials/contributing/strategy/writing_test_for_the_new_strategy.html

So this issue is about rewriting the various tests that have already been written for strategies using the new framework.

Here are some PRs that I've written that do this:

• https://github.com/Axelrod-Python/Axelrod/pull/893/files
• https://github.com/Axelrod-Python/Axelrod/pull/909/files
• https://github.com/Axelrod-Python/Axelrod/pull/914/files

So the idea is to look at the files in axelrod/tests/strategies/ and modify them as you can see in those PRs.

I've assumed there that you're ok with the notion of automated testing but please let me know if you're not and I'd be happy to walk you through anything/everything :)

To help coordinate (I'm slowly working on these myself) perhaps once you're ready to go, comment on this issue when you start working on a particular test file (just so we don't work on the same one). Also, to help keep the reviews simple: let's stick to just one file per PR.

Let me know if anything I've said is unclear :) Looking forward to your first contribution!

[eric-s-s]

Hello Dr Knight,

I'm also interested in helping with this. should i simply write a comment on which strategy i'm doing? This is my first time contributing to someone else's project. while trying to understand the process, i came across a possible issue with backstabber.py The DoubleCrosser class does not follow its documentation and the test to demonstrate it doesn't seem to do anything. i'm not sure if this is the appropriate place for this, but here are code snippets.

backstabber.py

    @FinalTransformer((D, D), name_prefix=None) # End with two defections
    class DoubleCrosser(Player):
        """
        Forgives the first 3 defections but on the fourth
        will defect forever. If the opponent did not defect
        in the first 6 rounds the player will cooperate until
        the 180th round. Defects on the last 2 rounds unconditionally.
        """
        def strategy(self, opponent: Player) -> Action:
           cutoff = 6

            if not opponent.history:
                return C
            if len(opponent.history) < 180:
                if len(opponent.history) > cutoff:
                    if D not in opponent.history[:cutoff + 1]:
                        if opponent.history[-2:] != [D, D]:  # Fail safe
                            return C
            # a quick refactor so i can see what the author meant (not, i think, what was intended)
            # if cutoff < len(opponent.history) < 180:
            #     first_seven = opponent.history[:cutoff + 1]
            #     last_two = opponent.history[-2:]
            #     if D not in first_seven and last_two != [D, D]:
            #         return C

the test strategy doesn't seem to show this. here's from test_backstabber.py

        # If opponent did not defect in the first six rounds, cooperate until
        # round 180
        self.responses_test([C] * 174, [C] * 6, [C] * 6, length=200)
        self.responses_test([C] * 160, [C] * 12, [C] * 6 + [D] + [C] * 5,
                            length=200)

if i understand these tests correctly, DoubleCrosser will always return C vs a MockPlayer who always returns C (except for one D in round 7 on the second test). This is no different from the other class BackStabber.

-Eric Shaw

[uglyfruitcake]

Hi @eric-s-s,

Welcome to the Axelrod-Python Project!

I'm Tom the original author of Backstabber and Doublecrosser.

The current code for Doublecrosser is very old and was written when I was very new to Python. As far as I can tell your refactor does do exactly the same thing (and in a much more elegant way).

You say that due to the tests you have shown, Backstabber and Doublecrosser do the same thing. From the situation described in the tests, they are supposed to. The difference between the two strategies should only come to light when an opponent did not defect in the first 6 turns and then subsequently defects more than 3 times (not consecutively). In this very specific situation, Doublecrosser will continue to cooperate whereas Backstabber will register the 4 defects and defect in response.

The tests that you showed did not highlight this situation and hence made it look like they were doing the same thing (which in that situation they are).

Hope this helps to clear things up!

Tom.

[ghost]

Hey Dr. Knight I can begin working on worse and worse, if it's open.

[eric-s-s]

@uglyfruitcake excellent. thank you very much. while i'm changing from self.response_test to self.versus_test i'll put that test situation in there. just to confirm for DoubleCrosser

opponent = 6 * [C] + 4 * [D] -> next move (and all other moves or will it revert if opponent cooperates?) = D
opponent = 6 * [C] + [D, C, D, D, D] -> next move = C
opponent = 6 * [C] + [D, C, D, D, D] + 171 * [C] -> next move = D

[uglyfruitcake]

@eric-s-s

Not quite. Here's what the tests should be I think.

6 * [C] + 2 * [D] -> D
6 * [C] + 20* [D] -> D
6 * [C] + 2 * [D] + [C] - > C
6 * [C] + 20 * [D] + [C] - > C

Between round 7 - 180 it will only defect if the opponents previous 2 were [D, D] or if the opponent defected in the first 6 rounds. Nothing else is relevant.

After 180 rounds it is EXACTLY like Backstabber. No difference.

Hope that helps.

Tom

[marcharper]

Since there are several people working on this issue please just state here which strategies you are handling. Feel free to get started wherever you like and open a PR when you think you are ready to merge.

[eric-s-s]

working on backstabber

@uglyfruitcake thank you. please review the PR i'll post later. DoubleCrosser had an off-by-one error. so i ?fixed? it and did a little refactor.

@marcharper sorry. i am new to all this, so i'm not sure where/how to raise the issue of test coverage/bug on a specific strategy.

[drvinceknight]

@marcharper sorry. i am new to all this, so i'm not sure where/how to raise the issue of test coverage/bug on a specific strategy. is there a protocol for doing so?

@eric-s-s you can just go ahead and address it in the PR under this issue (no need for a new issue). We'll discuss anything relevant on the PR itself :)

[drvinceknight]

Hey Dr. Knight I can begin working on worse and worse, if it's open.

Please do @alajara! :)

[eric-s-s]

i'll work on the b's and c's.
better_and_better
calculator
cooperator
cycler

[drvinceknight]

Picking up test_mathematicalconstants.

[drvinceknight]

Picking up test_memorytwo.

[drvinceknight]

Picking up test_memoryone.

[drvinceknight]

Picking up test_hunter.

[drvinceknight]

I believe #1035 might be the last of them. Will wait to adjust as required and get them all merged before taking stock 👍

[drvinceknight]

All the strategy tests have been refactored. I'm going to refactor some of the boiler plate code in test_player to remove what is no longer needed. We still use responses_test for first_play_test and second_play_test but I'll take a look at if anything can be simplified there. That should close this issue.

[drvinceknight]

Thanks to everyone for your efforts on this! Closed by #1036