"MetaClass update_histories issue" when running all strategies

Question

"MetaClass update_histories issue" when running all strategies

Closed this issue 3 years ago · 14 comments

How to reproduce:

git clone https://github.com/Axelrod-Python/Axelrod.git
cd Axelrod
python3.9 setup.py install
git checkout ebcdf99e   # current HEAD on dev branch
python3.9 --version   # Python 3.9.5

>>> import axelrod as axl
>>> players = [s() for s in axl.all_strategies]
>>> tournament = axl.Tournament(players, seed=1, turns=10)
>>> results = tournament.play()

Issue seems to occur always at 71%, no matter how many turns (so you can try with less for quicker debugging)

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/strategies/meta.py", line 97, in update_histories
    for player, play in zip(self.team, self._last_results):
TypeError: 'NoneType' object is not iterable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/tournament.py", line 147, in play
    self._run_serial(build_results=build_results)
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/tournament.py", line 176, in _run_serial
    results = self._play_matches(chunk, build_results=build_results)
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/tournament.py", line 456, in _play_matches
    match.play()
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/match.py", line 195, in play
    plays = self.simultaneous_play(
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/match.py", line 157, in simultaneous_play
    coplayer.update_history(s2, s1)
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/strategies/meta.py", line 117, in update_history
    self.update_histories(coplay)
  File "/usr/local/lib/python3.9/dist-packages/Axelrod-4.10.0-py3.9.egg/axelrod/strategies/meta.py", line 107, in update_histories
    raise TypeError(
TypeError: MetaClass update_histories issue, expected a reclassifier.

Answer 1 · 2021-05-21T15:00:38.000Z

Thanks @xjcl I confirm I'm getting the same error with:

>>> import axelrod as axl
>>> players = [s() for s in axl.all_strategies]
>>> tournament = axl.Tournament(players, seed=1, turns=2, repetitions=1)
>>> results = tournament.play()

Note that if I run a tournament just with meta players it runs fine:

>>> players = [
...     axl.MetaHunter(), 
...     axl.MetaHunterAggressive(), 
...     axl.MetaMajority(),
...     axl.MetaMajorityFiniteMemory(),
...     axl.MetaMajorityLongMemory(),
...     axl.MetaMajorityMemoryOne(),
...     axl.MetaMinority(),
...     axl.MetaMixer(),
...     axl.MetaWinnerDeterministic(),
...     axl.MetaWinnerEnsemble(),
...     axl.MetaWinnerFiniteMemory(),
...     axl.MetaWinnerLongMemory(),
...     axl.MetaWinnerMemoryOne(),
...     axl.MetaWinnerStochastic(),
... ]
>>> tournament = axl.Tournament(players=players, seed=1, turns=2, repetitions=1)
>>> tournament.play()

Answer 2 · 2021-05-21T15:09:25.000Z

Could it be due to one of the "cheating" strategies?

Answer 3 · 2021-05-21T15:22:20.000Z

Could it be due to one of the "cheating" strategies?

Yup spot on. Just about to comment with code.

Answer 4 · 2021-05-21T15:27:08.000Z

I just ran:

import axelrod as axl
players = [s() for s in axl.all_strategies]
tournament = axl.Tournament(players, seed=1, turns=2, repetitions=1)
chunks = tournament.match_generator.build_match_chunks()
for chunk in chunks:
    try:
        tournament._play_matches(chunk)
    except TypeError:
        index_pair, match_params, repetitions, r = chunk
        problem_players = players[index_pair[0]], players[index_pair[1]]
        print(f"Match failed for match chunk: with player index pair = {index_pair} corresponding to play pair = {problem_players}")

(That replicates what the tournament does with an error catch for the problem)

That outputs:

Match failed for match chunk: with player index pair = (114, 223) corresponding to play pair = (Mind Bender, Memory Decay: 0.1, 0.03, -2, 1, Tit For Tat, 15)
Match failed for match chunk: with player index pair = (114, 224) corresponding to play pair = (Mind Bender, Meta Hunter: 6 players)
Match failed for match chunk: with player index pair = (114, 225) corresponding to play pair = (Mind Bender, Meta Hunter Aggressive: 7 players)
Match failed for match chunk: with player index pair = (114, 226) corresponding to play pair = (Mind Bender, Meta Majority: 213 players)
Match failed for match chunk: with player index pair = (114, 227) corresponding to play pair = (Mind Bender, Meta Majority Memory One: 36 players)
Match failed for match chunk: with player index pair = (114, 228) corresponding to play pair = (Mind Bender, Meta Majority Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (114, 229) corresponding to play pair = (Mind Bender, Meta Majority Long Memory: 134 players)
Match failed for match chunk: with player index pair = (114, 230) corresponding to play pair = (Mind Bender, Meta Minority: 213 players)
Match failed for match chunk: with player index pair = (114, 231) corresponding to play pair = (Mind Bender, Meta Mixer: 213 players)
Match failed for match chunk: with player index pair = (114, 232) corresponding to play pair = (Mind Bender, Meta Winner: 213 players)
Match failed for match chunk: with player index pair = (114, 233) corresponding to play pair = (Mind Bender, Meta Winner Deterministic: 146 players)
Match failed for match chunk: with player index pair = (114, 234) corresponding to play pair = (Mind Bender, Meta Winner Ensemble: 213 players)
Match failed for match chunk: with player index pair = (114, 235) corresponding to play pair = (Mind Bender, Meta Winner Memory One: 36 players)
Match failed for match chunk: with player index pair = (114, 236) corresponding to play pair = (Mind Bender, Meta Winner Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (114, 237) corresponding to play pair = (Mind Bender, Meta Winner Long Memory: 134 players)
Match failed for match chunk: with player index pair = (114, 238) corresponding to play pair = (Mind Bender, Meta Winner Stochastic: 67 players)
Match failed for match chunk: with player index pair = (114, 239) corresponding to play pair = (Mind Bender, NMWE Deterministic: 146 players)
Match failed for match chunk: with player index pair = (114, 240) corresponding to play pair = (Mind Bender, NMWE Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (114, 241) corresponding to play pair = (Mind Bender, NMWE Long Memory: 134 players)
Match failed for match chunk: with player index pair = (114, 242) corresponding to play pair = (Mind Bender, NMWE Memory One: 36 players)
Match failed for match chunk: with player index pair = (114, 243) corresponding to play pair = (Mind Bender, NMWE Stochastic: 67 players)
Match failed for match chunk: with player index pair = (114, 244) corresponding to play pair = (Mind Bender, Nice Meta Winner: 213 players)
Match failed for match chunk: with player index pair = (114, 245) corresponding to play pair = (Mind Bender, Nice Meta Winner Ensemble: 213 players)
Match failed for match chunk: with player index pair = (115, 223) corresponding to play pair = (Mind Controller, Memory Decay: 0.1, 0.03, -2, 1, Tit For Tat, 15)
Match failed for match chunk: with player index pair = (115, 224) corresponding to play pair = (Mind Controller, Meta Hunter: 6 players)
Match failed for match chunk: with player index pair = (115, 225) corresponding to play pair = (Mind Controller, Meta Hunter Aggressive: 7 players)
Match failed for match chunk: with player index pair = (115, 226) corresponding to play pair = (Mind Controller, Meta Majority: 213 players)
Match failed for match chunk: with player index pair = (115, 227) corresponding to play pair = (Mind Controller, Meta Majority Memory One: 36 players)
Match failed for match chunk: with player index pair = (115, 228) corresponding to play pair = (Mind Controller, Meta Majority Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (115, 229) corresponding to play pair = (Mind Controller, Meta Majority Long Memory: 134 players)
Match failed for match chunk: with player index pair = (115, 230) corresponding to play pair = (Mind Controller, Meta Minority: 213 players)
Match failed for match chunk: with player index pair = (115, 231) corresponding to play pair = (Mind Controller, Meta Mixer: 213 players)
Match failed for match chunk: with player index pair = (115, 232) corresponding to play pair = (Mind Controller, Meta Winner: 213 players)
Match failed for match chunk: with player index pair = (115, 233) corresponding to play pair = (Mind Controller, Meta Winner Deterministic: 146 players)
Match failed for match chunk: with player index pair = (115, 234) corresponding to play pair = (Mind Controller, Meta Winner Ensemble: 213 players)
Match failed for match chunk: with player index pair = (115, 235) corresponding to play pair = (Mind Controller, Meta Winner Memory One: 36 players)
Match failed for match chunk: with player index pair = (115, 236) corresponding to play pair = (Mind Controller, Meta Winner Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (115, 237) corresponding to play pair = (Mind Controller, Meta Winner Long Memory: 134 players)
Match failed for match chunk: with player index pair = (115, 238) corresponding to play pair = (Mind Controller, Meta Winner Stochastic: 67 players)
Match failed for match chunk: with player index pair = (115, 239) corresponding to play pair = (Mind Controller, NMWE Deterministic: 146 players)
Match failed for match chunk: with player index pair = (115, 240) corresponding to play pair = (Mind Controller, NMWE Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (115, 241) corresponding to play pair = (Mind Controller, NMWE Long Memory: 134 players)
Match failed for match chunk: with player index pair = (115, 242) corresponding to play pair = (Mind Controller, NMWE Memory One: 36 players)
Match failed for match chunk: with player index pair = (115, 243) corresponding to play pair = (Mind Controller, NMWE Stochastic: 67 players)
Match failed for match chunk: with player index pair = (115, 244) corresponding to play pair = (Mind Controller, Nice Meta Winner: 213 players)
Match failed for match chunk: with player index pair = (115, 245) corresponding to play pair = (Mind Controller, Nice Meta Winner Ensemble: 213 players)
Match failed for match chunk: with player index pair = (117, 223) corresponding to play pair = (Mind Warper, Memory Decay: 0.1, 0.03, -2, 1, Tit For Tat, 15)
Match failed for match chunk: with player index pair = (117, 224) corresponding to play pair = (Mind Warper, Meta Hunter: 6 players)
Match failed for match chunk: with player index pair = (117, 225) corresponding to play pair = (Mind Warper, Meta Hunter Aggressive: 7 players)
Match failed for match chunk: with player index pair = (117, 226) corresponding to play pair = (Mind Warper, Meta Majority: 213 players)
Match failed for match chunk: with player index pair = (117, 227) corresponding to play pair = (Mind Warper, Meta Majority Memory One: 36 players)
Match failed for match chunk: with player index pair = (117, 228) corresponding to play pair = (Mind Warper, Meta Majority Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (117, 229) corresponding to play pair = (Mind Warper, Meta Majority Long Memory: 134 players)
Match failed for match chunk: with player index pair = (117, 230) corresponding to play pair = (Mind Warper, Meta Minority: 213 players)
Match failed for match chunk: with player index pair = (117, 231) corresponding to play pair = (Mind Warper, Meta Mixer: 213 players)
Match failed for match chunk: with player index pair = (117, 232) corresponding to play pair = (Mind Warper, Meta Winner: 213 players)
Match failed for match chunk: with player index pair = (117, 233) corresponding to play pair = (Mind Warper, Meta Winner Deterministic: 146 players)
Match failed for match chunk: with player index pair = (117, 234) corresponding to play pair = (Mind Warper, Meta Winner Ensemble: 213 players)
Match failed for match chunk: with player index pair = (117, 235) corresponding to play pair = (Mind Warper, Meta Winner Memory One: 36 players)
Match failed for match chunk: with player index pair = (117, 236) corresponding to play pair = (Mind Warper, Meta Winner Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (117, 237) corresponding to play pair = (Mind Warper, Meta Winner Long Memory: 134 players)
Match failed for match chunk: with player index pair = (117, 238) corresponding to play pair = (Mind Warper, Meta Winner Stochastic: 67 players)
Match failed for match chunk: with player index pair = (117, 239) corresponding to play pair = (Mind Warper, NMWE Deterministic: 146 players)
Match failed for match chunk: with player index pair = (117, 240) corresponding to play pair = (Mind Warper, NMWE Finite Memory: 79 players)
Match failed for match chunk: with player index pair = (117, 241) corresponding to play pair = (Mind Warper, NMWE Long Memory: 134 players)
Match failed for match chunk: with player index pair = (117, 242) corresponding to play pair = (Mind Warper, NMWE Memory One: 36 players)
Match failed for match chunk: with player index pair = (117, 243) corresponding to play pair = (Mind Warper, NMWE Stochastic: 67 players)
Match failed for match chunk: with player index pair = (117, 244) corresponding to play pair = (Mind Warper, Nice Meta Winner: 213 players)
Match failed for match chunk: with player index pair = (117, 245) corresponding to play pair = (Mind Warper, Nice Meta Winner Ensemble: 213 players)

We can confirm that with:

>>> problem_players = (axl.MindWarper(), axl.MetaWinnerEnsemble())
>>> match = axl.Match(players=problem_players)
>>> match.play()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/anaconda3/lib/python3.8/site-packages/axelrod/strategies/meta.py in update_histories(self, coplay)
     96        
     ...

Answer 5 · 2021-05-21T15:29:19.000Z

I would suggest that given the maturity of the library as a research project it could be time to remove the cheating strategies? We could open an issue to possibly add them as a separate "add on" library?

Answer 6 · 2021-05-21T15:44:52.000Z

Yeah or at least remove the cheating strategies from all_strategies

Answer 7 · 2021-05-21T15:45:09.000Z

In the meantime @xjcl, assuming you are not specifically interesting in the cheating strategies, you should be able to run:

>>> import axelrod as axl
>>> players = [s() for s in axl.all_strategies if s not in axl.cheating_strategies]
>>> tournament = axl.Tournament(players, seed=1, turns=10)
>>> results = tournament.play()

(Some documentation on this here: https://axelrod.readthedocs.io/en/stable/tutorials/advanced/strategies.html)

Answer 8 · 2021-05-21T15:46:20.000Z

Yeah or at least remove the cheating strategies from all_strategies

Good call, in which case we could document how to access them if someone really wants to play with them (With a bit of a disclaimer that they might not work against some players like the Meta players for example).

Answer 9 · 2021-05-21T15:49:24.000Z

I'm in favor of removing them as well. They are fun but cause much more trouble than they are fun.

Answer 10 · 2021-05-21T15:52:04.000Z

Yep. Shifting them elsewhere seems a good idea

Answer 11 · 2021-05-21T20:01:27.000Z

Can confirm the issue is also present in the PyPI version (Axelrod==4.10.0).

My fix for now:

players = [p for p in players if not str(p).startswith('Mind')]

Answer 12 · 2021-05-22T02:16:51.000Z

Hi @xjcl, @drvinceknight's method is recommended, since it knows if the strategies obey the standard rules or not.

One can also use:

[p for p in players if axl.Classifiers.obey_axelrod(p)]

or simply use something like axl.short_run_time_strategies().

Answer 13 · 2021-05-24T09:47:17.000Z

Yep. Shifting them elsewhere seems a good idea

Putting a PR together now :)

Answer 14 · 2021-05-25T08:29:11.000Z

@xjcl the latest release of Axelrod 4.11.0 has this bug fixed now: https://pypi.org/project/Axelrod/.