Player legal moves are accessed before being set

Question

Player legal moves are accessed before being set

dnns92 opened this issue 4 years ago · 1 comments

Describe the bug
Using a blank installation from the repo following your description and running "python .\main.py selfplay dqn_train" leads to self.legal_moves being None instead of an iteratable when called using list comprehention, leading to a type error.

Steps To Reproduce
Steps to reproduce the behavior:
1.) be on win10, dl your repo using github and unzip it
2.) install the repo packages following your instructions using conda and pip -r requirements.txt
3.) cd to dir, call "python .\main.py selfplay dqn_train" from powershell
4.) self.legal_moves_limit = [move.value for move in self.legal_moves_limit]
TypeError: 'NoneType' object is not iterable

Expected behavior
training

Proposed Next Steps
There are a lot of deprecation warnings appearing which may be related to the problem. Maybe it is a simple updated package that does not work anymore as the devs intended. Could you maybe put your package version information to the requirements.txt? using pip list and conda list should be suitable!

Additional context
System Information:
Win10 version 2004
gpu: RTX2060

----------- SCROLLBACK ------------------------

(base) PS D:\poker\neuron_poker-master\tests> conda activate neuron_poker
(neuron_poker) PS D:\poker\neuron_poker-master\tests> cd ..
(neuron_poker) PS D:\poker\neuron_poker-master> python .\main.py
Usage:
main.py selfplay random [options]
main.py selfplay keypress [options]
main.py selfplay consider_equity [options]
main.py selfplay equity_improvement --improvement_rounds=<> [options]
main.py selfplay dqn_train [options]
main.py selfplay dqn_play [options]
main.py learn_table_scraping [options]
(neuron_poker) PS D:\poker\neuron_poker-master> python .\main.py selfplay dqn_train
Using default log file
Saving log file to: D:\poker\neuron_poker-master\log\default.log
Saving info file to: D:\poker\neuron_poker-master\log\default_info.log
Saving error only file to: D:\poker\neuron_poker-master\log\default_errors.log
Screenloglevel: 20
INFO - Initializing program
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorflow\python\framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorflow\python\framework\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorflow\python\framework\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
D:\Continuum\envs\neuron_poker\lib\site-packages\tensorboard\compat\tensorflow_stub\dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Using TensorFlow backend.
DQN_TRAIN_KERAS_RL
INFO -
INFO - ++++++++++++++++++
INFO - Starting new hand.
INFO - ++++++++++++++++++
INFO - Dealer is at position 0
INFO - Player 0 got ['KD', '2H'] and $100
INFO - Player 1 got ['9D', 'JC'] and $100
INFO - Player 2 got ['QD', '6H'] and $100
INFO - Player 3 got ['7D', 'AD'] and $100
INFO - Player 4 got ['8D', 'JS'] and $100
INFO - Player 5 got ['JH', '4H'] and $100
INFO -
INFO - ===Round: Stage: PREFLOP
INFO - Seat 1 (equity/20/30): Action.SMALL_BLIND - Remaining stack: 99, Round pot: 1, Community pot: 0, player pot: 1
INFO - Seat 2 (Random): Action.BIG_BLIND - Remaining stack: 98, Round pot: 3, Community pot: 0, player pot: 2
INFO - Seat 3 (Random): Action.FOLD - Remaining stack: 100, Round pot: 3, Community pot: 0, player pot: 0
INFO - Seat 4 (Random): Action.RAISE_3BB - Remaining stack: 92, Round pot: 11, Community pot: 0, player pot: 8
WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\backend\tensorflow_backend.py:66: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\backend\tensorflow_backend.py:541: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.

WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\backend\tensorflow_backend.py:4432: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.

WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\backend\tensorflow_backend.py:148: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.

WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\backend\tensorflow_backend.py:3733: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob.
WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\backend\tensorflow_backend.py:190: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.

2020-08-31 20:44:20.497962: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\optimizers.py:793: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.

WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\callbacks.py:1122: The name tf.summary.merge_all is deprecated. Please use tf.compat.v1.summary.merge_all instead.

WARNING - From D:\Continuum\envs\neuron_poker\lib\site-packages\keras\callbacks.py:1125: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.

Training for 1000 steps ...
INFO -
INFO - ++++++++++++++++++
INFO - Starting new hand.
INFO - ++++++++++++++++++
INFO - Dealer is at position 0
INFO - Player 0 got ['5D', '4C'] and $100
INFO - Player 1 got ['KH', '7H'] and $100
INFO - Player 2 got ['4H', 'JC'] and $100
INFO - Player 3 got ['AH', '9C'] and $100
INFO - Player 4 got ['AC', '3D'] and $100
INFO - Player 5 got ['KD', '6H'] and $100
INFO -
INFO - ===Round: Stage: PREFLOP
INFO - Seat 1 (equity/20/30): Action.SMALL_BLIND - Remaining stack: 99, Round pot: 1, Community pot: 0, player pot: 1
INFO - Seat 2 (Random): Action.BIG_BLIND - Remaining stack: 98, Round pot: 3, Community pot: 0, player pot: 2
INFO - Seat 3 (Random): Action.RAISE_2POT - Remaining stack: 94, Round pot: 9, Community pot: 0, player pot: 6
INFO - Seat 4 (Random): Action.RAISE_2POT - Remaining stack: 82, Round pot: 27, Community pot: 0, player pot: 18
INFO - Chosen action by keras-rl 1 - probabilities: [0.11855229 0.14552703 0.17519688 0.1530902 0.15422752 0.1201125
0.13329357]
Traceback (most recent call last):
File ".\main.py", line 261, in
command_line_parser()
File ".\main.py", line 76, in command_line_parser
runner.dqn_train_keras_rl(model_name)
File ".\main.py", line 208, in dqn_train_keras_rl
dqn.train(env_name=model_name)
File "D:\poker\neuron_poker-master\agents\agent_keras_rl_dqn.py", line 100, in train
start_step_policy=self.start_step_policy, callbacks=[tensorboard])
File "D:\Continuum\envs\neuron_poker\lib\site-packages\rl\core.py", line 171, in fit
action = self.processor.process_action(action)
File "D:\poker\neuron_poker-master\agents\agent_keras_rl_dqn.py", line 211, in process_action
self.legal_moves_limit = [move.value for move in self.legal_moves_limit]
TypeError: 'NoneType' object is not iterable

Answer 1 · 2020-12-03T13:28:44.000Z

Same issue for MacOs with python 3.7.1 and

absl-py 0.11.0
asgiref 3.3.1
astor 0.8.1
astroid 2.4.2
astunparse 1.6.3
attrs 20.3.0
cached-property 1.5.2
cachetools 4.1.1
certifi 2020.11.8
chardet 3.0.4
cloudpickle 1.6.0
cppimport 20.8.4.2
cycler 0.10.0
Django 3.1.4
docopt 0.6.2
future 0.18.2
gast 0.3.3
google-auth 1.23.0
google-auth-oauthlib 0.4.2
google-pasta 0.2.0
grpcio 1.34.0
gym 0.17.3
h5py 2.10.0
idna 2.10
image 1.5.33
importlib-metadata 3.1.1
iniconfig 1.1.1
isort 5.6.4
Keras 2.2.5
Keras-Applications 1.0.8
Keras-Preprocessing 1.1.2
keras-rl 0.4.2
kiwisolver 1.3.1
lazy-object-proxy 1.4.3
Mako 1.1.3
Markdown 3.3.3
MarkupSafe 1.1.1
matplotlib 3.3.3
mccabe 0.6.1
numpy 1.19.4
oauthlib 3.1.0
opt-einsum 3.3.0
packaging 20.7
pandas 1.1.4
Pillow 8.0.1
pip 20.3.1
pluggy 0.13.1
protobuf 3.14.0
py 1.9.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pybind11 2.6.1
pydocstyle 5.1.1
pyglet 1.5.0
pylint 2.6.0
pyparsing 2.4.7
pytest 6.1.2
python-dateutil 2.8.1
pytz 2020.4
PyYAML 5.3.1
requests 2.25.0
requests-oauthlib 1.3.0
rsa 4.6
scipy 1.4.1
setuptools 50.3.2
six 1.15.0
snowballstemmer 2.0.0
sqlparse 0.4.1
tensorboard 1.14.0
tensorboard-plugin-wit 1.7.0
tensorflow 1.14.0
tensorflow-estimator 1.14.0
termcolor 1.1.0
toml 0.10.2
treys 0.1.3
typed-ast 1.4.1
urllib3 1.26.2
Werkzeug 1.0.1
wheel 0.36.0
wrapt 1.12.1
zipp 3.4.0