Trouble with A3C and Breakout
Closed this issue · 1 comments
Hi, I was trying to use a3c with the game 'breakout', but some error popped:
my command : python3 main.py --type A3C --env BreakoutNoFrameskip-v4 --is_atari --nb_episodes 10000 --n_threads 4
the result:
Score: 0%| | 0/10000 [00:00<?, ? episodes/s]Exception in thread Thread-3:
Traceback (most recent call last):
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread
a = agent.policy_action(np.expand_dims(old_state, axis=0))
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action
return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0]
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict
return self.model.predict(self.reshape(inp))
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict
check_batch_axis=False)
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)
Exception in thread Thread-4:
Traceback (most recent call last):
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread
a = agent.policy_action(np.expand_dims(old_state, axis=0))
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action
return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0]
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict
return self.model.predict(self.reshape(inp))
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict
check_batch_axis=False)
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)
Exception in thread Thread-5:
Traceback (most recent call last):
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread
a = agent.policy_action(np.expand_dims(old_state, axis=0))
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action
return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0]
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict
return self.model.predict(self.reshape(inp))
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict
check_batch_axis=False)
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)
Exception in thread Thread-6:
Traceback (most recent call last):
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/thread.py", line 24, in training_thread
a = agent.policy_action(np.expand_dims(old_state, axis=0))
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action
return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0]
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict
return self.model.predict(self.reshape(inp))
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict
check_batch_axis=False)
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)
Traceback (most recent call last):
File "main.py", line 115, in
main()
File "main.py", line 108, in main
a = algo.policy_action(old_state)
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/a3c.py", line 58, in policy_action
return np.random.choice(np.arange(self.act_dim), 1, p=self.actor.predict(s).ravel())[0]
File "/home/kexin/Desktop/CHRISTOPHE/Deep-RL-Keras-master/A3C/agent.py", line 22, in predict
return self.model.predict(self.reshape(inp))
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 1817, in predict
check_batch_axis=False)
File "/home/kexin/anaconda3/envs/tensorflow_gpuenv/lib/python3.6/site-packages/keras/engine/training.py", line 113, in _standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking : expected input_1 to have 5 dimensions, but got array with shape (1, 84, 84, 4)
Thank you!
Hi, thanks for reporting the error,
There was indeed an error on the input shape when using Atari games on A3C, due to the number of consecutive frames. Just updated the code, it should work better now!