Run Gym.CartPole-v1
Closed this issue · 6 comments
Hi,
I am trying to run gpu-implementation es.py using gym enviroment gym.CartPole-v1
I edited the configuration file es_atari_config.json by changing "frostbrite" to "gym.CartPole-V1" and encounter error below
2018-08-18 12:09:54.669655: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/device:GPU:0 with 7396 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
08/18/2018 12:09:54 PM {
"episode_cutoff_mode": 5000,
"game": "gym.CartPole-v1",
"l2coeff": 0.005,
"model": "ModelVirtualBN",
"mutation_power": 0.02,
"num_test_episodes": 200,
"num_validation_episodes": 30,
"optimizer": {
"args": {
"stepsize": 0.01
},
"type": "adam"
},
"population_size": 5000,
"return_proc_mode": "centered_rank",
"timesteps": 250000000.0
}
08/18/2018 12:09:54 PM Logging to: /tmp/tmp59hdhjso
2018-08-18 12:09:54.745411: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1435] Adding visible gpu devices: 0
2018-08-18 12:09:54.745453: I tensorflow/core/common_runtime/gpu/gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-08-18 12:09:54.745472: I tensorflow/core/common_runtime/gpu/gpu_device.cc:929] 0
2018-08-18 12:09:54.745476: I tensorflow/core/common_runtime/gpu/gpu_device.cc:942] 0: N
2018-08-18 12:09:54.745588: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 7396 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
WARN: gym.spaces.Box autodetected dtype as <class 'numpy.float32'>. Please provide explicit dtype.
Traceback (most recent call last):
File "es.py", line 293, in <module>
main(**exp)
File "es.py", line 148, in main
worker = ConcurrentWorkers(make_env, Model, batch_size=64)
File "/home/Desktop/deepn/deep-neuroevolution/gpu_implementation/neuroevolution/concurrent_worker.py", line 135, in __init__
ref_batch = gym_tensorflow.get_ref_batch(make_env_f, sess, 128)
File "/home/Desktop/deepn/deep-neuroevolution/gpu_implementation/gym_tensorflow/__init__.py", line 19, in get_ref_batch
assert env.discrete_action
AssertionError
I notice that the GymEnv in tf_env.py having property of discrete_action that will only return False which is triggering the assert in the get_ref_batch
class GymEnv(PythonEnv): ... @property def discrete_action(self): return False
Is openai gym's enviroment ready for run?
The code only supports discrete action environments currently. Please use the non-gpu version for continuous action spaces.
Based on my understanding, openai gym's CartPole-v0 is a discrete action environment.
https://github.com/openai/gym/wiki/CartPole-v0
Please correct me if I am wrong.
@ieow you are correct, the Gym wrapper I created was experimental and clearly has some issues.
The appropriate way is probably to check if the gym environment return a spaces.Box or spaces.Discrete. Either way, gym.* environments still needs some work.
For now, you can just modify it to return isinstance(env[0].action_space, gym.spaces.Discrete)
Thanks.
Somehow, I managed to get it running after some modification based on tf_atari.py