Error implementing ppo_4x4grid.py code
Kunal-Kumar-Sahoo opened this issue · 2 comments
Kunal-Kumar-Sahoo commented
I get the following error while executing the code given in experiments/ppo_4x4grid.py. The console output is as follows:
2024-06-05 15:05:36,177 INFO worker.py:1743 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265
2024-06-05 15:05:36,577 INFO tune.py:622 -- [output] This will use the new output engine with verbosity 2. To disable the new output and use the legacy output engine, set the environment variable RAY_AIR_NEW_OUTPUT=0. For more information, please see https://github.com/ray-project/ray/issues/36949
/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/box.py:130: UserWarning: WARN: Box bound precision lowered by casting to float32
gym.logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/utils/passive_env_checker.py:164: UserWarning: WARN: The obs returned by the `reset()` method was expecting numpy array dtype to be float32, actual type: float64
logger.warn(
/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/utils/passive_env_checker.py:188: UserWarning: WARN: The obs returned by the `reset()` method is not within the observation space.
logger.warn(f"{pre} is not within the observation space.")
╭────────────────────────────────────────────────────────╮
│ Configuration for experiment PPO │
├────────────────────────────────────────────────────────┤
│ Search algorithm BasicVariantGenerator │
│ Scheduler FIFOScheduler │
│ Number of trials 1 │
╰────────────────────────────────────────────────────────╯
View detailed results here: /home/rankit/ray_results/PPO
To visualize your results with TensorBoard, run: `tensorboard --logdir /tmp/ray/session_2024-06-05_15-05-34_652990_98780/artifacts/2024-06-05_15-05-36/PPO/driver_artifacts`
Trial status: 1 PENDING
Current time: 2024-06-05 15:05:36. Total running time: 0s
Logical resource usage: 5.0/32 CPUs, 0/1 GPUs (0.0/1.0 accelerator_type:G)
╭────────────────────────────────────╮
│ Trial name status │
├────────────────────────────────────┤
│ PPO_4x4grid_f6ab8_00000 PENDING │
╰────────────────────────────────────╯
(RolloutWorker pid=101043) Retrying in 1 seconds
(RolloutWorker pid=101043) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 18ms, vehicles TOT 0 ACT 0 BUF 0)
(RolloutWorker pid=101043) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101043, ip=10.158.21.84, actor_id=2e663024f15594f45f8c23ae01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x73a20e032d00>)
(RolloutWorker pid=101043) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 407, in __init__
(RolloutWorker pid=101043) self.env = env_creator(copy.deepcopy(self.env_context))
(RolloutWorker pid=101043) File "decentralized_ppo.py", line 29, in <lambda>
(RolloutWorker pid=101043) lambda _: ParallelPettingZooEnv(
(RolloutWorker pid=101043) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 198, in __init__
(RolloutWorker pid=101043) self.observation_space = gym.spaces.Dict(self.par_env.observation_space)
(RolloutWorker pid=101043) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/dict.py", line 84, in __init__
(RolloutWorker pid=101043) assert isinstance(
(RolloutWorker pid=101043) AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'>
2024-06-05 15:05:41,606 ERROR tune_controller.py:1332 -- Trial task failed for trial PPO_4x4grid_f6ab8_00000
Traceback (most recent call last):
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/air/execution/_internal/event_manager.py", line 110, in resolve_future
result = ray.get(future)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
return fn(*args, **kwargs)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/worker.py", line 2667, in get
values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/_private/worker.py", line 866, in get_objects
raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 229, in _setup
self.add_workers(
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 682, in add_workers
raise result.get()
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/utils/actor_manager.py", line 497, in _fetch_result
result = ray.get(r)
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101040, ip=10.158.21.84, actor_id=d3525b9ea91a53ffeae781f501000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x74d31ffd2ca0>)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/rollout_worker.py", line 407, in __init__
self.env = env_creator(copy.deepcopy(self.env_context))
File "decentralized_ppo.py", line 29, in <lambda>
lambda _: ParallelPettingZooEnv(
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/env/wrappers/pettingzoo_env.py", line 198, in __init__
self.observation_space = gym.spaces.Dict(self.par_env.observation_space)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/dict.py", line 84, in __init__
assert isinstance(
AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'>
During handling of the above exception, another exception occurred:
ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/algorithms/algorithm.py", line 533, in __init__
super().__init__(
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/tune/trainable/trainable.py", line 161, in __init__
self.setup(copy.deepcopy(self.config))
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/algorithms/algorithm.py", line 631, in setup
self.workers = WorkerSet(
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 181, in __init__
raise e.args[0].args[2]
AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'>
Trial PPO_4x4grid_f6ab8_00000 errored after 0 iterations at 2024-06-05 15:05:41. Total running time: 4s
Error file: /tmp/ray/session_2024-06-05_15-05-34_652990_98780/artifacts/2024-06-05_15-05-36/PPO/driver_artifacts/PPO_4x4grid_f6ab8_00000_0_2024-06-05_15-05-36/error.txt
2024-06-05 15:05:41,608 WARNING experiment_state.py:205 -- Experiment state snapshotting has been triggered multiple times in the last 5.0 seconds. A snapshot is forced if `CheckpointConfig(num_to_keep)` is set, and a trial has checkpointed >= `num_to_keep` times since the last snapshot.
You may want to consider increasing the `CheckpointConfig(num_to_keep)` or decreasing the frequency of saving checkpoints.
You can suppress this error by setting the environment variable TUNE_WARN_EXCESSIVE_EXPERIMENT_CHECKPOINT_SYNC_THRESHOLD_S to a smaller value than the current threshold (5.0).
2024-06-05 15:05:41,608 INFO tune.py:1016 -- Wrote the latest version of all result files and experiment state to '/home/rankit/ray_results/PPO' in 0.0017s.
Trial status: 1 ERROR
Current time: 2024-06-05 15:05:41. Total running time: 4s
Logical resource usage: 0/32 CPUs, 0/1 GPUs (0.0/1.0 accelerator_type:G)
╭────────────────────────────────────╮
│ Trial name status │
├────────────────────────────────────┤
│ PPO_4x4grid_f6ab8_00000 ERROR │
╰────────────────────────────────────╯
Number of errored trials: 1
╭───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ Trial name # failures error file │
├───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ PPO_4x4grid_f6ab8_00000 1 /tmp/ray/session_2024-06-05_15-05-34_652990_98780/artifacts/2024-06-05_15-05-36/PPO/driver_artifacts/PPO_4x4grid_f6ab8_00000_0_2024-06-05_15-05-36/error.txt │
╰───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
Traceback (most recent call last):
File "decentralized_ppo.py", line 62, in <module>
tune.run(
File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/tune/tune.py", line 1042, in run
raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_4x4grid_f6ab8_00000])
(PPO pid=100933) 2024-06-05 15:05:41,597 ERROR actor_manager.py:517 -- Ray error, taking actor 1 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101040, ip=10.158.21.84, actor_id=d3525b9ea91a53ffeae781f501000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x74d31ffd2ca0>)
(PPO pid=100933) 2024-06-05 15:05:41,598 ERROR actor_manager.py:517 -- Ray error, taking actor 2 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101041, ip=10.158.21.84, actor_id=ba1dbac0fd1f53db94adb0fa01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x73e99a674be0>)
(PPO pid=100933) 2024-06-05 15:05:41,598 ERROR actor_manager.py:517 -- Ray error, taking actor 3 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101042, ip=10.158.21.84, actor_id=62441173fc95b5d7f98b65c401000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x727dee6acbe0>)
(PPO pid=100933) 2024-06-05 15:05:41,598 ERROR actor_manager.py:517 -- Ray error, taking actor 4 out of service. The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101043, ip=10.158.21.84, actor_id=2e663024f15594f45f8c23ae01000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x73a20e032d00>)
(PPO pid=100933) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
(PPO pid=100933) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 229, in _setup
(PPO pid=100933) self.add_workers(
(PPO pid=100933) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/evaluation/worker_set.py", line 682, in add_workers
(PPO pid=100933) raise result.get()
(PPO pid=100933) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/utils/actor_manager.py", line 497, in _fetch_result
(PPO pid=100933) result = ray.get(r)
(PPO pid=100933) ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101040, ip=10.158.21.84, actor_id=d3525b9ea91a53ffeae781f501000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x74d31ffd2ca0>)
(PPO pid=100933)
(PPO pid=100933) During handling of the above exception, another exception occurred:
(PPO pid=100933)
(PPO pid=100933) ray::PPO.__init__() (pid=100933, ip=10.158.21.84, actor_id=4096c08740cb2f2db594866301000000, repr=PPO)
(PPO pid=100933) super().__init__(
(PPO pid=100933) self.setup(copy.deepcopy(self.config))
(PPO pid=100933) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/ray/rllib/algorithms/algorithm.py", line 631, in setup
(PPO pid=100933) self.workers = WorkerSet(
(PPO pid=100933) raise e.args[0].args[2]
(RolloutWorker pid=101043) Exception ignored in: <function SumoEnvironment.__del__ at 0x73a20db78280>
(RolloutWorker pid=101043) Traceback (most recent call last):
(RolloutWorker pid=101043) File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 460, in __del__
(RolloutWorker pid=101043) self.close()
(RolloutWorker pid=101043) File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 450, in close
(RolloutWorker pid=101043) traci.close()
(RolloutWorker pid=101043) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/main.py", line 262, in close
(RolloutWorker pid=101043) connection.check().close(wait)
(RolloutWorker pid=101043) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 396, in close
(RolloutWorker pid=101043) self._sendCmd(tc.CMD_CLOSE, None, None)
(RolloutWorker pid=101043) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 231, in _sendCmd
(RolloutWorker pid=101043) return self._sendExact()
(RolloutWorker pid=101043) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 137, in _sendExact
(RolloutWorker pid=101043) raise FatalTraCIError("Connection closed by SUMO.")
(RolloutWorker pid=101043) traci.exceptions.FatalTraCIError: Connection closed by SUMO.
(RolloutWorker pid=101042) Retrying in 1 seconds [repeated 7x across cluster] (Ray deduplicates logs by default. Set RAY_DEDUP_LOGS=0 to disable log deduplication, or see https://docs.ray.io/en/master/ray-observability/ray-logging.html#log-deduplication for more options.)
(RolloutWorker pid=101042) Step #0.00 (0ms ?*RT. ?UPS, TraCI: 17ms, vehicles TOT 0 ACT 0 BUF 0) [repeated 3x across cluster]
(RolloutWorker pid=101042) Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=101042, ip=10.158.21.84, actor_id=62441173fc95b5d7f98b65c401000000, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x727dee6acbe0>) [repeated 3x across cluster]
(RolloutWorker pid=101042) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/gymnasium/spaces/dict.py", line 84, in __init__ [repeated 27x across cluster]
(RolloutWorker pid=101042) self.env = env_creator(copy.deepcopy(self.env_context)) [repeated 8x across cluster]
(RolloutWorker pid=101042) File "decentralized_ppo.py", line 29, in <lambda> [repeated 8x across cluster]
(RolloutWorker pid=101042) lambda _: ParallelPettingZooEnv( [repeated 8x across cluster]
(RolloutWorker pid=101042) self.observation_space = gym.spaces.Dict(self.par_env.observation_space) [repeated 8x across cluster]
(RolloutWorker pid=101042) assert isinstance( [repeated 8x across cluster]
(RolloutWorker pid=101042) AssertionError: Unexpected Dict space input, expecting dict, OrderedDict or Sequence, actual type: <class 'method'> [repeated 9x across cluster]
(RolloutWorker pid=101042) Exception ignored in: <function SumoEnvironment.__del__ at 0x727dee1de280> [repeated 3x across cluster]
(RolloutWorker pid=101042) Traceback (most recent call last): [repeated 3x across cluster]
(RolloutWorker pid=101042) File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 460, in __del__ [repeated 3x across cluster]
(RolloutWorker pid=101042) self.close() [repeated 3x across cluster]
(RolloutWorker pid=101042) File "/home/rankit/marl/src/sumo-rl/sumo_rl/environment/env.py", line 450, in close [repeated 3x across cluster]
(RolloutWorker pid=101042) traci.close() [repeated 3x across cluster]
(RolloutWorker pid=101042) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 396, in close [repeated 6x across cluster]
(RolloutWorker pid=101042) connection.check().close(wait) [repeated 3x across cluster]
(RolloutWorker pid=101042) self._sendCmd(tc.CMD_CLOSE, None, None) [repeated 3x across cluster]
(RolloutWorker pid=101042) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 231, in _sendCmd [repeated 3x across cluster]
(RolloutWorker pid=101042) return self._sendExact() [repeated 3x across cluster]
(RolloutWorker pid=101042) File "/home/rankit/miniconda3/envs/sumo/lib/python3.8/site-packages/traci/connection.py", line 137, in _sendExact [repeated 3x across cluster]
(RolloutWorker pid=101042) raise FatalTraCIError("Connection closed by SUMO.") [repeated 3x across cluster]
(RolloutWorker pid=101042) traci.exceptions.FatalTraCIError: Connection closed by SUMO. [repeated 3x across cluster]
lvyx21 commented
I have the same error. Did you solve it?
VeigarJia commented
I have the same error.