Error in SAC/BipedalWalker-v3 main.py

[Ishihara-Masabumi]

When I run SAC/BipedalWalker-v3/main.py, the following error occurred.

$ python3 main.py 
2023-02-24 11:20:24.047917: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-24 11:20:24.168698: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-02-24 11:20:24.697014: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dl/.local/lib/python3.8/site-packages/nvidia/cublas/lib:/usr/local/cuda/lib64:
2023-02-24 11:20:24.697128: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dl/.local/lib/python3.8/site-packages/nvidia/cublas/lib:/usr/local/cuda/lib64:
2023-02-24 11:20:24.697137: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-02-24 11:20:25.633243: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-24 11:20:26.129306: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21654 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:17:00.0, compute capability: 8.6
WARNING:absl:`lr` is deprecated, please use `learning_rate` instead, or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
WARNING:absl:`lr` is deprecated, please use `learning_rate` instead, or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
WARNING:absl:`lr` is deprecated, please use `learning_rate` instead, or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
Traceback (most recent call last):
  File "main.py", line 263, in <module>
    main(n_episodes=1000, n_testplay=4)
  File "main.py", line 232, in main
    agent = SAC(env_id="BipedalWalker-v3", action_space=4, action_bound=1)
  File "main.py", line 54, in __init__
    self._initialize_weights()
  File "main.py", line 63, in _initialize_weights
    dummy_state = (dummy_state[np.newaxis, ...]).astype(np.float32)
TypeError: tuple indices must be integers or slices, not tuple

I have no idea on this error, so please let me know how to fix it.

[horoiwa]

hi @Ishihara-Masabumi, thank you for reporting.
This error is due to the recent change of gym API. env.reset() in older version returns only np.ndarray, but in the latest version returns tuple of (np.ndarray, {}) . as a consequence, dummy_state[np.newaxis, ...] causes TypeError.

Therefore, there are two solution. One is simply downgrade your gym. I confirmed gym==0.25.2 works fine. Another solutions is changing dummy_state = env.reset() to dummy_state, _ = env.reset().

[Ishihara-Masabumi]

OK, thanks.