layer 'gaussian_policy' error

[Ishihara-Masabumi]

When I run Pendulum-v0, the following error occurred.

$ python3 main.py 
2023-02-27 11:48:28.296204: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-27 11:48:28.417605: I tensorflow/core/util/port.cc:104] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2023-02-27 11:48:28.937501: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dl/.local/lib/python3.8/site-packages/nvidia/cublas/lib:/usr/local/cuda/lib64:
2023-02-27 11:48:28.937565: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/dl/.local/lib/python3.8/site-packages/nvidia/cublas/lib:/usr/local/cuda/lib64:
2023-02-27 11:48:28.937574: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-02-27 11:48:30.682323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1613] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 21978 MB memory:  -> device: 0, name: NVIDIA GeForce RTX 3090, pci bus id: 0000:17:00.0, compute capability: 8.6
/home/dl/RL/RL/lib/python3.8/site-packages/gym/core.py:317: DeprecationWarning: WARN: Initializing wrapper in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
  deprecation(
/home/dl/RL/RL/lib/python3.8/site-packages/gym/wrappers/step_api_compatibility.py:39: DeprecationWarning: WARN: Initializing environment in old step API which returns one bool instead of two. It is recommended to set `new_step_api=True` to use new step API. This will be the default behaviour in future.
  deprecation(
WARNING:absl:`lr` is deprecated, please use `learning_rate` instead, or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
WARNING:absl:`lr` is deprecated, please use `learning_rate` instead, or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
WARNING:absl:`lr` is deprecated, please use `learning_rate` instead, or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
Traceback (most recent call last):
  File "main.py", line 259, in <module>
    main(n_episodes=150, n_testplay=4)
  File "main.py", line 228, in main
    agent = SAC(env_id="Pendulum-v1", action_space=1, action_bound=2)
  File "main.py", line 54, in __init__
    self._initialize_weights()
  File "main.py", line 68, in _initialize_weights
    self.policy(dummy_state)
  File "/home/dl/RL/RL/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/tmp/__autograph_generated_fileuvo38ycg.py", line 10, in tf__call
    x = ag__.converted_call(ag__.ld(self).dense1, (ag__.ld(x),), None, fscope)
ValueError: Exception encountered when calling layer 'gaussian_policy' (type GaussianPolicy).

in user code:

    File "/home/dl/RL/deep_reinforcement_learning_gallery/SAC/Pendulum-v0/models.py", line 31, in call  *
        x = self.dense1(x)
    File "/home/dl/RL/RL/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 70, in error_handler  **
        raise e.with_traceback(filtered_tb) from None
    File "/home/dl/RL/RL/lib/python3.8/site-packages/keras/engine/input_spec.py", line 250, in assert_input_compatibility
        raise ValueError(

    ValueError: Input 0 of layer "dense" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (3,)


Call arguments received by layer 'gaussian_policy' (type GaussianPolicy):
  • x=tf.Tensor(shape=(3,), dtype=float32)

[horoiwa]

This would be shape error, gaussian policy requires matrix of (batch_dim, obs_dim), but your input seems to be (obs_dim, ).
Please make sure your code includes reshaping ops; dummy_state = (dummy_state[np.newaxis, ...]).astype(np.float32)

[Ishihara-Masabumi]

OK, thanks.