SB3 Connect four tutorial segfaults for me with latest libraries

[metamorphy]

Describe the bug

I simply paste in the code from the Training and Evaluation section of "SB3: Action Masked PPO for Connect Four" and run it directly with no changes under Visual Code with the latest stable versions of pythons and libraries and it quickly gets a segmentation fault.

I realize it's your code calling sb3_contrib calling sb3 calling torch, so it won't be easy to address from this unless it happens to you too, but someone else just submitted that this sample is not training properly(#1147) and maybe this bug offers a small clue?

This would be the perfect sample for me because I'm using a custom AECEnv environment with action masking. I don't see any other samples of how to make this work with SB3, which is why I switched to PettingZoo, so I'm stuck at the moment. I'll look at SB3 alteratives if I can find an AEC&action-masking example. Thanks.

Code example

The code I ran is here:
https://pettingzoo.farama.org/tutorials/sb3/connect_four/

It happens on this line of code:
q, r = torch.linalg.qr(flattened)   [torch/nn/init.py]

flattened.shape == torch.Size([84, 64])

My stack crawl:
orthogonal_ (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/torch/nn/init.py:544)
init_weights (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/stable_baselines3/common/policies.py:312)
apply (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/torch/nn/modules/module.py:891)
apply (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/torch/nn/modules/module.py:890)
apply (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/torch/nn/modules/module.py:890)
_build (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/sb3_contrib/common/maskable/policies.py:233)
__init__ (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/sb3_contrib/common/maskable/policies.py:111)
_setup_model (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/sb3_contrib/ppo_mask/ppo_mask.py:145)
__init__ (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/site-packages/sb3_contrib/ppo_mask/ppo_mask.py:137)
train_action_mask (/Users/jeffmallett/dev/python/kings/pz2cb3.py:77)
<module> (/Users/jeffmallett/dev/python/kings/pz2cb3.py:178)
_run_code (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/runpy.py:88)
_run_module_as_main (/Users/jeffmallett/miniconda3/envs/kings/lib/python3.12/runpy.py:198)

System info

My versions:
pettingzoo v1.24.3 (requires>=1.24.0)
stable-baselines3 v2.2.1 (requires >=2.0.0)
sb3-contrib v2.2.1 (requires >=2.0.0)

Running with python v3.12.2 on Macbook Pro OS 14.2.1.

Additional context

No response

Checklist

• I have checked that there is no similar issue in the repo

[dm-ackerman]

Which version of pytorch are you using?

I used torch=2.2.2 with the versions you listed and I am unable to reproduce this.
I'm running on Ubuntu 22.04.4 LTS

[metamorphy]

libtorch 2.1.2 cpu_mkl_h91b7611_103 conda-forge
pytorch 2.1.2 cpu_mkl_py312h5f4a04e_103 conda-forge
torch 2.2.1 pypi_0 pypi

[metamorphy]

Thanks for the hint. I upgraded torch to 2.2.2 and that seems to have fixed it. That is to say, it fixes the torch bug. I now get some version incompatibility between Gymnasium and SB3 with Gymnasium expecting 2 arguments and SB3 passing 4...

class Wrapper(Env[ObsType, ActType]): [gymnasium/core.py]
class StickyActionEnv(gym.Wrapper[np.ndarray, int, np.ndarray, int]): [stable_baselines3/common/atari_wrapper.py]

even though this doesn't upgrade anything... pip install --upgrade gymnasium stable_baselines3 sb3-contrib

[dm-ackerman]

Are you using version 0.29.1 of gymnasium? If not, please try that. There is a version 1.0 of gymnasium but it's still in testing and last I heard, SB3 doesn't fully support it.

If that isn't the issue, you might try posting an issue with SB3 and/or gymnasium. I don't think the connect4 example would be using an atari wrapper, so it doesn't seem like a pettingzoo issue

[metamorphy]

According to conda I'm using 0.29.1, but looking at the release notes and gynasium code in github, somehow I don't think I am. Thanks for your help. Agreed this has nothing to do with pettingzoo.

[elliottower]

I would recommend not using conda gymnasium releases as they are not as well supported (conda install vs pip install)