Cannot use MPS with torch multiprocessing share_memory

[swswsws583]

Hi, I am not sure how to set the pytorch mps fallback setting on Mac. My default setting is: TORCH_DEVICE: Optional[str] = None and I got this error:

Loaded detection model vikp/surya_det2 on device cpu with dtype torch.float32
Loaded detection model vikp/surya_layout2 on device cpu with dtype torch.float32
Loaded reading order model vikp/surya_order on device mps with dtype torch.float16
Loaded recognition model vikp/surya_rec on device mps with dtype torch.float16
Loaded texify model to mps with torch.float16 dtype
Traceback (most recent call last):
  File "/opt/homebrew/bin/marker", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/convert.py", line 116, in main
    raise ValueError("Cannot use MPS with torch multiprocessing share_memory.  You have to use CUDA or CPU.  Set the TORCH_DEVICE environment variable to change the device.")
ValueError: Cannot use MPS with torch multiprocessing share_memory.  You have to use CUDA or CPU.  Set the TORCH_DEVICE environment variable to change the device.

I tried changing to TORCH_DEVICE: Optional[str] = cpu and TORCH_DEVICE: Optional[str] = mps and it didn't work. my command was something like this: marker /path/to/input/folder /path/to/output/folder --workers 1.

[VikParuchuri]

You have to set it as an environment variable, like TORCH_DEVICE=mps marker /path/to/input/folder /path/to/output/folder --workers 1

[swswsws583]

The same error showed up after I set the environment variable TORCH_DEVICE=mps before marker as you mentioned. I was able to get marker_single working without setting this environment variable before. I wonder if it's something to do with the marker command?

Loaded detection model vikp/surya_det2 on device cpu with dtype torch.float32
Loaded detection model vikp/surya_layout2 on device cpu with dtype torch.float32
Loaded reading order model vikp/surya_order on device mps with dtype torch.float16
Loaded recognition model vikp/surya_rec on device mps with dtype torch.float16
Loaded texify model to mps with torch.float16 dtype
Traceback (most recent call last):
  File "/opt/homebrew/bin/marker", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/convert.py", line 116, in main
    raise ValueError("Cannot use MPS with torch multiprocessing share_memory.  You have to use CUDA or CPU.  Set the TORCH_DEVICE environment variable to change the device.")
ValueError: Cannot use MPS with torch multiprocessing share_memory.  You have to use CUDA or CPU.  Set the TORCH_DEVICE environment variable to change the device.

[VikParuchuri]

Yes, I wrote mps in the command, but it was just an example, you actually have to set TORCH_DEVICE to cpu . This is specific to the multiprocessing in the marker command. If you only need one process, use marker_single