Error Loading fbgemm.dll with PyTorch on Windows: OSError [WinError 126]

[ltkien2003]

I encountered an error while running the local-gemma command with the --model 2b --preset speed options. The error message is as follows:

The error indicates that the fbgemm.dll file, or one of its dependencies, could not be found or loaded by the system.
This issue seems to be related to the PyTorch library, as the missing DLL is part of the torch package.
Possible Causes:

Missing DLL: The fbgemm.dll file might be missing or not properly installed.
Incompatibility: There might be a compatibility issue between the version of PyTorch and the system (e.g., Python version, Windows version, or GPU/CUDA configuration).
Missing Dependencies: Some dependencies required by fbgemm.dll might be missing.
Request for Assistance:

Could you please advise on how to resolve this issue? Specifically, I would appreciate guidance on whether I need to reinstall PyTorch, check for specific dependencies, or make any system adjustments to ensure compatibility.

[gante]

Hi @ltkien2003 👋

That sounds indeed like a torch issue, the stack trace errors out loading torch.

The solution would probably be to locate the fix on the torch side, update your python environment/local packages according to the fix, and then it should work :)

If you're using pipx to install the package, you main need to inject the fix