Help with: No such file or directory: '/fsx/hailey/math-lm/gpt-neox/megatron/fused_kernels'
andrewarrow opened this issue · 5 comments
After python ./megatron/fused_kernels/setup.py install I try pip install -r requirements.txt and get this everytime:
Processing /fsx/hailey/math-lm/gpt-neox/megatron/fused_kernels (from -r requirements.txt (line 28))
ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/fsx/hailey/math-lm/gpt-neox/megatron/fused_kernels'
Thank you so much!
The file requirements.txt doesn't exist in this repo. You should be running pip install -r requirements/requirements.txt
I'm actually trying to get:
https://github.com/EleutherAI/math-lm
working which uses this version of gpt-neox
https://github.com/EleutherAI/gpt-neox/tree/5dd366539803dbf1fd725cc057013fd002a4cfd4
But thank you I will try installing normal gpt-neox and see where that gets me!
I'm actually trying to get:
https://github.com/EleutherAI/math-lm
working which uses this version of gpt-neox
https://github.com/EleutherAI/gpt-neox/tree/5dd366539803dbf1fd725cc057013fd002a4cfd4
But thank you I will try installing normal gpt-neox and see where that gets me!
It looks like there's a line that erroneously points to @haileyschoelkopf's personal directory on one of our computing clusters. Try removing the /fsx/hailey/math-lm/ from the line and rerunning it.
If using that Llemma branch, you can simply comment out the fused_kernels line in that requirements.txt and install the fused kernels using megatron/fused_kernels/setup.py install on that commit as you have done already.
@andrewarrow did this solve your problem?