Missing file when trying to run evaluation script

[iwinterknight]

Thanks for sharing your code!
I downloaded the datasets and the 7B model ckpt from the download bash script you uploaded. I'm unable to get the run_test_eval.sh to run.

I get the error : [Errno 2] No such file or directory: './output/StructLM-13B/predictions_predict.json'

[kanseaveg]

I have encountered this problem before. Later, I manually modified the code and completed the missing packages in requirements.txt.

Firstly, you should add the following sentence before that one.
os.makedirs(output_path, exist_ok=True)

Next, you should add these two packages to the requirements.txt file and execute pip install for installation.
datasets
megablocks
scipy

Finally, by using the execute script command: ./run_test_eval.sh StructLM-7B , it should be possible to evaluate.
Because my contribution is too small, I won't submit a Pull request.

Although I'm grateful for their contributions, it seems that they haven't provided reproducible training scripts and training tutorials. This might be a regret, perhaps. @iwinterknight

In terms of time, the evaluation of 7B is very fast.

[azhx]

Hi, thanks for the suggestions, I updated the requirements.txt. I'm continuing to test and improve this repo.