Error in unibench list_benchmarks: ValueError when converting 'zero-shot' to float
jun297 opened this issue · 3 comments
(unibench) [jun297@server1 unibench]$ unibench list_benchmarks
There are 80 models to evaluate
There are 53 benchmarks to evaluate
Downloading only aggregate results.../home/jun297/.cache/unibench/outputs
/home/jun297/micromamba/envs/unibench/lib/python3.10/site-packages/huggingface_hub/file_download.py:1212: UserWarning: local_dir_use_symlinks parameter is deprecated and will be ignored. The process to download files to a local folder has been updated and do not rely on symlinks anymore. You only need to pass a destination folder aslocal_dir.
For more details, check out https://huggingface.co/docs/huggingface_hub/main/en/guides/download#download-files-to-local-folder.
warnings.warn(
Traceback (most recent call last):
File "/home/jun297/micromamba/envs/unibench/lib/python3.10/site-packages/unibench/evaluator.py", line 194, in list_benchmarks
df_to_table(
File "/home/jun297/micromamba/envs/unibench/lib/python3.10/site-packages/unibench/common_utils/utils.py", line 50, in df_to_table
row += [str(round(float(x), 2)) for x in value_list]
File "/home/jun297/micromamba/envs/unibench/lib/python3.10/site-packages/unibench/common_utils/utils.py", line 50, in
row += [str(round(float(x), 2)) for x in value_list]
ValueError: could not convert string to float: 'zero-shot'
Hi, it should be fixed in v0.3.0, can you try to reinstall and let us know if it works?