Evaluation with a container stops halfway without error message
Closed this issue · 4 comments
Hi, thanks for providing the nice multi-lingual evaluation framework.
The generation went smoothly. When it comes to evaluation, I used my mac to run evaluation w/ a container (following this tutorial). The execution stops at here 53%|█████▎ | 504/955 [00:09<00:08, 53.40it/s]. Is there a good way to tell what went wrong? FYI, the following is the anonymized debug log of podman:
DEBU[0000] DoRequest Method: GET URI: http://d/v4.8.1/libpod/_ping
DEBU[0000] DoRequest Method: GET URI: http://d/v4.8.1/libpod/networks/pasta/exists
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf"
DEBU[0000] DoRequest Method: POST URI: http://d/v4.8.1/libpod/images/pull
DEBU[0000] User or group ID mappings not available: open /proc/self/uid_map: no such file or directory
DEBU[0000] User or group ID mappings not available: open /proc/self/uid_map: no such file or directory
DEBU[0000] User mount xxx:/xxx options [rw]
DEBU[0000] DoRequest Method: GET URI: http://d/v4.8.1/libpod/images/multipl-e-eval/json
DEBU[0000] DoRequest Method: POST URI: http://d/v4.8.1/libpod/containers/create
DEBU[0000] Enabling signal proxying
DEBU[0000] Enabling signal proxying
DEBU[0000] DoRequest Method: GET URI: http://d/v4.8.1/libpod/containers/41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d/json
DEBU[0000] DoRequest Method: POST URI: http://d/v4.8.1/libpod/containers/41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d/attach
DEBU[0000] Copying standard streams of container "41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d" in non-terminal mode
DEBU[0000] DoRequest Method: POST URI: http://d/v4.8.1/libpod/containers/41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d/start
53%|█████▎ | 504/955 [00:09<00:08, 55.18it/s]DEBU[0017] DoRequest Method: POST URI: http://d/v4.8.1/libpod/containers/41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d/wait
DEBU[0019] DoRequest Method: POST URI: http://d/v4.8.1/libpod/containers/41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d/shouldrestart
DEBU[0019] DoRequest Method: DELETE URI: http://d/v4.8.1/libpod/containers/41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d
DEBU[0019] Container 41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d does not exist: no container with ID or name "41237923cc99a526b678300f112a4f2cb5d2feed497078ed0a8a6086cbea635d" found: no such container
As my ubuntu server doesn't allow me to easily run docker/podman, to walk-around,
--> I tried to run "evaluation w/o a container", this did finish but some scores of some PL are 0 because I didn't install certain library.
--> Could you provide a installation script for properly preparing the envs, ensuring a consistent evaluation w & w/o a container?
Let me know if you need more info. Thanks in advance!
Which language are you trying to evaluate?
Thanks for the question. I am evaluating against multiple languages: {cpp, java, php, ts, cs, sh}. I checked that the podman evaluation on first 5 languages can be run without error.
The culprit seems to bash language. The eval of sh will be stopped halfway.
Well I find the evaluation container works fine with kubernetes (ubuntu). So I guess it might be a mac issue. I will close the issue.
Yeah, sorry. I use a Mac too, but I can't guarantee that this evaluation container will work ok on a Mac.