Getting started example doesn't work - oieval attempts to update a None type object
jswang opened this issue · 1 comments
jswang commented
Describe the bug
When running the example provided in https://github.com/openai/evals/blob/main/docs/run-evals.md: oaieval gpt-3.5-turbo test-match
, the following error is encountered out of the box:
File "/Users/juliewang/Documents/evals/evals/cli/oaieval.py", line 167, in run
eval_spec.args.update(extra_eval_params)
^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'update'
This is due to this line: https://github.com/openai/evals/blob/main/evals/cli/oaieval.py#L158. The args
field of EvalSpec
is optional:
Line 58 in 0dc0ba4
test-match
case, eval_spec.args
is None
.
To Reproduce
- Run
oaieval gpt-3.5-turbo test-match
Code snippets
No response
OS
macOS
Python version
Python 3.12.2
Library version
openai-evals commit b5853eb
thesofakillers commented
Thx for raising this! I've opened a PR that should address it.