Creating a New Project with a bad key results in null values
Opened this issue · 1 comments
Describe the bug
After creating a New Project with a bad key for OpenAI GPT-3.5-Turbo, the results are returned as Null. I was expecting to get a bad key response or error. Perhaps rather than"null", the error code or reason is what is captured from the request? Either an invalid_api_key or if/when rate limits are hit a 429?
To Reproduce
Steps to reproduce the behavior:
- Create a New Project and fill in all the fields and add a jsonl file for your dataset
- Select an OpenAI LLM for Eval
- Enter in the key
sk
(Should there be a key-length check here and warning?) - Click Next
- Select any/all metrics
- Click Finish
- The Evaluations screen shows "null" for your metric score
Should scores be null based on a key failure? Should there be an error reason shown? Null is vague. Is my key wrong or is my system?
Expected behavior
If a bad/expired key is used, the evaluation to reflect that there is a key problem and not present "null"
Screenshots
If applicable, add screenshots to help explain your problem.
Seeing null for a bad key is confusing. Where is the error actually happening?
Additional context
Entering in a bad key or having an expired key might lead to confusion when "null" starts to present itself. Should an error code or reason be presented instead?
Thanks for raising this. This is currently in our pipeline, and should be fixed in the next dashboard release.