requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
vanshgupta7 opened this issue · 3 comments
Bug Description
RuntimeError: Evaluation of positive_sentiment failed on inputs:
{'text': 'An agreement between a policyholder and insurer wherein the '
'insurance company protects your valuable asset
Expecting value: line 1 column 1 (char 0).
Relevant Logs/Tracebacks
Traceback (most recent call last):
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\feedback.py", line 513, in run
result_and_meta, part_cost = Endpoint.track_all_costs_tally(
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 492, in track_all_costs_tally
result, cbs = Endpoint.track_all_costs(
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 473, in track_all_costs
return Endpoint._track_costs(
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 570, in _track_costs
result: T = __func(*args, **kwargs)
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\hugs.py", line 72, in wrapper
return func(*bindings.args, **bindings.kwargs)
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\hugs.py", line 215, in positive_sentiment
hf_response = self.endpoint.post(
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 247, in post
j = ret.json()
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\requests\models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\feedback.py", line 519, in run
raise RuntimeError(
RuntimeError: Evaluation of positive_sentiment failed on inputs:
{'text': 'An agreement between a policyholder and insurer wherein the '
'insurance company protects your valuable asset
Expecting value: line 1 column 1 (char 0).
Feedback Function exception caught: Traceback (most recent call last):
File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\requests\models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\Vansh.Gupta\AppData\Local\Programs\Python\Python310\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\Vansh.Gupta\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Vansh.Gupta\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Environment:
`llama-index==0.10.12
trulens-eval @ 0.24.1
python - 3.10`
Additional context
I am trying to use llama_index and huggingface provider to evaluate Palm model response on a RAG that I built using PineCone Vector Store. I am able to successfully get an output and display it but am not able to evaluate the model.
Before the Llama_Index 0.10 update it was working successfully but after upgrading it , it is not working.
The Feedback functions I am using are Toxic, Positive-Sentiment and groundedness_with_nli.
Hey @vanshgupta7 - do you have a contained example you can share?
It was a HuggingFace API error.. it got resolved... sorry for the trouble
No worries - thanks for the update @vanshgupta7 !