truera/trulens

Errors due do not exctly matching strings in LLMprovider class

Closed this issue · 1 comments

In line 157 of the base.py file in the providerst folder (

line.index("Supporting Evidence:") +
) the line.index command does not use the exact same string as the if statement in L155/L144 (One has a ":", one doesnt). I try to use a local huggingface llm pipeline with llama_index and when trying to apply a feedbackfunction ("relevance_with_cot_reasoning") it crashes at this point with the error "ValueError: substring not found".
Not sure if thats the only cause for that error but I think using the same string for the index command and the if statement will avoid possible unwanted behavior.

@munila - thanks! This was resolved in trulens_eval==0.23.0