Replace validators/converters with retry_strategy
jakethekoenig opened this issue · 2 comments
Currently get_response supports converters and validators as ways to coerce model response and retry it it's bad. But we want to support more complicated behaviors such as:
- increasing the temperature on failure
- switching to a bigger model on failure
- showing the model it's output so it can understand why it failed and self correct
To support these let's make get_response accept a retry_strategy argument which is of abstract class RetryStrategy which should have one function decide (name TBD, feel free to choose the name of you have a good idea) which accepts the call_args the number of the previous attempt (0 indexed) and the model output as text. It then returns a tuple, (behavior, next_call_args, result) where behavior is a new enum Behavior which can be either RETRY or RETURN. If it is set to Return the text and result are added to the spice response and returned. Otherwise we try again with next_call_args. It's up to the strategy to throw an exception or otherwise define a failure case after a certain number of iterations.
This will deprecate the current converter/validator/retries arguments. We should support them for now and if passed create a RetryStrategy called Default which reproduces the current behavior.
@MentatBot can you do this?
I will start working on this issue