Add stop sequence parameter for controlled generation

Question

Add stop sequence parameter for controlled generation

Closed this issue 5 months ago · 4 comments

Description:

It would be valuable to include a stop sequence parameter in the API client to facilitate controlled generation termination.

Reference Implementation:
Referencing OpenAI's implementation, please check: OpenAI API Reference - Chat Create Stop

Use Case:
This feature becomes crucial when working with agents, allowing users to halt generation at precise steps

discord topic

Answer 1 · 2024-01-21T20:02:41.000Z

I second this. It's much needed.

Answer 2 · 2024-02-22T12:58:09.000Z

Adding to this. For now that limits my usage of Mistral API. I have tons of requests to do with very short yes/no answers. For now local hosting is my only solution sadly (and I can't try medium on my workload).

Answer 3 · 2024-07-08T07:29:23.000Z

Any updates?

Answer 4 · 2024-08-12T07:57:23.000Z

Stop parameter added. Closing the issue.