jackmpcollins/magentic

Support `stop` sequences

mnicstruwig opened this issue · 0 comments

Hi @jackmpcollins,

Do you think it would be possible at all to implement stop sequences (docs)?

These are incredibly useful when doing chain-of-thought style agentic prompting (eg. ReAct), where you want the model to stop generating at specific stop sequences. I have a hackathon project here that makes use of this pattern with this prompt, that utilizes a forked version of magentic with a hacked and very basic implementation of stop sequences on a branch here for the non-async variant.

Hopefully this example demonstrates how powerful this can be: Using stop sequences, we can implement a ReAct agent with custom error handling, fault correction + more using a very thin abstraction layer. This essentially "unlocks" custom agent design using magentic, and is loads simpler than doing the same thing using langchain.

I'm more than happy to help on collaborating on this, but will need a bit of guidance on your end for the best way of implementing this.

Thanks again for a wonderful library!

All the best,
Michael.