Stream option

Question

Stream option

rasbt opened this issue 4 months ago · 3 comments

Add a stream=True option to generate

Answer 1 · 2024-05-15T16:45:11.000Z

Hey @rasbt, are you looking for contributions to this enhancement? If so, can you share your ideas on where and how to start with streaming so I can contribute?

Answer 2 · 2024-05-16T11:55:41.000Z

Thanks for your interest in this. This was requested by a colleague for a downstream application, but I haven't had a chance to think this through in terms of how this would look like (am currently traveling and haven't had a chance to follow up). I added it as a note for now but will probably have to implement it in tandem with the application.

Answer 3 · 2024-05-17T13:24:41.000Z

@rasbt Got it, thank you.