PrefectHQ/ControlFlow

All the user to specify the `stream` argument of `Agent._run_model` somewhere

Opened this issue · 0 comments

Enhancement Description

The default orchestrator calls Agent._run_model without a streaming argument: https://github.com/PrefectHQ/ControlFlow/blob/main/src/controlflow/orchestration/orchestrator.py#L375.

In effect, the default of True is always used unless the user constructs a new Orchestrator.

Use Case

Users may not be able to use a specific LLM API in streaming mode, but may have authorization to it otherwise.

Proposed Implementation

A stream parameter can be passed to the agent at construction and set on the object, and then Agent._run_model function could then use then use that value. Arguably whether or not an agent should be used in streaming mode is an Agent-level decision rather than a call-level decision, so it may make sense to specify it at construction rather than in the function that triggers that LLM call.

https://github.com/PrefectHQ/ControlFlow/blob/main/src/controlflow/agents/agent.py#L284