Missing parameter "format" in GenerateCompletionRequest
Puresoft opened this issue · 4 comments
Hi, I'm missing the format parameter for JSON mode in the GenerateCompletionRequest class.
DOC: https://github.com/ollama/ollama/blob/main/docs/api.md
I'm using JSON mode for long running request, 1 request will take up to 5 minutes.
Is there a way to set the timeout?
You mean this one?
format: the format to return a response in. Currently the only accepted value is json
As it does not support any other values than json it is not part of the model.
As for the timeout, you should be able to simply cancel any operation with a CancellationToken.
timeout
I mean, my request take long by design, I'm not using ollama as a chatbot.
Your http request times out before the request has finished and I want to extend the timeout.
I solved it for now by passing my own instance of HttpClient to the OllamaApiClient constructor.
JSON mode
Yes, but it is an optional parameter and you have to provide it if you want to use it right?
Enable JSON mode by setting the format parameter to json. This will structure the response as a valid JSON object. See the JSON >mode example below.
When format is set to json, the output will always be a well-formed JSON object. It's important to also instruct the model to >respond in JSON.
timeout
You can preconfigure your HttpClient as you wish and pass it as constructor argument to the OllamaApiClient. Don't forget to set the BaseAddress on it if doing so.
JSON mode
Understood. Might implement this.