implement streaming in miner and validator
GuiBibeau opened this issue · 0 comments
GuiBibeau commented
Currentl calling the chat endpoint on validators results in a 5-10 second delay since the LLM workload is slow. We should implement streaming methods that are equivalent in both the miner and the validator to let them act closer to the open AI API