Perform Inference using Python instead of command line

Question

Perform Inference using Python instead of command line

vashiegaran opened this issue a year ago · 1 comments

Hi, is there any example of python code that can run inference instead of using the command line? If so can you kindly share

Answer 1 · 2024-01-14T07:18:05.000Z

Thanks for your interest in LMFlow! You may check speculative_inference.py, where you can replace SpeculativeInferencer with Inferencer (along with their arguments) in examples/chatbot.py. Please feel free to contact us again if there is any problem during the usage. Thanks very much 😄