bentoml/Yatai

How can I deploy bentoDeployment with both remote runner and embedded runner?

KimSoungRyoul opened this issue · 0 comments

As far as I know, this feature is not currently supported, can you please consider this for yatai 2.0?


a_runner = bentoml.xxx.to_runner(embedded=True)
b_runner = bentoml.xxx.to_runner()

svc = bentoml.Service()

@svc.api(...)
async predict(...): 
    a_result = await a_runner.async_run(...) # embedded runner
    b_result = await b_runner.async_run(...) # remote runner ( other pods )