Have Deployments Call Extend API When No Gateway

[NarekA]

Describe your proposal/problem

If you are using standalone deployments as we are, there is no way to extend the REST API. Below I have a workaround, but it would be useful to have jina.serve.runtimes.worker.http_fastapi_app.get_fastapi_app call extend_rest_interface. We have some endpoints that do not fit into the jina model, and would like to serve them in the same deployment.

Current Workaround

For anyone looking for a way to extend deployments currently, this seems to work:

import wrapt
from pydantic import BaseModel


class MyInput(BaseModel):
    input: str

def extend_rest_function(app):

    @app.post(
        "/api/openai/v1/chat/completions",
        tags=["OpenAI Interface"],
    )
    async def post(
        input: MyInput
    ):
        return {"output": "hello world"}

    return app



def extend_fastapi_wrapper(wrapped, instance, args, kwargs):
    app = wrapped(*args, **kwargs)
    return extend_rest_function(app)



wrapt.wrap_function_wrapper(
    "jina.serve.runtimes.worker.http_fastapi_app",
    'get_fastapi_app',
    extend_fastapi_wrapper
)



from jina import Executor, requests, Deployment
from docarray import DocList
from docarray.documents import TextDoc


class MyExecutor(Executor):
    @requests(on='/foo')
    def foo(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        docs[0].text = 'hello, world!'
        docs[1].text = 'goodbye, world!'
        return docs

with Deployment(
    uses=MyExecutor
) as dep:
    dep.block()

[JoanFM]

Hey @NarekA,

thanks for reporting the issue.

I have a concern about this feature, and is its cloud nativeness. I am not sure how it would play with all the cloud nativeness we have. if you use one of these, then you kind of lose the support we have in our orchestration layer.

What do you think?

[NarekA]

@JoanFM We are using Jina as web framework. We would like to use the orchestration layer, but at the moment it is not compatible with our internal infrastructure. At times Jina is restrictive as a web framework, so being able to access FastAPI directly makes it much more feasible in the long term.