towhee-io/towhee

[Bug]: When building the triton backend, using custom operators may result in errors

bug400400 opened this issue · 4 comments

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

@register(name='embedding')
class Embedding(NNOperator):
def init(self):

def __call__(self, object, type, model):
    return vec

embedding_pipe = (
pipe.input('object')
.map('object', 'vec', ops.embedding())
.output('vec')
)

towhee.build_docker_image(
dc_pipeline=embedding_pipe,
image_name='triton_embedding:v1.0',
cuda_version='11.7',
format_priority=['onnx'],
parallelism=4,
inference_server='triton'
)

but it will go to the hub to download the operator
image

Expected Behavior

build success

Steps To Reproduce

@register(name='embedding')
class Embedding(NNOperator):
    def __init__(self):
      ####
    def __call__(self, object, type, model):
        ####
        return vec

embedding_pipe = (
    pipe.input('object')
        .map('object', 'vec', ops.embedding())
        .output('vec')
)

towhee.build_docker_image(
    dc_pipeline=embedding_pipe,
    image_name='triton_embedding:v1.0',
    cuda_version='11.7',
    format_priority=['onnx'],
    parallelism=4,
    inference_server='triton'
)

but it will go to the hub to download the operator

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93):
- OS(Ubuntu or CentOS): Ubuntu 
- CPU/Memory: CPU
- GPU:
- Others:

Anything else?

No response

i find ,while build_docker_image,REGISTRY: Dict[str, Any] ,is null。

Only supports hub operators and lambda function when using triton backend

Only supports hub operators and lambda function when using triton backend

Can this feature be added?I think this feature is very common in projects. I have found a temporary solution to avoid it. I need to add @ register code to the/usr/local/bin/triton_builder and model.py files

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Close the stale issues and pull requests after 7 days of inactivity. Reopen the issue with /reopen.