[Bug]: When building the triton backend, using custom operators may result in errors
bug400400 opened this issue · 4 comments
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
@register(name='embedding')
class Embedding(NNOperator):
def init(self):
def __call__(self, object, type, model):
return vec
embedding_pipe = (
pipe.input('object')
.map('object', 'vec', ops.embedding())
.output('vec')
)
towhee.build_docker_image(
dc_pipeline=embedding_pipe,
image_name='triton_embedding:v1.0',
cuda_version='11.7',
format_priority=['onnx'],
parallelism=4,
inference_server='triton'
)
but it will go to the hub to download the operator
Expected Behavior
build success
Steps To Reproduce
@register(name='embedding')
class Embedding(NNOperator):
def __init__(self):
####
def __call__(self, object, type, model):
####
return vec
embedding_pipe = (
pipe.input('object')
.map('object', 'vec', ops.embedding())
.output('vec')
)
towhee.build_docker_image(
dc_pipeline=embedding_pipe,
image_name='triton_embedding:v1.0',
cuda_version='11.7',
format_priority=['onnx'],
parallelism=4,
inference_server='triton'
)
but it will go to the hub to download the operator
Environment
- Towhee version(e.g. v0.1.3 or 8b23a93):
- OS(Ubuntu or CentOS): Ubuntu
- CPU/Memory: CPU
- GPU:
- Others:
Anything else?
No response
i find ,while build_docker_image,REGISTRY: Dict[str, Any] ,is null。
Only supports hub operators and lambda function when using triton backend
Only supports hub operators and lambda function when using triton backend
Can this feature be added?I think this feature is very common in projects. I have found a temporary solution to avoid it. I need to add @ register code to the/usr/local/bin/triton_builder and model.py files
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Close the stale issues and pull requests after 7 days of inactivity. Reopen the issue with /reopen
.