pytorch/serve

Issues

• ROCm support missing from Dockerfile

  #3380 opened 4 days ago by jakki-amd
  0

• Clarify interactions between TorchServe and KServe

  #3378 opened 16 days ago by kimminw00
  0

• Getting started guide client samples broken ?

  #3348 opened 3 months ago by nikste
  1

• Fix missing system metrics for Apple M1

  #3376 opened 21 days ago by jakki-amd
  0

• Add Detectron2 Support to TorchServe Object Detection Examples

  #3344 opened 3 months ago by Mudassar-MLE
  3

• 503 InternalServerException, prediction failed

  #3375 opened 25 days ago by Jax29
  1

• Torchserve-kfs docker images for other python version (3.10+)

  #3374 opened a month ago by cjidboon94
  1

• vLLM ZeroDivisionError

  #3373 opened a month ago by mfahadyousaf
  0

• Incorrect Metric Type for HPA Scaling

  #3286 opened 5 months ago by liaddrori1
  2

• Quickstart vLLM examples do not work as expected

  #3365 opened 2 months ago by prashant-warrier-echelonvi
  0

• Unable to build frontend after following the documentation

  #3364 opened 2 months ago by mihaidusmanu
  0

• Trying to find a doc explaining how the scaling works (min_worker to max_worker)

  #3362 opened 2 months ago by lschaupp
  0

• Issue when sending parallel requests

  #3361 opened 2 months ago by lschaupp
  0

• `weights_only` default flip for `torch.load`

  #3360 opened 2 months ago by mikaylagawarecki
  0

• Low Throughput and High Latency with TorchServe Deployment on AWS

  #3359 opened 2 months ago by dummyuser-123
  0

• Model & Instance scaling

  #3358 opened 2 months ago by markcNewell
  0

• 413 Request Entity Too Large

  #3357 opened 2 months ago by pengxin233
  0

• GPU not detected inside torchserve docker container

  #3352 opened 3 months ago by dummyuser-123
  1

• Allow to provide custom model_service_worker.py

  #3353 opened 3 months ago by racinmat
  0

• Make TorchServe usable without torch installed

  #3350 opened 3 months ago by racinmat
  0

• Torchserve not starting for diffusers example

  #3345 opened 3 months ago by dummyuser-123
  10

• Error in MetricCollector when starting pytorch/torchserve:0.12.0-gpu container

  #3349 opened 3 months ago by Hspix
  0

• "TorchServe Fails to Reload Models from S3 After k8s Pod Restart"

  #3324 opened 4 months ago by koscevicb
  1

• io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer

  #3279 opened 3 months ago by KD1994
  7

• throughput increase non-linearly with number of workers

  #3338 opened 3 months ago by vandesa003
  2

• Respond status 429 Too Many Requests instead of broad Internal Server Error

  #3342 opened 3 months ago by tysg
  2

• UnboundLocalError: local variable 'model_snapshot_path' referenced before assignment

  #3335 opened 3 months ago by johnathanchiu
  0

• Clarification on minWorkers and maxWorkers parameters

  #3339 opened 3 months ago by krzwaraksa
  0

• Kserve MNIST CI failure

  #3308 opened 4 months ago by maaquib
  1

• Allow .tar.gz models to be loaded using load_models=all

  #3320 opened 4 months ago by m10an
  2

• Kserve management api for registering new models

  #3325 opened 4 months ago by matej14086
  0

• Examples for authorization model

  #3260 opened 6 months ago by RyanKadri
  10

• CI: missing security check for security issues in the codebase

  #3311 opened 4 months ago by ChengyuZhu6
  1

• Kserve TorchserveModel can't handle torch's auth

  #3301 opened 4 months ago by AntPeixe
  0

• when to use ArgumentParser, raise "unrecognized arguments: --sock-type unix --sock-name /tmp/.ts.sock.9000"

  #3299 opened 5 months ago by james-joobs
  3

• model_yaml_config usage is not explained well enough

  #3290 opened 5 months ago by Foundsheep
  1

• integrating the Torch Serve hosted model with a third party application

  #3296 opened 5 months ago by tarunsk1998
  1

• Ability to return Pydantic Models

  #3255 opened 6 months ago by mhashas
  1

• Startup timeout should be configurable separately

  #3261 opened 5 months ago by Isalia20
  5

• Model handler response structure enforced by ts.service.Service class but should be defined by model

  #3284 opened 5 months ago by stf976
  1

• Websocket Support

  #3252 opened 5 months ago by tiefucai
  3

• Support for a CUDA 12.5.x

  #3278 opened 5 months ago by jasonsmithio
  1

• StreamPredictions2 gPRC method execution produces a server error while changing from version *0.10.0* to *0.11.1*

  #3264 opened 6 months ago by ferugit
  1

• Problem when run custom Docker image with GPU

  #3258 opened 6 months ago by hungtrieu07
  1

• GPU memory not released after inference

  #3253 opened 6 months ago by Di-Gu
  1

• TorchServe docker/kserve nightly image size increase

  #3244 opened 6 months ago by agunapal
  1

• Docker regression tests must include check for docker image size increase

  #3248 opened 6 months ago by agunapal
  0

• TorchServe docker image with vllm, trt-llm dependencies

  #3247 opened 6 months ago by agunapal
  0

• Alarming log(false alarm) about xpu-smi

  #3236 opened 6 months ago by agunapal
  0

• misaligned argument between `args.disable_token` and `--disable_token-auth` in llm_launcher.py

  #3229 opened 6 months ago by jinyichao
  2