aws/sagemaker-inference-toolkit

Issues

• SageMaker inference should be able to run as non-root user.

  #72 opened 4 years ago by akulk314
  3

• Server Timeout Unit is Minutes in MMS, but docstring says Seconds

  #99 opened 7 months ago by kastman
  2

• Be able to change SageMaker endpoint log level

  #70 opened 4 years ago by oonisim
  12

• Is streaming supported?

  #137 opened 9 months ago by inf3rnus
  0

• Warning: "Calling MMS with mxnet-model-server. Please move to multi-model-server."

  #69 opened 4 years ago by mb-dev
  2

• Config parsing can be improved

  #16 opened 5 years ago by ericangelokim
  5

• Add support for downloading inference script and code (entry_point) from s3

  #136 opened a year ago by urirosenberg
  0

• Default output function encodes results to JSON and that seems to add to response latency.

  #84 opened 4 years ago by vdantu
  1

• Triton container documentation implies py38

  #134 opened a year ago by david-waterworth
  0

• psutil 5.9.6 seems to be throwing ZombieProcess when retrieving the mms process

  #132 opened a year ago by charlietruong-wk
  5

• Enhancing Multi-Model Server Logging Support - JSON Format Integration

  #131 opened a year ago by ViliamSerecun
  0

• Support CodeArtifact repositories for installing Python packages

  #85 opened a year ago by setu4993
  4

• End-to-End example for *NON* multi model deployment

  #53 opened 5 years ago by nvs-abhilash
  7

• Tensorflow Inference toolkit

  #128 opened 2 years ago by vincentvic
  0

• Support for parquet encoder and decoder

  #127 opened 2 years ago by lorenzwalthert
  0

• OOM errors creating an endpoint for LLMs

  #124 opened 2 years ago by jsleight
  1

• Support user-defined batch inference logic

  #123 opened 2 years ago by dgcnz
  0

• Document default_pre_model_fn and default_model_warmup_fn

  #117 opened 2 years ago by l3ku
  0

• stop_server function

  #116 opened 2 years ago by Duncan-Haywood
  0

• Launch MMS without repackaging model contents

  #102 opened 2 years ago by fm1ch4
  0

• Custom `model_fn` function not found when extending the PyTorch inference container

  #86 opened 4 years ago by e13h
  0

• Long Model Loading times in Multimodel Server

  #113 opened 2 years ago by AlexRaschl
  0

• Support CustomAttributes in script mode (for PyTorch, HuggingFace, etc)

  #110 opened 2 years ago by athewsey
  0

• Add (and test) Support for Python 3.8

  #107 opened 3 years ago by ddluke
  0

• Include "Requires-Python" in source distributions

  #106 opened 3 years ago by ddluke
  0

• Issue attaching eia device for huggingface transformers roBERTa model

  #100 opened 3 years ago by kjhenner
  0

• Model Server fails to start with multi-model-server version 1.1.5

  #94 opened 3 years ago by joseproura
  1

• Deploying multiple model artifacts, each having their own inference handler

  #55 opened 5 years ago by dean-cpi
  5

• JVM detect the CPU count as 1 when more CPUs are available for the container.

  #82 opened 3 years ago by amaharek
  2

• How to overwrite batch transform output in S3

  #68 opened 4 years ago by BaoshengHeTR
  5

• In a multi-model scenario pass model name as argument to input_fn() and output_fn()

  #77 opened 4 years ago by RajeshRamchander
  0

• Documentation for creating multi model end point for unsupported algorithms is not clear

  #78 opened 4 years ago by raj-vuram
  0

• Allow model_fn to get more arguments

  #65 opened 4 years ago by ehsanmok
  2

• Enhance UX for inference

  #64 opened 4 years ago by ehsanmok
  0

• logging in JSON format

  #59 opened 4 years ago by alext234
  2

• Local development WorkerLifeCycle skip

  #41 opened 5 years ago by danielmapar
  1

• Cloudwatch does not log stack trace when error occurs during invocation.

  #50 opened 4 years ago by Olamyy
  3

• Controlling multiple copies of same model within Sagemaker multi-model-server

  #40 opened 4 years ago by Kuntal-G
  3

• Request encoding function for application/jsonlines output format

  #56 opened 5 years ago by BaoshengHeTR
  5

• Ability to define custom log4j properties

  #52 opened 5 years ago by arvarik
  1

• Update README example

  #46 opened 5 years ago by ajaykarpur
  0

• Error in batch transform with custom image

  #9 opened 5 years ago by velociraptor111
  15

• model-server archives model in /opt/ml/model

  #17 opened 5 years ago by ericangelokim
  0

• UTF8 Decoding depends on customer content type

  #10 opened 5 years ago by ericangelokim
  2