Lightning-AI/LitServe

Issues

• Improve the debugging experience of LitServe

  #344 opened 12 days ago by tchaton
  1

• how to use Litserve to serve vllm engine

  #382 opened 18 days ago by GhostXu11
  2

• [Windows] OSError when multiple workers

  #384 opened 17 days ago by FrsECM
  0

• `max_tokens` in OpenAI spec is deprecated

  #373 opened 19 days ago by zcasanova
  0

• LitServe generates a `client.py` file at the current directory

  #227 opened 4 months ago by laclouis5
  7

• [Windows] Server Hangs while closing

  #372 opened 23 days ago by FrsECM
  2

• Info route

  #366 opened 24 days ago by lorenzomassimiani
  2

• Embedding model support with openai spec

  #305 opened 25 days ago by riyajatar37003
  15

• Support additional content types in post requests

  #362 opened a month ago by ktrapeznikov
  1

• Raising HTTPexception on single request from encode_request or decode_request raise same for all request in batch

  #340 opened 2 months ago by bimarshak7
  3

• terminate early if accelerator is missing

  #292 opened a month ago by Borda
  1

• Default Input-Output schema

  #354 opened a month ago by aniketmaurya
  0

• Is it possible to support multiple endpoints for one server?

  #271 opened 4 months ago by arkohut
  17

• Add custom path option for healthcheck

  #341 opened a month ago by akuma12
  2

• Support for async requests or webhooks

  #348 opened a month ago by brian316
  3

• bug: OpenAI-Spec should send text/event-stream

  #359 opened a month ago by fabigr8
  6

• How to route /docs path in litserve behind a proxy?

  #352 opened 2 months ago by Mayurji
  6

• LitServe server doesn't crash when there is a failure in setup

  #347 opened a month ago by tchaton
  3

• GET Request

  #355 opened a month ago by gururaser
  5

• Can't start the demo example without bug

  #350 opened 2 months ago by anonym-ai
  3

• Prometheus logger is not pickable + monitoring metrics set via self.log are not tracked

  #339 opened 2 months ago by miguelalba96
  5

• Add dockerize example in the README

  #345 opened 2 months ago by tchaton
  2

• Litserve entrypoint wasn't properly registered on first install in Studios

  #343 opened 2 months ago by tchaton
  1

• How to load models from S3

  #329 opened 2 months ago by AbhishekBose
  6

• Example in documentation on how to setup an OpenAI-spec API with LlamaIndex-RAG

  #286 opened 3 months ago by PierreMesure
  4

• Websocket Support for Streaming Input and Output

  #320 opened 3 months ago by ChenghaoMou
  4

• More complex model management (multiple models, model reloading etc...)

  #282 opened 3 months ago by bsergean
  2

• Add Keep-Alive Functionality for GPU Resource Optimization in LitServe

  #304 opened 3 months ago by skyking363
  12

• Benchmarking gRPC with LitServe – Surprising Results

  #323 opened 3 months ago by deependujha
  1

• May I ask if it is possible to deploy complex pipelines such as ControlNet and IP Adapter in Stable Diffusion? Do you have any examples

  #236 opened 3 months ago by libai-lab
  5

• CUDA Multiprocessing Issue

  #321 opened 3 months ago by floschne
  2

• Pyright issues [Name mismatches]

  #310 opened 3 months ago by grumpyp
  2

• dry run after server started

  #293 opened 3 months ago by Borda
  1

• Custom HTTPException Not Working as Expected

  #301 opened 3 months ago by randydl
  3

• unexpected output for HF model with matching

  #294 opened 3 months ago by Borda
  5

• Handle case when Logger.process is stuck

  #289 opened 3 months ago by aniketmaurya
  0

• Feature Request: Customize FastAPI Metadata

  #270 opened 4 months ago by bhimrazy
  5

• Can't appear to load the server

  #273 opened 3 months ago by B-M-S-West
  2

• Supporting query params alongside file upload

  #272 opened 3 months ago by energydrink9
  4

• Add Support to Huggingface Diffusers!

  #275 opened 3 months ago by KaifAhmad1
  2

• Placeholder storage has not been allocated on MPS device

  #239 opened 3 months ago by candlep
  6

• Setup step is not awaited

  #263 opened 3 months ago by andreieuganox
  2

• If `workers_per_device` is set to 1, how can I set a model instance to be able to concurrency?

  #267 opened 4 months ago by ALEXuH
  8

• The current openai spec is only compatible with version openai==0.28.0 and cannot recognize newer versions

  #245 opened 4 months ago by GhostXu11
  2

• Feature: Pass custom middlewares to the app

  #228 opened 4 months ago by lorenzomassimiani
  5

• Replace uvicorn server with gunicorn

  #243 opened 4 months ago by vrdn-23
  1

• Create Makefile to run common development tasks locally

  #237 opened 4 months ago by AdolfoVillalobos
  2

• client.py and fastapi /docs "try it out" both are not working with BERT example

  #230 opened 4 months ago by sebastianschramm
  2

• Decorator design for `LitServer`

  #218 opened 4 months ago by baggiponte
  1

• Add support for `response_format` in OpenAI Spec

  #196 opened 4 months ago by bhimrazy
  2