sgl-project/sglang

Issues

• [BUG] srt throws KeyError when sgl.gen(...) regex parameter contains Chinese characters

  #377 opened 2 months ago by m0g1cian
  3

• Type of self.variables

  #455 opened 21 days ago by ChuyueSun
  0

• I can't use the OpenAI endpoint with images?

  #443 opened a month ago by vedantroy
  1

• DBRX not working

  #454 opened 21 days ago by Ying1123
  0

• /generate request possibly hanging when `CUDA out of memory` is thrown

  #435 opened a month ago by Gintasz
  3

• Trace OpenAI backend usage

  #453 opened 22 days ago by Ying1123
  0

• Refactor the openai speculative execution module in interpreter

  #452 opened 22 days ago by Ying1123
  0

• Regex generation causes 37x lower performance

  #450 opened 23 days ago by Gintasz
  1

• OOM CUDA error on 8 * L4 machine when launching sglang server

  #445 opened 25 days ago by mounamokaddem
  2

• Does sglang do automatic batching?

  #444 opened 25 days ago by vedantroy
  2

• How do I get logprobs

  #442 opened a month ago by tom-doerr
  1

• Llama-3 regex generation can get stuck in infinite generation beyond max_tokens and crash server (reproduction example)

  #414 opened a month ago by Gintasz
  2

• Support for multimodal models

  #421 opened a month ago by babla9
  0

• vLLM import error

  #391 opened 2 months ago by jlin816
  5

• Use `__all__` to explictly control the APIs.

  #386 opened a month ago by hnyls2002
  0

• Please add Phi3 support

  #407 opened a month ago by Curiosity007
  2

• run python3 test_httpserver_llava.py get ValueError: 64002 is not in list

  #413 opened a month ago by Aurorana
  0

• Chat role prefixes for SRT backend broken in latest

  #410 opened a month ago by qeternity
  3

• Support InternVL 1.5

  #398 opened 2 months ago by themrzmaster
  1

• Logprobs are almost the same for all choices

  #388 opened 2 months ago by tom-doerr
  8

• LLaVA-v1.6 RuntimeError in llava image encoding

  #409 opened a month ago by lukashelff
  0

• Switch to non gated models

  #387 opened 2 months ago by tom-doerr
  1

• Choices functionality breaking with images

  #408 opened a month ago by dexius-ram-depop
  0

• Import Errors occurring even when dependencies are installed

  #403 opened a month ago by david-vectorflow
  2

• Launch server using local LLaVA checkpoints

  #397 opened a month ago by lukashelff
  2

• [Question] offload kv cache to disk/database ?

  #352 opened a month ago by s7ev3n
  0

• Repetitive zeros for structuring JSON (float type)

  #405 opened a month ago by timothylimyl
  0

• no batch run when using openai's format for calling.

  #404 opened a month ago by xjw00654
  0

• How does RadixAttention implements multi-head/multi-query/grouped-query attention.

  #402 opened a month ago by Griffintaur
  0

• Is it possible to define the prompts for KV caching up-front?

  #401 opened a month ago by timothylimyl
  1

• Support Datetime in JSON mode

  #400 opened a month ago by timothylimyl
  1

• Don't get API response when sending images

  #357 opened 2 months ago by tom-doerr
  1

• with default setting, LoRA has no improvement is normal? 默认设置下，LoRA 完全没效果正常吗？

  #396 opened a month ago by fisher75
  0

• Loading a BNB 4 bit model + adapter

  #374 opened 2 months ago by timothelaborie
  1

• Prefill out of memory occur when deployed with servers

  #393 opened 2 months ago by for-just-we
  3

• Regenerate benchmark results for latest vLLM

  #389 opened 2 months ago by nilesh-c
  1

• ImportError: cannot import name 'function' from partially initialized module 'sglang'

  #384 opened 2 months ago by lambda7xx
  0

• compatibility issues and memory leak problems --enable-flashinfer

  #356 opened 2 months ago by pj-ml
  3

• ImportError: cannot import name 'get_cuda_stream' from 'triton.runtime.jit' In triton-nightly(V100)

  #383 opened 2 months ago by nenomigami
  0

• VLLM version

  #373 opened 2 months ago by eaubin
  3

• How does or does sglang support multiple completions / samples given the same prompt?

  #379 opened 2 months ago by wenting-zhao
  0

• Supported LLaVa models 请问还支持哪些LLaVa的model？Lora后的模型如何批量推理？

  #366 opened 2 months ago by fisher75
  0

• Loading Chat Template in a more flexible way?

  #376 opened 2 months ago by for-just-we
  0

• JSON decoding result don't match regex

  #371 opened 2 months ago by DouHappy
  0

• System message not supported for Anthropic (anthropic.BadRequestError)

  #367 opened 2 months ago by fronx
  2

• Inference does not work in sglang with regex?

  #361 opened 2 months ago by randomcodelookup
  0

• Fails with latest vllm

  #362 opened 2 months ago by benglard
  1

• About parameter `max_tokens`

  #360 opened 2 months ago by for-just-we
  3

• How do I instantiate multiple sgl backends?

  #359 opened 2 months ago by accupham
  2

• Beam Search Support

  #353 opened 2 months ago by LiquidGunay
  0