mistralai/mistral-inference

Issues

• [BUG: Mamba-Codestral-7B-v0.1 Internal Triton PTX codegen error: Ptx assembly aborted due to errors

  #213 opened 4 months ago by andretisch
  1

• [BUG: AssertionError: Mamba is not installed. Please install it using `pip install mamba-ssm`.

  #192 opened 5 months ago by matbee-eth
  9

• [BUG] Transformer.from_folder() does not load the model on multiple GPU

  #197 opened 5 months ago by Cerrix
  4

• [BUG: pip install mistral_inference: ModuleNotFoundError: No module named 'torch'

  #228 opened 3 months ago by chrisstankevitz
  3

• [BUG: Using the fine-tuned Mistral-7B-v0.1 for inference, when encountering the backslash escape character '', the inference stalls, very slow, but after a few minutes, it continues generating.

  #229 opened 2 months ago by Essence9999
  0

• Pixtral-12B tokenizer error - special_token_policy=IGNORE does not ignore special tokens in decoding

  #227 opened 3 months ago by OmriKaduri
  0

• [BUG: RuntimeError: Boolean value of Tensor with more than one value is ambiguous]

  #225 opened 3 months ago by siwer
  0

• [BUG: Cannot build on Mac M1 Silicion

  #224 opened 3 months ago by timspannzilliz
  0

• [BUG: AttributeError: module 'torch.library' has no attribute 'custom_op'

  #222 opened 3 months ago by mruhlmannGit
  0

• [BUG: RuntimeError: Couldn't instantiate class <class 'mistral_inference.args.TransformerArgs'> using init args dict_keys(['dim', 'n_layers', 'vocab_size', 'model_type'])

  #221 opened 3 months ago by NM5035
  0

• [BUG: TypeError: generate_mamba() takes 2 positional arguments but 3 positional arguments were given

  #220 opened 3 months ago by NM5035
  0

• Q: Why rotary embedding applied only to queries and keys?

  #147 opened 8 months ago by pmgautam
  1

• [BUG] Device error when running on other cuda device than cuda:0

  #215 opened 4 months ago by cornzz
  1

• Suggested improvement of eos logic in generate.py

  #180 opened 4 months ago by vvatter
  2

• JSON response format failing to retrieve clean JSON

  #146 opened 8 months ago by serferdinand2
  3

• [BUG: Could not find consolidated.00.pth or consolidated.safetensors in Mistral model path but mistralai/Mistral-Large-Instruct-2407 surely not contains it

  #205 opened 5 months ago by ShadowTeamCN
  9

• [BUG: ModuleNotFoundError: No module named 'mistral_inference.transformer' ​

  #202 opened 5 months ago by yafangwang9
  6

• [Feat] Add streaming support to Codestral Mamba

  #212 opened 4 months ago by xNul
  0

• [BUG: rate limit exceeded on basic examples

  #210 opened 4 months ago by AlbertoMQ
  0

• [BUG: ImportError: cannot import name 'Transformer' from 'mistral_inference.model' (/usr/local/lib/python3.10/dist-packages/mistral_inference/model.py)

  #206 opened 5 months ago by rabeeqasem
  9

• [BUG: config.json in mamba-codestral-7B-v0.1 is error

  #207 opened 5 months ago by Fly-Pluche
  2

• Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW

  #144 opened 8 months ago by guidoveritone
  2

• [BUG: ModuleNotFoundError: No module named 'triton']

  #200 opened 5 months ago by MaxAkbar
  3

• [BUG: mistralai/mamba-codestral-7B-v0.1 AttributeError: 'Mamba2' object has no attribute 'dconv'

  #196 opened 5 months ago by s-natsubori
  0

• Tokenizer skips the special tokens while decoding

  #162 opened 7 months ago by anandsarth
  2

• Missing the params.json

  #140 opened 9 months ago by littlewwwhite
  3

• !mistral-demo $7B_DIR issue

  #154 opened 7 months ago by shaimaa0000
  3

• mistral-demo $M7B_DIR issue

  #160 opened 7 months ago by chaima-bd
  1

• [BUG: Mistral 7B Instruct Models from Huggingface limited to 4096tokens?

  #182 opened 6 months ago by MaxS3552284
  1

• Did Mistral-7B-Instruct-v0.2 use Sliding Window Attention (SWA)?

  #148 opened 8 months ago by matrixssy
  2

• Question about Mixtral MLP section

  #139 opened 7 months ago by lhallee
  4

• speed up inference?

  #169 opened 7 months ago by xxyp
  0

• Using base model on GPU with no bfloat16

  #163 opened 7 months ago by yichen0104
  1

• PAD token missing ?

  #150 opened 7 months ago by omkar-12bits
  2

• License

  #156 opened 7 months ago by fakerybakery
  2

• os X pip install fail

  #157 opened 7 months ago by edmondja
  1

• num of training tokens?

  #151 opened 7 months ago by wgwang
  0

• How to use a prompt for text analysis?

  #145 opened 8 months ago by rsoika
  6

• AI generates responses or conversation without any human input

  #149 opened 8 months ago by Vishalsub
  0

• I am unable to build the vLLM Container

  #142 opened 9 months ago by AMGI-Pipeline
  4

• Training code

  #138 opened 9 months ago by sartimo
  1

• Not completing answer

  #143 opened 9 months ago by KEYURBODAR
  0

• Fine Tuning Mistral 7b

  #141 opened 9 months ago by nourolive
  3

• CUDA EXTENSION NOT INSTALLED nvcr.io/nvidia/pytorch:22.12-py3

  #136 opened 9 months ago by skr3178
  1

• [MISTRAL AI ERROR] Mistral AI responding with Unexpected role RoleEnum.tool error

  #135 opened 9 months ago by muhammadfaizan027915
  2

• Mistral's tokenizer is not optimized

  #134 opened 9 months ago by Yarflam
  0

• Evaluation Pipeline

  #133 opened 10 months ago by nikhil0360
  0

• Friendly Reminder while Generating the output

  #132 opened 10 months ago by BadrinathMJ
  0

• [Mistral 7B mistral-7b-instruct-v0.1.Q8_0.gguf] Wrong text "quoted" while presented as real

  #131 opened 10 months ago by SINAPSA-IC
  3

• "evaluation pipeline" public?

  #130 opened 10 months ago by kijlk
  0