mlcommons/inference

Issues

• An error occurred when using `mlcr` to conduct a BERT benchmark test.

  #2158 opened 20 days ago by huagongdiyitupo
  0

• Seems not support running mlc inferenece on multiple GPUs?

  #2157 opened 20 days ago by Bob123Yang
  0

• [SDXL] v4.1 nvidia automated benchmark issue

  #2154 opened a month ago by Xi0131
  0

• Problem with run GPTJ-99

  #2146 opened a month ago by Agalakdak
  0

• Inference 5.0 submission failure for Llama3.1-405b caused by hardcode in submission checker

  #2142 opened a month ago by zhonglidaniel
  1

• Fail to build the docker for mlc command on Ubuntu 22.04

  #2132 opened a month ago by Bob123Yang
  7

• Refactor TEST01 message so it's less confusing

  #2138 opened a month ago by nvzhihanj
  0

• RGAT minor issues , listed in description

  #2126 opened a month ago by mkankana
  2

• Llama3.1-405B non-zero temperature

  #2116 opened 2 months ago by psyhtest
  4

• Resnet50 MLPerf 4.1 Attribute Error: ResNet50EngineBuilder object has no attribute calibrator

  #2129 opened 2 months ago by SomeGuyFarAway
  0

• Error in RsNet50 server run

  #2128 opened 2 months ago by KathpaliaChirag
  0

• LLAMA3_1-405B-99 Docker Cmind not found issues

  #2105 opened 2 months ago by hhuo24pm
  12

• Question: Benchmark results.

  #2122 opened 2 months ago by Agalakdak
  2

• [URGENT] Submission UI error while running generate_final_report.py

  #2121 opened 2 months ago by taran2210
  0

• [GNN] Acc eval script change: incompatibility or corrupted file?

  #2119 opened 2 months ago by attafosu
  3

• Is beam search strategy mandatory in gptj-6b benchmark?

  #2107 opened 2 months ago by phoenixysj
  1

• Offline scenario Crashed for GPT-J (Failed to create file system for "mlc-inference:mlcommons-inference-wg-public/gpt-j": didn't find section in config file)

  #2102 opened 2 months ago by KathpaliaChirag
  1

• Error : Please envoke mlcr "get cudnn" --tar_file={full path to the cuDNN tar file}

  #2106 opened 2 months ago by kingjuno
  2

• Is beam search strategy mandatory in gptj-6b benchmark?

  #2108 opened 2 months ago by phoenixysj
  0

• RuntimeError: list index out of range on RetinaNet

  #2103 opened 2 months ago by kingjuno
  1

• Launch issue with Medical Imaging using 3d-unet

  #2095 opened 2 months ago by Agalakdak
  2

• SingleStream Scenario crashes

  #2101 opened 2 months ago by kingjuno
  3

• SingleStream scenario crashes for Resnet50

  #2100 opened 2 months ago by nintendoit
  2

• Inference [4.1.5] Inference nvidia-smi command exits but failed to execute

  #2098 opened 2 months ago by msalaz03
  0

• Create MLCommons Croissant Metadata for the Datasets created by MLCommons Inference WG

  #2076 opened 2 months ago by arjunsuresh
  0

• Official Readme files are having Huggingface Download Links for the Llama models

  #2075 opened 2 months ago by arjunsuresh
  0

• Release mlcommons inference submission tools as a Python package

  #2074 opened 2 months ago by arjunsuresh
  0

• Need to fix TEST01 performance check

  #2072 opened 2 months ago by arjunsuresh
  0

• Getting 401 unauthorised error white using interface of api for text to image generator

  #2065 opened 2 months ago by pawan999333
  3

• sample_concatenate_permutation ineffective in user.conf

  #2028 opened 2 months ago by psyhtest
  13

• Add code, measurement and results links in the result summary sheet

  #2038 opened 2 months ago by arjunsuresh
  1

• [405b-SUT] Max number of output tokens

  #2060 opened 2 months ago by attafosu
  1

• Default behaviour of torch.load changed in v2.6 breaking some of the reference pytorch implementations

  #2056 opened 2 months ago by arjunsuresh
  0

• The documents are outdated

  #2048 opened 3 months ago by MohdShariq91
  1

• R-GAT Offline Test failed with file not found in the dataset directory

  #2043 opened 3 months ago by dineshchitlangia
  4

• CM script not recognizing custom nvmitten wheel on ARM CPU + Nvidia GPU setup

  #1991 opened 4 months ago by ChrisHuang96
  20

• Run command issue

  #2027 opened 3 months ago by Xi0131
  1

• requirements.txt script missing in mixtral-8x7b standalone script

  #2040 opened 3 months ago by mrmhodak
  1

• R-GAT reference implementation is giving the below warning related to numpy array being non-writable.

  #2007 opened 3 months ago by arjunsuresh
  4

• mixtral-8x7b: Reference Implementation Accuracy Failure on H200

  #2018 opened 3 months ago by mrmhodak
  8

• GPT-J Huggingface validation error

  #2010 opened 3 months ago by Xi0131
  3

• nvmitten issue

  #2024 opened 3 months ago by Xi0131
  3

• Save the docker with Resnet50 running successfully and load it on another system with the same config but fail to run Resnet50

  #2023 opened 3 months ago by Bob123Yang
  1

• Fail to build the GPT-J docker after successfully installation of tensorrt-llm

  #2022 opened 3 months ago by Bob123Yang
  2

• mixtral accuracy script is failing with nltk==3.9.1

  #2008 opened 3 months ago by arjunsuresh
  0

• Metrics definition and settings used for LLM benchmarks

  #2012 opened 3 months ago by vineel96
  1

• why did the specified data precision not work for MLPerf inference?

  #2006 opened 3 months ago by Bob123Yang
  4

• llama2-70b cmake issue

  #2009 opened 3 months ago by Xi0131
  6

• CM script failed to run harness after docker done

  #1998 opened 3 months ago by Bob123Yang
  2

• Docker container not found

  #1992 opened 4 months ago by Xi0131
  5